Project: Yet Another Latency Measuring Device

Measuring device for end-to-end (click to photon) latency based on cheap off-the-shelf components.

Status: ongoing

Runtime: 2020 -

Participants: Andreas Schmid, Raphael Wimmer

Keywords: latency, end-to-end latency, measuring device, open hardware

This page is work in progress!

Background

Latency is an inherent property of human-computer interfaces. Due to processing and transmission times between input device, operating system, application, and display, each component in this chain adds to the overall delay between user input and system response. This so called end-to-end latency directly influences task difficulty and should therefore be minimized to ensure optimal user experience. The influence of latency is especially important for real time applications, as it can decide about virtual life and death in video games or influence the outcome of psychological experiments.

To build and test low latency systems it is important to have a method for measuring a system's latency. As there are numerous different input and output modalities for human-computer interfaces, there is no general one size fits all method for measuring end-to-end latency. Therefore, measuring methods have to be tailored specifically for the system under test and with regards to further aspects such as accuracy, automation, replicability, and access system components.

A straight forward approach for measuring end-to-end latency is to use a high speed camera to record input and output device and calculate the delay time by either counting frames between two events. However, annotating video footage is a very time consuming task and the accuracy of the measurement is limited by the used camera's refresh rate. Affordable high speed cameras (for example included in modern smartphones) can capture up to 240 frames per second leading to a temporal resolution of 4 milliseconds, which is not precise enough for reliable latency measurements.

Therefore, automatic methods based on microcontrollers are preferrable in many cases.

Goals

Our goal with Yet Another Latency Measuring Device is to provide a method to accurately measure end-to-end latency which can be easily replicated by anyone interested, such as researchers, gamers and hobbyists. Our method exclusively uses cheap and off-the-shelf components so the total cost is under 25€ in total. As a microcontroller is used to trigger input events and sense the system's response, the device is far more accurate than camera-based methods. Manual annotation of video footage is not necessary - the device conducts measurements automatically. This way, measurements series' can be performed in order to report latency distributions instead of only one latency value.

Method

Components

The latency measuring device is based on the broadly available Arduino Micro microcontroller which controls the measuring process, measures the time between input and output and sends results to a PC via USB. An optocoupler is used to close the connection of a button on an input device so an input event triggered. A photo resistor attached to the monitor is used to sense a change in brightness on the display. Additionally, two LEDs indicate the state of the device so it can be validated using a high speed camera.

The measuring device can be seen in Figure ??. The button contacts of the modified input device are connected to the transistor side of an optocoupler (a). The photo resistor is connected to an ADC pin of the microcontroller via a voltage divider circuit (b). Two LEDs indicate the time when a button press is triggered (c, left) and when a change in brightness is detected (c, right). A pyhsical switch (d) can be used to manually start and stop the measurement.

The circuit is kept simple intentionally so it can be replicated easily. Our implementation is soldered to a prototyping board, but the device can also be built on a breadboard so no soldering is required.

Figure ??. Close up view of the measuring device.


Measuring Process

The measuring setup can be seen in Figure ??. A microcontroller (a) starts a timer and triggers a button press on a modified input device (b) by closing a button contact with an optocoupler. The application reacts to the button press by changing its background color from black to white. The microcontroller measures the display's brightness with a photo resistor (c) attached to the monitor. Once a change in brightness is recognized, the timer is stopped. Results sent to the PC via USB.

The measuring process is depicted in Figure ??. (1) The measurement starts. (2) The mouse button is triggered and the green LED is turned on. (3) The input event is recognized by the PC and the screen changes its color from black to white. (4) Once the photo sensor attached to the display measures a change in brightness, the blue LED turns on. This way, the results of our device can be compared to footage from a high speed camera.

Figure ??. Setup for measuring end-to-end latency.


Figure ??. Step by step visualization of the measuring process.


Validation

We validated the latency measuring device by comparing its results to annotated footage of a high speed camera (Google Pixel 3a smartphone with 240 fps, Figure ??).

The results of our measuring device match the latencies determined with annotated video footage with a error margin of below 4 milliseconds, which is the high speed camera's temporal resolution. Results can be seen in Figure ??.

Figure ??. Setup for capturing the measuring process with a high speed camera.


Figure ??. Results of the comparison of our device and annotated slow motion footage.


Limitations

Replicating the Device

Resources