Project: DIY Approach to Measuring Display Reaction Time

Easily replicable device for measuring the reaction time of displays.

Status: ongoing

Runtime: 2019 -

Participants: Patrick Stadler, Andreas Schmid, Raphael Wimmer

Keywords: latency, display, hardware, measurement

Background

End-to-end latency, the time between user input and system output, is an important factor in human-computer interaction as it directly influences the user's task perfomance. It is composed of several partial latencies which have to be understood (and measured) individually to identify bottlenecks and reduce overall latency.

Schematic illustration of the measuring setup.

Goals

We already measured input device latency of several mice, keyboards and game pads in our LagBox project. We now extend this approach with a device which is capable of measuring display latency (also known as response time). Similarly to LagBox, only cheap and broadly available components have been used so the device can be easily replicated by anyone with basic knowledge of electronics and Linux.

We share the results of our measurements publicly on this website and plan to offer a platform for everybody who rebuilt the device to contribute their measurements to our database.

The circuitry of the device, as well as a guide for assembly, will also be published on this website in the near future. The source code of the measuring program can be found on GitHub: Code for Microcontroller, Raspberry Pi program.

The display latency measuring device.

Status

In his bachelor's thesis, Patrick Stadler built the prototype of a device to measure display latency. It is based on broadly available hardware components so it can be replicated easily. We measured the latencies of several displays that were available in our lab. The measurements have been validated by comparing the results to measurements with an oscilloscope. Additionally, we tested each device with the commercially available Video Signal Input Lag Tester by Leo Bodnar to further validate our results.

For now, only refresh rates up to 60 Hz are supported and only displays capable of HDMI can be tested.

Future Work

extend the pool of measured displays
use a more powerful microcontroller to increase the temporal resolution of the measurements
provide a guide for assembling the device
provide a guide for using the device
offer a platform for others to share their measurement results

Method

Our measurement process follows the guidelines described in chapter 10.3 of the Information Display Measuring Standard (IDMS) by the Society of Information Displays (SID). Therefore, we define display latency as the time differene from the HDMI signal leaving the video source until the pixels on the diplay have reached 50% brightness.

Mesuring Process

A video source changes the color of the display under test from black to white and triggers the start of a timer on a microcontroller. The timer is stopped when the microcontroller detects a brightness of over 50% with a photo sensor attached to the center of the display. This 50% threshold is determined during a calibration phase before the actual measurements. During the measuring process, the microcontroller logs the brightness value combined with a time stamp and sends those values to a host device after each measurement series. Multiple measurements are performed automatically to detect latency variance and possible outliers.

Model of the measuring process.

Components

To ensure easy replicability of our device, we focused on using cheap and broadly available components. A Raspberry Pi 4 is used to render the user interface of the measuring software and the black and white images used for the measurements. We decided to use the Raspberry Pi because of its low acquisition cost, the broad availablity, and the comparatively well documented and open source graphics stack.

The Raspberry Pi is attached to a custom HAT with an STM8 microcontroller; communication to the microcontoller takes place via UART. The STM8 performs the actual measurement by reading the value of a Osram BPW 34 photo diode through its built-in 10 bit ADC. The photo diode is enclosed in a separate 3d printed case so it can be easily placed on the screen and fixed in position with a velcro belt. Additionally, the case shields the sensor from ambient light to reduce disturbances caused by external influences. The probe is connected to the Raspberry Pi HAT with three wires for supply voltage, ground, and signal.

The latencies introduced by the photo diode and the ADC are in the range of microseconds, thus they do not confound the measurements in a significant way.

The total cost of the device is around 65€ (price includes Raspberry Pi 4), making it affordable for non-professionals. Furthermore, we estimate the time a layperson needs to assemble the device to be under one day.

Opened case of the device. It consists of a sensor probe (left), a microcontroller and a single board computer (right).

Conducting a measurement

Connect the display under test to the measuring device via HDMI.
Dial in default settings and full brightness on the display.
Attach the sensor probe to the center of the display and fasten it with a velcro belt.
Power up the measuring device and wait until the main menu is shown.
Dial in the desired test mode and measurement ID with the control knob on the measuring device.
Start the calibration process.
Start the measurements.
To retrieve the result data, connect a USB memory stick to the measuring device and the data (in CSV format) will be transferred automatically.

Results

We measured the latencies of several monitors found in our lab. Each measurement series consists of 190 measurements so we can report a distribution of latencies for each device (right). The visualisation also includes the latency reported by the Leo Bodnar Video Signal Input Lag Tester, as well as an IDMS test setup, both indicated by vertical lines.

For each individual measurement, a brightness value is captured 10 times per millisecond over a duration of 30 ms. This way, we can plot a brightness curve for each display which also contains information on how long the pixels take to reach full brightness (left). The 50% brightness values determined during calibration before each measurement is indicated by a horizontal bar.

All enhancement features of the displays, like overdrive, were turned disabled for the measurements. For devices that offer such enhancement features, we performed a separate measurement series with overdrive enabled (results are shown in orange).

Supplementary Material

Work in progress!

Publications

LagBox – Measuring the Latency of USB-Connected Input Devices.

Florian Bockes, Raphael Wimmer, Andreas Schmid

CHI EA '18 Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems

Development of a tool for measuring latency of different USB devices (Tweet this with link)

On the Latency of USB-Connected Input Devices

Raphael Wimmer, Andreas Schmid, Florian Bockes

Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19)

Summary of our current understanding of latency and our approach of measuring the latency of USB-connected devices. (Tweet this with link)

DispLagBox: Simple and Replicable High-Precision Measurements of Display Latency

Patrick Stadler, Andreas Schmid, Raphael Wimmer

Proceedings of the Conference on Mensch und Computer 2020

Easily replicable device for measuring the reaction time of displays. (Tweet this with link)

Yet Another Latency Measuring Device

Andreas Schmid, Raphael Wimmer

Workshop on Esports and High Performance HCI 2021

Measuring device for end-to-end latency based on cheap off-the-shelf components. (Tweet this with link)

Measuring the Latency of Graphics Frameworks on X11-Based Systems

Andreas Schmid, Raphael Wimmer

Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

We implemented and evaluated a new method for measuring the latency added by different graphics frameworks and toolkits on X11-based systems. (Tweet this with link)