---- datatemplateentry project ----
template            : projects:display_template
title               : DIY Approach to Measuring Display Reaction Time
shorttitle          : display_latency
participants_       : Patrick Stadler, [[people:andreas_schmid|Andreas Schmid]], [[people:raphael_wimmer|Raphael Wimmer]]
keywords_           : latency, display, hardware, measurement
status_             : ongoing
start               : 2019
end                 : 
photo_img           : projects:display_latency:setup_website.png
shortdescription    : Easily replicable device for measuring the reaction time of displays.
abstract            : Easily replicable device for measuring the reaction time of displays.
---- 

==== Background ====

End-to-end latency, the time between user input and system output, is an important factor in human-computer interaction as it directly influences the user's task perfomance.
It is composed of several partial latencies which have to be understood (and measured) individually to identify bottlenecks and reduce overall latency.
\\
\\
<WRAP center>
[{{:projects:display_latency:circuit.png?1200|Schematic illustration of the measuring setup.}}]
</WRAP>

==== Goals ====

<WRAP group>
<WRAP column left 60%>

We already measured input device latency of several mice, keyboards and game pads in our [[projects:latency|LagBox]] project.
We now extend this approach with a device which is capable of measuring display latency (also known as response time).
Similarly to LagBox, only cheap and broadly available components have been used so the device can be easily replicated by anyone with basic knowledge of electronics and Linux.

We share the results of our measurements publicly on this website and plan to offer a platform for everybody who rebuilt the device to contribute their measurements to our database.

The circuitry of the device, as well as a guide for assembly, will also be published on this website in the near future.
The source code of the measuring program can be found on GitHub: [Code for Microcontroller](https://github.com/pstadler1990/latencytest_stm8_iar), [Raspberry Pi program](https://github.com/pstadler1990/latencytest_fbutils).

</WRAP>

<WRAP column right 30%>
[{{:projects:display_latency:device.jpg?400|The display latency measuring device.}}] \\
</WRAP>
</WRAP>

==== Status ====

In his bachelor's thesis, Patrick Stadler built the prototype of a device to measure display latency.
It is based on broadly available hardware components so it can be replicated easily.
We measured the latencies of several displays that were available in our lab.
The measurements have been validated by comparing the results to measurements with an oscilloscope.
Additionally, we tested each device with the commercially available *Video Signal Input Lag Tester* by [[http://www.leobodnar.com/shop/index.php?main_page=product_info&cPath=89&products_id=212|Leo Bodnar]] to further validate our results.

For now, only refresh rates up to 60 Hz are supported and only displays capable of HDMI can be tested.

\\
<WRAP center>
{{ youtube>-uXzbRlFHxU?1000x600 }}
</WRAP>
\\

=== Future Work ===

 * extend the pool of measured displays
 * use a more powerful microcontroller to increase the temporal resolution of the measurements
 * provide a guide for assembling the device
 * provide a guide for using the device
 * offer a platform for others to share their measurement results

==== Method ====

Our measurement process follows the guidelines described in chapter 10.3 of the [[https://www.icdm-sid.org/downloads/idms1.html|Information Display Measuring Standard]] (IDMS) by the Society of Information Displays (SID).
Therefore, we define display latency as the time differene from the HDMI signal leaving the video source until the pixels on the diplay have reached 50% brightness.

=== Mesuring Process ===

<WRAP group>
<WRAP column left 60%>

A video source changes the color of the display under test from black to white and triggers the start of a timer on a microcontroller.
The timer is stopped when the microcontroller detects a brightness of over 50% with a photo sensor attached to the center of the display.
This 50% threshold is determined during a calibration phase before the actual measurements.
During the measuring process, the microcontroller logs the brightness value combined with a time stamp and sends those values to a host device after each measurement series.
Multiple measurements are performed automatically to detect latency variance and possible outliers.

</WRAP>

<WRAP column right 30%>
[{{:projects:display_latency:measurement_model.png?400|Model of the measuring process.}}] \\
</WRAP>
</WRAP>

=== Components ===

<WRAP group>
<WRAP column left 60%>

To ensure easy replicability of our device, we focused on using cheap and broadly available components.
A Raspberry Pi 4 is used to render the user interface of the measuring software and the black and white images used for the measurements.
We decided to use the Raspberry Pi because of its low acquisition cost, the broad availablity, and the comparatively well documented and open source graphics stack.

The Raspberry Pi is attached to a custom HAT with an [[https://www.st.com/content/st_com/en/products/microcontrollers-microprocessors/stm8-8-bit-mcus/stm8s-series.html|STM8 microcontroller]]; communication to the microcontoller takes place via UART.
The STM8 performs the actual measurement by reading the value of a *Osram BPW 34* photo diode through its built-in 10 bit ADC.
The photo diode is enclosed in a separate 3d printed case so it can be easily placed on the screen and fixed in position with a velcro belt.
Additionally, the case shields the sensor from ambient light to reduce disturbances caused by external influences.
The probe is connected to the Raspberry Pi HAT with three wires for supply voltage, ground, and signal.

The latencies introduced by the photo diode and the ADC are in the range of microseconds, thus they do not confound the measurements in a significant way.

The total cost of the device is around 65€ (price includes Raspberry Pi 4), making it affordable for non-professionals.
Furthermore, we estimate the time a layperson needs to assemble the device to be under one day.

</WRAP>

<WRAP column right 30%>
[{{:projects:display_latency:deviceopen.jpg?400|Opened case of the device. It consists of a sensor probe (left), a microcontroller and a single board computer (right).}}] \\
</WRAP>
</WRAP>

=== Conducting a measurement ===

 * Connect the display under test to the measuring device via HDMI.
 * Dial in default settings and full brightness on the display.
 * Attach the sensor probe to the center of the display and fasten it with a velcro belt.
 * Power up the measuring device and wait until the main menu is shown.
 * Dial in the desired test mode and measurement ID with the control knob on the measuring device.
 * Start the calibration process.
 * Start the measurements.
 * To retrieve the result data, connect a USB memory stick to the measuring device and the data (in CSV format) will be transferred automatically.

==== Results ====

We measured the latencies of several monitors found in our lab.
Each measurement series consists of 190 measurements so we can report a distribution of latencies for each device (right).
The visualisation also includes the latency reported by the *Leo Bodnar Video Signal Input Lag Tester*, as well as an IDMS test setup, both indicated by vertical lines.

For each individual measurement, a brightness value is captured 10 times per millisecond over a duration of 30 ms.
This way, we can plot a brightness curve for each display which also contains information on how long the pixels take to reach full brightness (left).
The 50% brightness values determined during calibration before each measurement is indicated by a horizontal bar.

All enhancement features of the displays, like overdrive, were turned disabled for the measurements.
For devices that offer such enhancement features, we performed a separate measurement series with overdrive enabled (results are shown in orange).

<WRAP group>
<WRAP column left 45%>
{{:projects:display_latency:measurements:curve_aus-xg248q.png?600}}
</WRAP>
<WRAP column right 45%>
{{:projects:display_latency:measurements:swarm_aus-xg248q.png?600}}
</WRAP>
</WRAP>

<WRAP group>
<WRAP column left 45%>
{{:projects:display_latency:measurements:curve_DEL-DELL_U2415.png?600}}
</WRAP>
<WRAP column right 45%>
{{:projects:display_latency:measurements:swarm_DEL-DELL_U2415.png?600}}
</WRAP>
</WRAP>


<WRAP group>
<WRAP column left 45%>
{{:projects:display_latency:measurements:curve_DEL-DELL_U2417H.png?600}}
</WRAP>
<WRAP column right 45%>
{{:projects:display_latency:measurements:swarm_DEL-DELL_U2417H.png?600}}
</WRAP>
</WRAP>

<WRAP group>
<WRAP column left 45%>
{{:projects:display_latency:measurements:curve_DEL-DELL_U2713H.png?600}}
</WRAP>
<WRAP column right 45%>
{{:projects:display_latency:measurements:swarm_DEL-DELL_U2713H.png?600}}
</WRAP>
</WRAP>

<WRAP group>
<WRAP column left 45%>
{{:projects:display_latency:measurements:curve_DEL-DELL_U2715H.png?600}}
</WRAP>
<WRAP column right 45%>
{{:projects:display_latency:measurements:swarm_DEL-DELL_U2715H.png?600}}
</WRAP>
</WRAP>

<WRAP group>
<WRAP column left 45%>
{{:projects:display_latency:measurements:curve_ENC-EV2451.png?600}}
</WRAP>
<WRAP column right 45%>
{{:projects:display_latency:measurements:swarm_ENC-EV2451.png?600}}
</WRAP>
</WRAP>

<WRAP group>
<WRAP column left 45%>
{{:projects:display_latency:measurements:curve_GSM-W2442.png?600}}
</WRAP>
<WRAP column right 45%>
{{:projects:display_latency:measurements:swarm_GSM-W2442.png?600}}
</WRAP>
</WRAP>

==== Supplementary Material ====

<WRAP center round tip 40%>
Work in progress!
</WRAP>

 * [Source code for microcontoller](https://github.com/pstadler1990/latencytest_stm8_iar)
 * [Source code for Raspberry Pi program](https://github.com/pstadler1990/latencytest_fbutils)
 * Circuits and PCB layouts [TODO]

==== See also ====

* [Our research on latency of input devices](:projects:latency) including *LagBox*, a tool for measuring input device latency.
* [NVIDIA Latency Display Analysis Tool (LDAT)](https://www.nvidia.com/en-us/geforce/news/nvidia-reviewer-toolkit/). A device for measuring end-to-end latency. Not freely available but given to select [reviewers and influencers](https://videocardz.com/newz/nvidia-reviewer-kit-ldatpcat-will-make-gpu-testing-more-accurate)

## Publications
<WRAP group>
---- datatemplatelist ----
template: publications:list_template
cols    : %title%, authors_, date, short-description, abstract, pdf_url, bibtex_url, video_url, photo, published-in
sort    : date
filter  : %pageid%~publications:*
and     : project=latency
----
</WRAP>

## All Related Posts
<WRAP group>
---- datatemplatelist ----
template: news:list_template
cols    : %title%, date, description
sort    : ^date
filter  : tags ~~ latency, Latency
----
</WRAP>