Project: Input Device Latency

We measure and model the latency of input devices and its effect on user performance. Our research strengthens theoretical foundations and gives practical guidance for researchers, developers, and users.

Status: ongoing

Runtime: 2017 -

Participants: Raphael Wimmer, Florian Bockes, Andreas Schmid, Vitus Maierhöfer

Keywords: WIP, latency, prototype, USB, mouse, keyboard, joystick, gamepad, input device, input

News

PDA group at Workshop on Esports and High Performance HCI 2021 (2021-04-10)

Andreas Schmid will present a short paper on a new method for measuring end-to-end latency at the first Workshop on Esports and High Performance HCI, co-located with CHI'21. Further information on can be found on the project page. (more...)

DispLagBox at Mensch und Computer 2020 (2020-09-01)

We will present DispLagBox, a precise and easily replicable device for measuring display latency, as a short paper at Mensch und Computer 2020. (more...)

New paper: On the Latency of USB-Connected Input Devices (2019-05-09)

We presented our paper "On the Latency of USB-Connected Input Devices" at CHI 2019 in Glasgow. (more...)

PDA Group at CHI '2018 (2018-04-21)

We will present a poster and a workshop paper at CHI 2018. (more...)

Project Goals

Get a better understanding of latency and latency jitter
Develop a tool for exactly measuring the latency of input devices
Provide users with advice on which input devices to choose

Current Status (2021-10-10)

This project has spawned a few sub-projects in which we continue research on latency in human-computer interaction:

Yet Another Latency Measuring Device
DIY Approach to Measuring Display Reaction Time
Effect of Latency on User Performance (work in progress)

Measurements at a Glance

The following latency measurements were first presented in our CHI '19 paper.

Legend:

Blue: default latency distribution for each device

Orange: latency distribution if one manually sets a USB polling interval of 1 millisecond (=1000 Hz polling rate).

Click on image for larger version.

Gamepads:

Mice:

Keyboards:

FAQ - Frequently asked questions (work in progress)

What exactly is new about this research?

We suggest a simple theoretical model of latency in input devices, a precise measuring technique, and an appropriate visualization approach (swarmplots) that hightlights the peculiarities of each device's latency. Furthermore, we conducted more precise latency measurements for input devices.

Do latency differences between devices really matter?

It depends. In most everyday use cases, latency differences of 10-30 ms do not really matter. However, for professional gamers or in certain application areas (e.g. teleoperation, psychological reaction time experiments), low latency is extrremely imortant. An increase of latency by 14 ms means that one can only react one or two display frames later. This would eat up all performance gains from a 144 Hz computer monitor (compared to a common 60 Hz monitor). Scientific research is a little bit inconclusive about how strongly latency affects performance in games or similar tasks.

Why did you test pretty old or uncommon devices?

When selecting devices to test, we had two criteria:
1. devices whose latency had been measured earlier by other researchers or gamers (such as the Logitech G300 and G5 mice)
2. any devices which we had tested earlier or had lying around

Should I enable a higher polling rate?

Yes. We do not see of any negative side effects for most devices. In only one case (XBox 360 wireless) we noticed slightly higher average latency (by 1 ms) when polling every 1 ms.

Which device should I buy?

We cannot tell you this. In practice, other factors (ergonomics, haptics of the button press, tracking precision) are certainly more important than small differences in latency. On the other hand, you might want to avoid devices which exhibit significantly higher latency than others.

Model

The overall latency of an input device is determined by multiple components:

the mechanical delay of the button (BTN),
the microcontroller in the device (MC)
the USB polling interval (USB)
the kernel's process switching interval (OS)

Each component adds its own partial latency to the overall latency. Furthermore, the scanning/polling rates of microcontroller, USB host, and operating system add further latency that depends on when the polling occurs.
As the offset between the polling intervals shifts over time, the overall latency changes, too. This leads to characteristic latency distributions for each device.

LagBox

Hardware

The LagBox is based on a Raspberry Pi 2. A optocoupler, which can be switched by the RPi's GPIO interface, is connected to a button of the input device (Fig. 1) and can trigger a button press.
For this method of testing input latency, it is necessary to modify the input devices by connecting the transistor-side of an optocoupler to the button which should be tested (Fig. 2). Furthermore, this method is not suited to test the latency of input devices that offer a non-physical type of interaction, like touch displays.
An advantage of this method over non-invasive approaches is the fact, that external influences are reduced to a minimum due to the direct electric connection between the testing device and the input device. This method also allows it to trigger a large number of button presses in a very short period of time. Thus, it becomes easy to conduct stress tests on the hardware or to collect big amounts of data.

Figure 1: LagBox circuit

Figure 2: Hacked Gamepad

Software

On startup, the software of the LagBox requires the user to select the “hacked” button of the input device via command line parameters to determine which input event to listen to. Afterwards, the actual testing process begins. The RPi closes the optocoupler by writing to the corresponding GPIO pin, logs the current timestamp and waits for an input event by the input device connected via USB. Once the input event arrives, the optocoupler is opened again, another timestamp is logged and the difference is written to a logfile. Then we wait for a randomly selected time between 0.1ms and 10ms, so the circuit can discharge. The reason for the randomized waiting time is to avoid accidental synchronisation of the event readout with the polling rate. This testing process is repeated several (right now 1000) times to get a big enough sample size of input latencies.
The last latency as well as the overall progress of the test are shown on a minimalist GUI (Fig. 3).

Figure 3: GUI of the LagBox

Measuring Latency

How to build the LagBox and conduct your own measurements

Database of detailed latency measurements

Upcoming and Future work:

building a non-invasive method for precisely characterizing input latency (we are not very successful in this regard yet)
conducting a small study on the effects of input latency on performance in a simple game (first study conducted, second study needs to be prepared)
test more, current, devices (when time permits)

Publications

LagBox – Measuring the Latency of USB-Connected Input Devices.

Florian Bockes, Raphael Wimmer, Andreas Schmid

CHI EA '18 Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems

Development of a tool for measuring latency of different USB devices (Tweet this with link)

On the Latency of USB-Connected Input Devices

Raphael Wimmer, Andreas Schmid, Florian Bockes

Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19)

Summary of our current understanding of latency and our approach of measuring the latency of USB-connected devices. (Tweet this with link)

DispLagBox: Simple and Replicable High-Precision Measurements of Display Latency

Patrick Stadler, Andreas Schmid, Raphael Wimmer

Proceedings of the Conference on Mensch und Computer 2020

Easily replicable device for measuring the reaction time of displays. (Tweet this with link)

Yet Another Latency Measuring Device

Andreas Schmid, Raphael Wimmer

Workshop on Esports and High Performance HCI 2021

Measuring device for end-to-end latency based on cheap off-the-shelf components. (Tweet this with link)

Measuring the Latency of Graphics Frameworks on X11-Based Systems