Project: Input Device Latency

We measure and model the latency of input devices and its effect on user performance. Our research strengthens theoretical foundations and gives practical guidance for researchers, developers, and users.

Status: ongoing

Runtime: 2017 -

Participants: Raphael Wimmer, Florian Bockes, Andreas Schmid, Vitus Maierhöfer

Keywords: WIP, latency, prototype, USB, mouse, keyboard, joystick, gamepad, input device, input

Achtung: die untenstehenden Inhalte sind inzwischen teilweise nicht mehr aktuell. Die aktuellen Messergebnisse finden sich in unserem neuen Paper On the Latency of USB-Connected Input Devices und untenstehend. Das Paper enthält auch die aktuellsten Details. Kommentare und Fragen gerne an raphael.wimmer@ur.de –RW, 10.5.19

  • Get a better understanding of latency and latency jitter
  • Develop a tool for exactly measuring the latency of input devices
  • Provide users with advice on which input devices to choose

We have presented our paper On the Latency of USB-Connected Input Devices at CHI 2019. Currently, we are making our measurement system - LagBox - ready for external use. Furthermore, we are preparing a latency database for input devices to which users can also upload their test results.

The following latency measurements were first presented in our CHI '19 paper.

Legend:

Blue: default latency distribution for each device

Orange: latency distribution if one manually sets a USB polling interval of 1 millisecond (=1000 Hz polling rate).

Click on image for larger version.

Note: we are preparing a comprehensive database of latency measurements. Please stay tuned.

Gamepads:

Mice:

Keyboards:

Send us your question (English/German):

We suggest a simple theoretical model of latency in input devices, a precise measuring technique, and an appropriate visualization approach (swarmplots) that hightlights the peculiarities of each device's latency. Furthermore, we conducted more precise latency measurements for input devices.

(to come. see visualizations)

It depends. In most everyday use cases, latency differences of 10-30 ms do not really matter. However, for professional gamers or in certain application areas (e.g. teleoperation, psychological reaction time experiments), low latency is extrremely imortant. An increase of latency by 14 ms means that one can only react one or two display frames later. This would eat up all performance gains from a 144 Hz computer monitor (compared to a common 60 Hz monitor). Scientific research is a little bit inconclusive about how strongly latency affects performance in games or similar tasks.

When selecting devices to test, we had two criteria:
1. devices whose latency had been measured earlier by other researchers or gamers (such as the Logitech G300 and G5 mice)
2. any devices which we had tested earlier or had lying around

Yes. We do not see of any negative side effects for most devices. In only one case (XBox 360 wireless) we noticed slightly higher average latency (by 1 ms) when polling every 1 ms.

We cannot tell you this. In practice, other factors (ergonomics, haptics of the button press, tracking precision) are certainly more important than small differences in latency. On the other hand, you might want to avoid devices which exhibit significantly higher latency than others.

Model

The overall latency of an input device is determined by multiple components:

  • the mechanical delay of the button (BTN),
  • the microcontroller in the device (MC)
  • the USB polling interval (USB)
  • the kernel's process switching interval (OS)

Each component adds its own partial latency to the overall latency. Furthermore, the scanning/polling rates of microcontroller, USB host, and operating system add further latency that depends on when the polling occurs.
As the offset between the polling intervals shifts over time, the overall latency changes, too. This leads to characteristic latency distributions for each device.

Prototype

Hardware

The LagBox is based on a Raspberry Pi 2. A optocoupler, which can be switched by the RPi's GPIO interface, is connected to a button of the input device (Fig. 1) and can trigger a button press.
For this method of testing input latency, it is necessary to modify the input devices by connecting the transistor-side of an optocoupler to the button which should be tested (Fig. 2). Furthermore, this method is not suited to test the latency of input devices that offer a non-physical type of interaction, like touch displays.
An advantage of this method over non-invasive approaches is the fact, that external influences are reduced to a minimum due to the direct electric connection between the testing device and the input device. This method also allows it to trigger a large number of button presses in a very short period of time. Thus, it becomes easy to conduct stress tests on the hardware or to collect big amounts of data.

Figure 1: LagBox circuit
Figure 2: Hacked Gamepad


Software

On startup, the software of the LagBox requires the user to select the “hacked” button of the input device via command line parameters to determine which input event to listen to. Afterwards, the actual testing process begins. The RPi closes the optocoupler by writing to the corresponding GPIO pin, logs the current timestamp and waits for an input event by the input device connected via USB. Once the input event arrives, the optocoupler is opened again, another timestamp is logged and the difference is written to a logfile. Then we wait for a randomly selected time between 0.1ms and 10ms, so the circuit can discharge. The reason for the randomized waiting time is to avoid accidental synchronisation of the event readout with the polling rate. This testing process is repeated several (right now 1000) times to get a big enough sample size of input latencies.
The last latency as well as the overall progress of the test are shown on a minimalist GUI (Fig. 3).

Figure 3: GUI of the LagBox


  • building a non-invasive method for precisely characterizing input latency (we are not very successful in this regard yet)
  • conducting a small study on the effects of input latency on performance in a simple game (first study conducted, second study needs to be prepared)
  • test more, current, devices (when time permits)

Florian Bockes, Raphael Wimmer, Andreas Schmid

CHI EA '18 Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems

Development of a tool for measuring latency of different USB devices (Tweet this with link)


latency_thumbnail.jpg

Raphael Wimmer, Andreas Schmid, Florian Bockes

Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19)

Summary of our current understanding of latency and our approach of measuring the latency of USB-connected devices. (Tweet this with link)