Utilising the SUR40 for Physical Digital Workflows and Workspaces

Status: ongoing

Runtime: 2018 -

Participants: Jürgen Hahn, Raphael Wimmer

Keywords: Interaction Techniques, Interaction Design, Computer Vision


Development of a framework which allows easy access to robust detection, tracking and digitisation of physical documents or devices in combination with the affordances of virtual windows, files etc, so that application developers can build and evaluate interaction techniques.


In order to implement and research possible interaction techniques for physical-digital workflows and workspaces, an application developer friendly framework is required for fast user, hardware or software testing iterations. The framework's first iteration is targeted for the Samsung SUR40 Multi-Touch Table (MTT) using its camera-pixels in order to generate a 960×540 surface image of the otherwise 1080p display. This image is to be evaluated for markers, text, etc. in order to proof interaction technique concepts and potential new input modalities, like digitally stamping / tagging physical documents with a tangible interaction device.

Used Technologies:

  • Samsung SUR40
  • Custom Debian Driver by Florian Echtler
  • OpenCV 3.2.0
  • ARuco Markers
  • Custom Arduino-based Tangible Interaction Devices
  • Input Devices

Future Extensions:

  • Utilise a 4K Projector in order to visualise a workspace combining physical-digital affordances
  • Utilise a Depth-Camera or Stereo-Camera setup and order to track Paper from above
  • combine both approaches