Project: Incorporating Hand-Writing Detection and Recognition into Physical-Digital Workflows and Workspaces
to come.
Status: on hold
Runtime: 2018 -
Participants: Jürgen Hahn, Nicolas Symeou, Raphael Wimmer
Keywords: Interaction Techniques, Computer Vision, Machine Learning, Deep Learning
Goal
Enable the digitisation of hand-written annotations and signatures on physical paper or etc., in order to incorporate them into the digital twin for machine processing in regard of physical-digital affordances.
Status
Background
In order to optimally use the required technologies, the status quo of Machine Learning / Deep Learning in regard of Hand-Writing Detection and Recognition has to be familiarized with. Then, appropriate techniques are customized for the physical-digital use case. This requires custom collected data sets and a custom designed and trained neural network. Stroke coordinates, stroke progression and individual point coordinates of the strokes of letters, words, sentences serve as input for said trained NN. These are extracted via image processing and computer vision techniques. Ultimately, interaction techniques, familiar to users, are retained in the new digitised context of their workflow.
Used Technologies:
- Machine Learning / Deep Learning
- Computer Vision
- Multi-Touch-Table SUR40 and Framework (see https://hci.ur.de/projects/mtt)
Relevant Literature:
Future Extensions: