Schreibtrainer – intelligent handwriting support

Good handwriting is an important skill for acquiring knowledge efficiently. Current studies show that when students make handwritten notes, they retain significantly more knowledge in classes and lectures than when they simply listen. However, when schoolchildren move up to high school, one third of them have subpar handwriting and spelling, while for another third, these skills are even classed as poor. When children learn how to write, which is the first major hurdle of their school education, there are three key factors to consider:

1. Automaticity of handwriting
2. Legibility of handwriting
3. Spelling

The Schreibtrainer project aims to develop the technological basis for a pen that reliably provides information on the level of pupils’ handwriting automaticity, detects possible deficiencies in their motor skills and points out spelling mistakes as they write. Working in close cooperation with our partner STABILO, we are optimizing the steps in the signal processing chain at the sensor level and improving the fusion of the pen’s sensor data. We have also teamed up with Kinemic to apply state-of-the-art machine learning approaches and further develop them to recognize letters and words using the pen’s sensor system.

In a further step, we are testing prototypes of new practical exercises that will help improve handwriting automaticity, spelling and letter formation. Our project partner STABILO is implementing this with an app. Another project partner, Saarland University and its Chair of Teaching Methodology for German, is supplying the learning content based on current teaching practice.

Fraunhofer IIS is responsible for the overall coordination of the Schreibtrainer project and provides advice and assistance in relation to technological challenges. In addition, Fraunhofer IIS is working on solutions for improving calibration of the pen and for optimizing the fusion of its sensor data. Through the development of a complementary camera system, Fraunhofer IIS is also investigating how additional sensors can improve reconstruction of the pen strokes and detection of when the pen stops moving. Thanks to its existing measurement infrastructure and expertise in conducting measurement campaigns, Fraunhofer IIS can moreover provide ground-truth data pertaining to the pen strokes. This information can be used for validation and as reference data for training artificial neural networks to recognize words and characters.

Based on the know-how developed in this project, we can optimize work on sensor fusion and integration of machine learning approaches for your use case. We can also apply our expertise in measurement and evaluation of sensor and positioning technologies to offer you customized measuring services.

Our unique Test and Application Centre L.I.N.K. makes it possible to realistically simulate scenarios for test and demonstration purposes.

Thanks to our depth of expertise in positioning, sensor fusion and machine learning, Fraunhofer IIS is a reliable partner for joint, German and international R&D projects.