Interactive assistance system to support people with cognitive limitations or language barriers through comprehensible work instructions in multi-variant production

Supervisors: Steffen Kinkel (HKA), Matthias Wölfel(HKA)

Faculty: Informatics and Business Information Systems

Many B2C as well as B2B customers nowadays expect increasingly individualised or customized products and are no longer satisfied with solutions “off the rack”. In the production and assembly of these products, this leads to a significant increase in the number of manufactured variants and a decrease in the respective volumes and batch sizes. This variant-rich production places increasing demands on the cognitive abilities of the employees in production and assembly. The corresponding work instructions can no longer be standardised for large product families, but are differentiated accordingly for the many variants. This usually results in complex instructions of arrangements of components and work steps that predominate in hybrid representations of images and text. These instructions can be a major barrier for people with cognitive or language limitations. This can be limited language comprehension due to migration background, lack of qualification or command of the technical language, limited cognitive abilities as well as difficulties in spatial perception of images.

Against this background, the aim of the PhD project is to

  • to categorise and systematise the various possible complexities of visual and linguistic representations against the background of the possible limitations of the users' visual and linguistic perceptions,

  • to develop a socio-technical concept of an interactive assistance system that generates suggestions for the most urgent communication barriers in the presentation of different work instructions in variant-rich production, how these can be communicated as comprehensibly as possible to people with limited visual and linguistic cognitive abilities,

  • to develop a digital, possibly AI-supported prototype of an interactive assistance system that represents a suitable and realistic solution concept for one specific or several typical use cases,

  • to evaluate the prototypical solution concept using suitable qualitative and/or quantitative methods with regard to applicability and user acceptance.

Desired qualifications of the PhD student:

  • University degree (M.Sc.) with excellent grades in computer science, business information systems, industrial engineering, information or cognitive sciences

  • Programming or application knowledge in at least one programming language (preferably Python and with experience in TensorFlow, PyTorch or similar)

  • Experience or skills in the design and evaluation of socio-technical assistance systems

  • Good English language skills (your responsibilities include writing publications and giving international presentations)