Sensing the Unseen (SENSE-AI)

Sensory AI as a bridge between technology, art and learning

AI and sensor data are playing an increasingly important role in our surroundings, but the information they generate is often presented primarily in visual form. This makes data less accessible to some people and also limits the ways in which we can experience and understand information.

In SENSE-AI, The Hague University of Applied Sciences (Smart Sensor Systems research group), Sensory MoMa, AHA Leermiddelenbouwers and Equal Recruitment therefore explore how data can be translated into different sensory forms, such as sound, touch and light. We consider not only the technical translation of data, but also the context in which people experience this information and how they interact with it.

Objective

SENSE-AI develops and investigates a multisensory AI toolkit that translates sensor data and other types of data into accessible sensory experiences.

The project also explores how different forms of sensory feedback, environmental factors and user behaviour relate to the ways in which people experience information and their surroundings. The knowledge gained is translated into design principles, prototypes and educational materials for inclusive multisensory applications.

Living lab Sensory MoMA

Sensory MoMA is an important real-world setting within the project. Work here includes a multisensory interpretation of Mondrian’s Victory Boogie Woogie, in which the visual experience of the artwork is enriched with music, scent and touch.

We investigate how visitors move around and interact with the different elements of this experience. An AI camera is used to map patterns in visitor behaviour.

In addition, sensors both inside the museum and in Museumpark measure various environmental factors, including temperature, humidity, particulate matter, ambient light, sound and CO₂. This allows us to investigate how sensor data, human interaction and the physical environment can be used together to design multisensory and inclusive experiences.

Target groups

The project focuses on designers, artists, researchers, educational institutions and professionals in the creative industries.

People with visual impairments are an important target group and are involved in developing and evaluating accessible applications.

Students also play an active role in collecting, analysing and interpreting data, and in testing prototypes in a real-world setting.

Research methods

SENSE-AI combines co-creation, sensor research, AI and machine learning with user research and research in real-world settings.

Living lab experiments collect and analyse environmental data and patterns in visitor behaviour. At the same time, different approaches to translating data into multisensory experiences are explored. The results are used to test and further develop prototypes iteratively.

Expected results

• An experimental multisensory AI toolkit
• Knowledge about the relationship between sensor data, sensory translation, the environment and user interaction
• Design principles and guidelines for accessible multisensory applications
• Educational materials for students and professionals
• Practical demonstrations and knowledge sharing on inclusive AI and sensor technology

Project partners

• The Hague University of Applied Sciences – lead organisation
• Sensory MoMA
• AHA Leermiddelenbouwers
• Equal Recruitment

Contact

Femke van Geffen, [email protected]

More information

For more information about SENSE-AI, visit www.sense-ai.nl.