Chinese tech giant Huawei has developed an app that uses artificial intelligence to allow the visually impaired to “see” the emotion on the face of someone they are talking to by translating it into sound.
Called Facing Emotions, the app, developed with the Polish Blind Association, uses the rear camera of Huawei’s Mate 20 Pro phone to read the emotion on the face of whoever the blind user is talking to, by
An offline machine-learning algorithm integrated into the phone identifies the main facial features of the interlocutor, including eyebrows, eyes, nose and mouth. It can then determine the positions in relation to each other and identify the particular emotion.
The app translates these seven universal human emotions – anger, fear, disgust, happiness, sadness, surprise and contempt – into seven distinct sounds. The detected emotion is played through the phone’s microphone.
The sounds corresponding to each emotion were created by blind composer Tomasz Bilecki. “The sounds I wanted to create needed to be specific for each emotion, and not distract users from their conversations,” said Bilecki.
The team aimed to make each sound short, concise and simple, in the hope that they won’t feel invasive in a conversation between two people, as well as being recognisable and easy to recall. The app was tested by blind users and their insights were incorporated into the design.
“Sound samples were tested by our team of blind consultants, who rated them as easy to remember and understand,” said Huawei.
Visually-impaired people often need to use their hands to hold a cane or to help with other forms of navigation, so it was important that the app could be used hands-free.
To accommodate this, Warsaw-based product designer Janek Kochanski developed a 3D-printed phone holder. The holder is worn around the neck and is designed to make the app more intuitive for blind and visually impaired users.
“For me, as a designer,
The phone holders are available to buy online and a number will be made available through partnering charities to the visually-impaired in central and eastern Europe and in the Nordic countries.
“The Facing Emotions campaign embodies Huawei’s mission to bring new technology to everyone, especially those that need it most,” said Huawei consumer business group president Wang Tao.
Other projects aimed at helping the visually-impaired lead independent lives, including Design Academy Eindhoven graduate Aurore Brard’s coloured plates that help users to serve the correct amount of food and drink.
By Augusta Pownall