Question: How is Facebook helping visually impaired people to “see” its photos?
Answer: Through an AI – The AI can “read” photos and tell visually impaired people what appears in them
With the internet changing from almost entirely text based content, it is now becoming more and more picture-led. Approximately 1.8bn pictures are uploaded to social media website such as Facebook, Twitter and Instagram.
Facebook have introduced a revolutionary system which can read photographs and describe what appears in them through speech. This is going to give visually impaired people the chance to visualise what is happening in the photos. Click here to watch Facebooks announcement video.
The way that most blind people make computers usable is by using a sophisticated piece of navigation software called “screenreaders”. This software does exactly what you would expect it to do – and turns the contents of the screen into speech output or braille. Unfortunately, they can only currently read text and not “read” pictures.
Using artificial intelligence (or AI), Facebook’s servers can now reasonably accurately decode and describe the images that are uploaded to the site and provide them into a form which can then be read out by a screenreader.
Matt King, the man behind the development, is a Facebook engineer who lost his sight as a result of retinitis pigmentosa – a condition which destroys the light sensitive cells in the retina.
Matt feels like there is a lot of things on Facebook which are extremely visual, and being blind can really make you feel like you’re being left out of the experience.
The technology is a massive breakthrough which uses object-recognition software to recognise items such as food and vehicles in images. Although the software is still in its very early stages, Matt says that “It will help them move in the direction of that goal of including every single person who wants to participate in the conversation.”
The system currently describes images in fairly basic terms such as: “There are two people in this image and they are smiling.” However, Facebook has said that its software is trained to recognise approximately 80 familiar objects, from cars and trains to food. It also recognises settings such as mountains, and sports such as golf.
The more images that the software scans, the more sophisticated the software will become.
We at Education Quizzes think that this is a remarkable idea.
Is there anybody visually impaired in your life that use screenreaders to use the computer effectively? If so, what do they think about the software? We would love to hear your thoughts in the comment section below.