Teri Wilson
April 08, 2016 1:01 pm

Imagine what logging onto Facebook would be like if you were blind. Even if your laptop was powered by technology that could read text aloud, you’d miss out on a lot. I’m pretty sure memes alone account for half of all Facebook status updates (I totally made up that statistic, but it’s got to be close). And what about the photos? Those are super important. How could you properly stalk your secret crush without the ability to see the photos? (Just kidding…probably.)

Thanks to some super cool new technology, vision impaired individuals will no longer be missing out on Facebook images. Starting just this week, the social media giant is using artificial intelligence to describe photos to its blind users. According to Popular Science, special algorithms will scan the images and identify objects in the photos. The objects will then be tagged in a caption that will be read aloud to the user. It goes a little something like this:


Up until now, vision impaired Facebook users relied on screen reading technology that could only verbalize text (such as status updates minus the photos). This is the first time that science has found a way to interpret data from images on the site.

It’s not perfect, of course. For now, the artificial intelligence algorithms can simply tag objects in a photo. They can’t describe the relationships between the objects. Manohar Paluri, a lead A.I. researcher at Facebook, says, “Our goal is getting to a point where it’s describing much much more than tags. How do tags interact? What are the relationships between tags? Not just saying ‘cat’ and ‘bed.’ You want to say ‘cat on the bed,’ or ‘cat jumping over the bed.’ So this is a starting point.”

As a starting point, it’s pretty amazing.


We can’t wait to see what comes next!