What Facebook sees in your photos

Facebook has developed lots of image processing tools, but those are often kept from sight to avoid the wrath of privacy-defending organisms around the world. Even so, you can catch a glimpse of it in action by looking at the automated image descriptions it creates for your photos.

There's no technical reason for Facebook not to be able to identify any person in a photo that we upload (as long as it's in its user base), but you can imagine Europe would go "bananas" about it. But, interesting enough, when anyone uploads any photo to Facebook, it is automatically tagged by its image recognition engine. This means there's a description of the image that, sadly, most people won't ever see, but is highly valuable for search purposes as well as accessibility features.

For instance, a group photo can show up a description like: "Image may contain: 26 people, people smiling, people sitting, people standing and indoor". That's quite accurate, and although we don't expect it to work flawlessly for every photo, it's good enough for most cases.

If you're curious about it, you can dive into the web page HTML and search for the IMG tag and the "alt" field (you can try and search for "image may contain" to get there faster), or you can simply use this Chrome extension (also works on Firefox) that will simply shows you the description overlaid on the photos as you browse along.