How does reverse image search work?

If you have had a chance to try reverse image searching, you know how powerful this tool might be. But how does this technology work? Let us explain it a little.

What is a reverse image search engine?

A reverse image search engine is a type of website or app that allows searching for things, people, etc. using an image, not words. In contrast to a regular query, you don’t enter any phrase related to the object you want to find, but you upload a photo of something or someone to find its/their name or something/someone similar.

You can access most reverse image search engines using your browser. Most of them are available on your mobile phone (with Android, iOS, and other systems), tablet, or computer. You may also use a dedicated app that allows searching for content by image. The best-known reverse image search engines are online tools, but it is also possible to find examples of offline solutions. Usually, those are tailored business softwares ordered by some specific company.

Reverse image processing

What exactly does reverse image processing look like and how do reverse image searches work? The specifics vary for different search engines, so we can’t provide the details. However, most of them have similar mechanisms. That gives us the opportunity to generalize and share some information about how they work.

In a nutshell, reverse image processing works like this: when a user uploads a photo to a search bar, it is first analyzed and measured. Then, the algorithms of the search engine check the database and look for the exact pictures or images that have similar parameters or objects. In the end, the photos from the database and from the uploaded file are matched again and displayed as results. Included in the results is the URL of the source website which you can follow to find the images. All of these processes happen in a very short period of time (even the hundredth of a second). The crawled database can be as wide as the Internet, but it can also be a set of your own digital photos from your device or website.

Image fingerprinting

The reverse image search matching process is based on something called an “image fingerprint”. The association with a real fingerprint is quite legitimate. It is unique information about what is in the photo (like a unique alignment of lines on your finger), expressed not in words, but in specific parameters that are understandable for algorithms. It is important to mention that people are unable to decipher these parameters, only algorithms.

Image fingerprints are used in the process of comparing pictures. When you conduct a search, first the image fingerprint of your uploaded photo is created, and then it is compared to the image fingerprints of all the pictures in a given database.

Machine learning and reverse image search

The explanation of image fingerprinting leads us to the next technological issue: the connection between reverse image search technology and machine learning. Machine learning is fundamental to the algorithms’ teaching process. Using this technology, algorithms are trained to “see” objects in photos. Thanks to this type of learning they are able to recognize items, people, etc. in the photo that is used in the reverse image search.

A quick summary of what we know about the reverse image search process: machine learning accounts for the teaching part, image fingerprinting for the matching part. We need to explain just one more aspect to have the simplified overview of the reverse image search completed: searching and displaying results.

String and searching image

We can call these two types of search mechanisms alternatives, but there are many situations in which they are rather complimentary services. String is the most common search mechanism: it allows people to perform a search using words. We can call this one a “text search”. The second solution used in search engines is “image search”, which really stands for the reverse image search mechanism. This technology uses images for two purposes: conducting a search and displaying the results. If you need to find something that is in the photo, you can get the best results by performing an image search.

Reverse image search for faces

Reverse image search technology might be used in many types of software. One of them is our facial recognition search engine. Using it you can track down the images of yourself that are posted online, including ones that are published without your consent. We focus on faces, so we use a small modification of the “image fingerprint”, called a “face fingerprint”. Analogically to the image fingerprint, we measure the photos used in the search (both to perform a search and to match the results), but only the part with the face. Thanks to this we are able to find you in photos taken on different backgrounds, having other haircuts or contact lenses. If you are interested in the reverse image searching topic, look at the article on our blog in which we explain why it is better to use dedicated software like PimEyes to perform facial searches on the Internet.

In our index, there are face fingerprints from publicly available websites like media, blogs, company web pages. We don’t search social media.

To perform a search, upload your photo using the part of the search bar with the magnifier icon and then select an image from your device. To take a new photo, use the part with the camera icon.

Summary

Reverse image search engines work based on 3 technologies: image fingerprinting, machine learning, and reverse image search. They correspond, respectively, to:

• comparing photos (image fingerprinting)
• recognizing objects in the photos and the teaching process (machine learning)
• performing a search and displaying the results (reverse image search).

Reverse image search engines can be created as autonomic websites that allow users to conduct a search using only an image, but they can also be used in many types of software. An example of a specialized use of reverse image search technology is PimEyes: a search engine that allows you to track down your face online.