How visual recognition is set to change advertising

Ad relevancy is key to moving a customer down their purchase path. The consumer profiles we build on each individual user provide insight into their needs, brand biases, social stances and much, much more. When done correctly, we are able to better understand a prospect, allowing us to place a relevant ad in front of a person who is actively shopping for our brands or products.

To be even more successful, we need to go beyond what that individual wants at that specific moment. Behavioral economics has shown that consistent touch points over a period of time allow us to influence potential customers when they are activated to make a purchase decision at a later date. Being there throughout the funnel, not just at the bottom of the funnel, is key to increasing your pool of qualified customers.

One way to get a more complete picture of a customer is to perform textual and sentiment analysis on the words they share on social media. The technology to do this kind of thing has been around for a while, which is why innovators are taking such analyses to the next level. Now, visual recognition software is helping develop a deeper understanding of consumers in a similar way. (Note: The industry doesn’t seem to have settled on a single name for this technology. Besides visual recognition, it’s is also known as computer vision, image recognition and image analysis.)

The images and videos a person shares or likes via social media provide insights not only into what products they like, but can also give marketers a window into many other elements of a potential customer’s lifestyle. Powered by machine learning, algorithms can quickly identify unique elements of images that can help power the marketing machine that determines what ad should be delivered to whom for the best result.

“We do expect multimedia posts to become the predominant type of post on social media. Even the text that accompanies those posts is getting shorter and shorter. … It becomes increasingly important for companies to be able to understand what’s going on in those images,” Gartner VP of Research Jenny Sussin was quoted as saying in a post on Crimson Hexagon’s site.

Advances in visual recognition

Many companies such as IBM, Apple, Amazon, Google and Facebook continue to push the envelope when it comes to image recognition. Google’s advances in image search continue to evolve, while Facebook can spot almost anyone in a crowd of an uploaded image. IBM continues to evolve within its Watson platform, using machine learning to identify various elements within the image. IBM is going beyond the people in the picture. Their visual recognition capabilities allow for the identification of scenes, objects, faces, colors, food, text and much more.

And with the use of APIs, these technologies are ready to be tapped and integrated into any number of martech tools. Amazon is aggressively marketing its Rekognition service, which makes powerful image and video analysis available. Google, too, has made available its Cloud Video Intelligence and Vision APIs along with the Cloud AutoML Vision machine learning service. Additionally, some social media analytics players like Brandwatch, Crimson Hexagon and Netbase are already touting their own capabilities.

This is how Amazon describes its Rekognition capabilities on the AWS site. (Click to see a larger version of this image.)

As quickly as it is advancing, image or visual recognition software faces a challenge on the consumer side, where many fear the privacy consequences of having such a technology widely deployed. Think back to when Apple released Face ID — a biometric security measure — with its iPhone X in 2017, and how news about its inaccuracy and general “creepiness” dominated the tech headline.

Can't be the only one that thinks the whole Face ID system on the new iPhone is just weird, creepy and completely taking away our privacy ?

— milly (@MillyAmandaDunn) September 14, 2017

And with the more recent privacy discussion sparked by the General Data Protection Regulation (GDPR) pushing forward in Europe, companies like Facebook are constantly having their innovation in facial recognition questioned and stymied there due to associated privacy concerns.

How visual recognition will be applied to advertising

It seems likely, though, that visual recognition will eventually become a part of the digital landscape in some form and, even without faces, it offers a lot of potential opportunities. Within many images users upload onto social media there are hundreds, if not thousands of elements that can now be identified through visual recognition software.

It may seem like a picture of you and your friends at a bar, but it is so much more. Of course, standard elements like location, tagged individuals and the hashtag used are all up for grabs. But visual recognition analyzes everything within an image. It can tell if you are drinking whiskey or vodka based on the color of the liquor in your glass. It can determine the fabric of your shirt and the brand of your jeans. It can read the sign in the background and the name of the restaurant from the menu sitting on your table.

This type of information provides deep insight into who we are as consumers and is extremely valuable to advertisers. For example, if a picture someone shares on social media includes them holding a glass of high-end whiskey while also wearing a Gucci belt, chances are they are a good target for luxury retail goods. Compare that with someone holding a low-end beer brand and a shirt with an Old Navy logo on it.

By looking back through the various images shared on an individual’s profiles, the deep-learning algorithm can determine if luxury goods are a common part of this person’s lifestyle. Affinities toward brands and/or products can be used in real time to purchase programmatic inventory across display, video and even addressable TV. Being able to analyze images to this level enables us to go from targeting people by what companies they “like” or “follow” on social media to targeting them based on the products and brands they actually buy and use, leading to more relevant ads and ultimately a better-targeted customer.

As advertisers, we are just beginning to reap the benefits of leveraging visual recognition capabilities. As we move forward and the capability becomes more widespread and adopted, it will become a key asset.