Is Retail Ready for Facial-Recognition Technologies?

Albeit too fast for privacy advocates, face recognition technologies appear to be advancing far enough to finally offer a real solution for retailers.

NEC just launched a facial recognition system in Japan that lets retailers profile customers to estimate not only gender and age but whether the shopper has been at the store before and how frequently she shops there. Using only a security camera and an internet browser connected to NEC’s cloud computing technology, the service, NeoFace, also apparently comes at a budget-friendly $800 U.S. per month.

NeoFace is intended for retailers with multiple stores, enabling them to detect repeat customers across locations. Individuals are recognized regardless of changing facial expressions, facial hair or eyeglasses.

"Retailers can find out how many customers visit their stores at each time of day, and what customer’s attributes are," said an NEC spokesperson in an interview with Japan’s DigInfo.

face recog 2Another option is the EyeSee from Italy’s Almax SpA. A camera embedded in a mannequin’s eye gives reads on ages, sex and race. Store associates can even be scanned out.

Online, Facebook is employing facial-recognition technology to assist in photo tagging.

PC Magazine likened the technology’s potential at retail to the scene in the movie, "Minority Report," in which Tom Cruise walks into a Gap store and is "instantly recognized, prompting the system to suggest new purchases based on that customer’s purchase history." Knowing the general makeup of customer traffic can also help stores maximize window displays, layouts, personal and promotions by time of day.

But most facial recognition articles ultimately bring up security and privacy concerns. Encrypting the biometric data and not storing the images may address some of these issues.

face recog 1On the privacy side, the FTC in October released guidelines basically saying consumers had to opt-in to allow businesses to use facial recognition technology. Details on how the captured images are being used also have to be given. But some extra language added to those signs indicating that cameras are installed for security purposes may fulfill those guidelines.

While Almax claims a "few dozen" retailers in Europe and the U.S. are using its EyeSee mannequins since its launch last December, Nordstrom is at least one believing facial recognition crosses the privacy line. Nordstrom spokesman Colin Johnson told Bloomberg News, "It’s a changing landscape but we’re always going to be sensitive about respecting the customer’s boundaries."

BrainTrust

Discussion Questions

Will the trepidation facing the rollout of facial recognition technologies for retailers outweigh the promised benefits? What value do you think stores will ultimately realize from such technologies?

Poll

32 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Max Goldberg
Max Goldberg
11 years ago

Retailers may love it, but unless they are willing to provide significant benefits to customers, I don’t see this technology being well received. There is something too “big-brotherish” about facial recognition technology for most consumers. If my opinion is correct, I see more benefit to retailers like Nordstrom that publicly repudiate the technology.

Doug Stephens
Doug Stephens
11 years ago

In a word, yes. But the traction that this and other similar technologies get depends less on what retailers get out of it and more what consumers experience. Imagine being in a store that is literally changing its assortment, promotions and merchandising in real time, based on the precise cross-section of customers in the store at a given moment. Imagine being sent a note the instant you enter a store that tells you what your friends “like” there. This is where we’re headed. Getting there is always a little awkward at first.

Peter Fader
Peter Fader
11 years ago

To me, the privacy issue isn’t the biggest concern. Here’s the problem: few retailers are extracting even 10% of the potential value that is hidden in their existing customer databases. So they need to walk before they can run. They should first take advantage of what they already have before they take on an initiative like this (and the costs/risks that come with it).

Debbie Hauss
Debbie Hauss
11 years ago

If retailers are going to use facial recognition technology, they need to roll it out with a well-planned and executed educational program for shoppers. Is the technology just guessing my age and sex or is it identifying me as a specific individual? There’s a big difference there and that could determine whether or not shoppers will buy in.

Ralph Jacobson
Ralph Jacobson
11 years ago

Although the industry had anxiety over capturing shopper data with the original retailer loyalty programs, the vast majority of shoppers willingly gave their personal information in order to receive discounts. I believe this is the next step in the evolution of the savvy consumer.

The movie “Minority Report” was a glimpse into what may actually happen soon. I see no long-term issues with it. Retailers and CPGs will gain better insights to target shoppers even more effectively.

Warren Thayer
Warren Thayer
11 years ago

I’d expect serious consumer backlash, as we’re starting to see with people worried about things like Facebook photos and information. Identity theft is one of the fastest growing industries in the country, and people, even tell-all me, are becoming increasingly cautious. And, like Peter says, retailers are using only a small amount of the data they have already.

Sure, the technology is sexy, but I don’t expect many retailers will be disciplined enough to make much real use of this.

Ryan Mathews
Ryan Mathews
11 years ago

It all depends on whether or not retailers can deliver perceived value to the customer.

There are cameras at ATMs, but nobody notices them. On the other hand, a significant number of people are still concerned about the anonymous body scans at airports.

Why? Because people like the convenience of instant access to cash and getting scanned takes time and forces them to completely empty their pockets. Now, if TSA could figure out a compelling enough value proposition, people might strip down to go through security.

Okay, Okay, I was kidding, but the fact remains that people are willing to trade “freedoms” for “benefits,” but only if those benefits work for them and not a second or third party.

Steve Montgomery
Steve Montgomery
11 years ago

While it made for an interesting scene in a movie, face recognition carries with it a very strong invasion of privacy stigma. My concern is that retail establishments will only be required to post a small sign at the entry point that indicate the cameras are in place, and that by simply entering the location you are granting permission for their use. People will ignore the sign as many do now the terms and conditions of the software they install on their PC’s, Macs, and phones.

There are several different systems being used today to track customers’ shopping in a store. These already provide a great deal of information including heat mapping, dominant traffic patterns (consumer turned left or right at various points), how long customers stayed in section of the store, how long they spend making a selection, etc. all without creating a specific consumer profile.

I agree with several of the earlier comments that say retailers have a great deal of information that they don’t use now. Moving forward with facial recognition may create a consumer backlash B&M retailers don’t need.

Rick Moss
Rick Moss
11 years ago

These concerns over privacy and security are reminiscent of those voiced when e-commerce was young. (Remember the resistance to entering credit card numbers online?) Yes, we all are a little creeped out by the thought of being tracked online, but how many of us have ratcheted up the privacy settings in our browsers? Doing so would greatly limit the benefits we get from the good e-retailers and social platforms.

Ultimately, shoppers will visit stores that use this technology if it results in better experiences. As with the web, I see these concerns falling by the wayside because we love convenience and great service that much.

Raymond D. Jones
Raymond D. Jones
11 years ago

There is a big difference between using this technology for profiling a retailer’s customer base and using it to recognize and track individual customers. The former is certainly worthwhile and relatively easy to do, while the latter requires a high level of accuracy and raises serious privacy concerns.

As usual, technology is running well ahead of social acceptance. Facial recognition will have to demonstrate significant value to the customer before it gains acceptance. Just because the technology is available doesn’t make it a benefit.

Ed Rosenbaum
Ed Rosenbaum
11 years ago

Technology is going to allow this to happen. How soon it becomes a factor is yet to be determined. There are still some people who think a cell phone is something only to be used in emergencies. Believe it or not I have a friend, not a doctor, who still has a pager.

Cathy Hotka
Cathy Hotka
11 years ago

Every year a new technology is touted as the next big thing in retail. I’d guess that customers would prefer better in-stock positions and improved customer service instead.

Adrian Weidmann
Adrian Weidmann
11 years ago

I am a proponent of the use of video analytics in a retail environment, but the use of facial recognition as a marketing and sales tool crosses the respect and privacy line. There are plenty of other opportunities and physical ‘cookies’ available to monitor shopper behavior without having to resort to facial recognition technologies. These technologies are interesting as a tradeshow demonstration or for getting publicity and attention, but all too often fail miserably in providing any sustainable value.

If not leveraged properly, these technologies simply provide a massive pile of data—simply making ‘Big Data’ bigger, with statistical findings which are presented on a colorful ‘dashboard’ with pie charts and graphs of every geometric shape and color. These are certainly interesting for a short period of time, but where is the ROI, or more importantly, the benefit for the brand and their customer?

The truly sustainable value is discovering the trends embedded deep in this data and present predictive recommendations to retail and brand marketing, merchandising and operations executives. These technologies are always looking in the rear view mirror! The real value is to use these technologies to provide quantitative based forward-looking recommendations on what brand and retail executives should be doing next week, next month and next quarter to provide a relevant and better shopper experience, environment, and product mix.

James Tenser
James Tenser
11 years ago

So many fascinating nuances surround the use of facial recognition technology at retail.

One is the potential legal work-around of privacy laws afforded by using “security” cameras to capture images as opposed to systems dedicated for shopper research purposes. The difference may seem like splitting hairs, but in practice they may be more significant—the quality, quantity and location of the cameras, as well as the type of software analytics applied.

Another is very aptly stated by Prof. Fader and worth underscoring here: Retailers barely make use of the data captured today. What do we expect they will do with inputs from this new, more sophisticated, and much vaster data source?

There is also the matter of sunk costs. Cabling up thousands of stores with multiple cameras, backroom servers, and head-end connectivity is a significant capital investment. Retailers don’t pay for s*** like this, so the vendors will need to do some careful calculus regarding the payback potential. My prediction is that most systems will be obsolete long before they are fully amortized (just like digital signage).

Finally there is the matter of business practices surrounding facial rec technology. Simply put, there are none. Vendors may propose some tantalizing ways to analyze the data for market research, but putting the data flows to daily beneficial use is another matter. To the extent that retailers work out ways to use this innovation to improve shoppers’ success, it will gain their acceptance. Otherwise, it will just be a creepy and expensive waste of effort.

Ed Dennis
Ed Dennis
11 years ago

For a retailer, crossing the line isn’t the question. The question is how can a retailer use this information to improve their service? Retailers, especially the poor ones, will jump at any gimmick that promises something for nothing. If retailers would stick to working for a living (real focused work, not make work) they would be okay, but they don’t seem capable of focusing on hiring right, training right and pricing right.

Camille P. Schuster, Ph.D.
Camille P. Schuster, Ph.D.
11 years ago

The idea of being watched is creepy. However, there are lots of cameras installed in many places already—we just don’t think about them. In the US consumers trade privacy for benefits in social media, cameras to detect thieves, convenience for stronger security, etc. If surveillance cameras can offer the needed clarity for facial recognition then it can already happen.

Peter’s comment about utilizing information is very appropriate. Right now companies are being bombarded with a huge amount of social media data. Facial recognition is one more thing being added to the fire hose of information. Pick a form of data, pick an analytical tool and start. Anywhere. Knowing something is better than drowning.

richard mader
richard mader
11 years ago

Some years ago, ARTS created a standard for video analytics. Frankly there has been very limited implementation. VA used for improved shopper tracking, by simply recognizing men, women and children it is great; to identify individuals, privacy issues and really no longer required.

Mobile technology with opt-in can tell retailers when specific customers are in their stores, to ensure they are greeted and provided appropriate service.

I saw the NEC demo 5+ years ago in Japan and just last month in Chile. Does anyone know a major retailer that has implemented? If not, you’ve found the answer.

David Slavick
David Slavick
11 years ago

I vote for a chip embedded in every consumer’s scalp—like a ear piercing or tattoo. Perhaps straight out of the delivery room in true Orwellian fashion, the consumer is thus never anonymous, forever recognized.

We have cameras on most street corners to monitor behavior on the sidewalks and the streets, so what is the big hub-bub over facial or retina recognition? The point shared that web transactions over time once looked upon with fear, now overcome due to sheer convenience. The threat of identity theft or interception of the signal to capture your payment data is just as real if not more so and yet Cyber Monday hit almost $2B in sales.

In some environments I like the concept—the beauty counter at your favorite department store or strip location such as ULTA. Put your pretty face or delicate pre-registered/opt-in finger (index) in front of the device and the consultant gets an instant print out of your past purchases, preferences and status. At one point in time not too long ago I developed a business case for smart chips embedded in member cards to store your purchase history. Challenge was the cost of the behind-the-counter equipment and the chip itself.

Over time as illustrated with NEC and others, cost has and will come down with adoption. 3-5 years—no/highly unlikely, 10-15 absolutely, where convenience and real-time purchase history/preferences are very individualized and 1:1 service is a well supported standard.

Dave Haynes
Dave Haynes
11 years ago

It’s really important to acknowledge the distinction between technologies that actually recognize individuals and those that just recognize and long the machine-learned patterns of faces.

The former needs a database of stored faces to actually scour, and is invading privacy.

The latter just counts how many face patterns that come into view of a camera, and logs and parses them by gender, age range and time.

To me, the most interesting aspect of this tech is understanding how many people look at messages, digital or static, and for how long.

I agree with earlier comments that retailers already have no end of data that they don’t really mine for insights. This just adds to that pile.

Tim Callan
Tim Callan
11 years ago

Let’s not get confused between what people say and what people do. Were you to survey average computer users and ask them how they felt about their online behavior being tracked, they would say that it’s abhorrent and they’ll never shop at sites that do so. But all sites do. It’s called cookieing, and it’s everywhere, and virtually nobody disables cookies on his or her browser, let alone discontinues using the web.

The same is true for brick-and-mortar retailers. This kind of technology is HUGELY beneficial for the retailers. It is not actually harmful to a shopper’s privacy in any way for the retailer to know whether that person is male or female or about how old, or even if that shopper has been here before. We’re all on video all the time when we’re at stores—all you have to do is look at the ceiling to see the cameras—and that doesn’t prevent anyone from shopping. Taking that video and using it to draw conclusions about who’s at your store is simply the next step in creating a better shopping experience.

Liz Crawford
Liz Crawford
11 years ago

Trepidation will limit the implementation of facial recognition software for now … but trepidation won’t make it extinct.

When smartphone, television, laptop and tablet interfaces “recognize” the user, then it will be only natural to extend that into the bricks-and-mortar space. That may sound creepy to Baby Boomers, but in 20 years, it’ll be the norm.

Lee Kent
Lee Kent
11 years ago

There are so many forms of data available these days—hence the term Big Data—that retailers need to investigate them and determine what will work best for their customers. It IS all about bringing value and experience to the customer first and to the retailer second, IMHO!

Craig Sundstrom
Craig Sundstrom
11 years ago

Promised benefits? That sounds overly generous…maybe “hyped” or “hoped for” would better describe them. Anyway, far be it from me to criticize something that makes it easier to “estimate gender,” but as I see it, retailers have long had technologically sophisticated data-gathering devices in stores (frequently so well hidden that they go unnoticed). I’m referring, of course, to sales associates. As Peter notes, stores make little use of what info they already have…I don’t think (even) more will help.

Herb Sorensen, Ph.D.
Herb Sorensen, Ph.D.
11 years ago

So if your friendly retailer greets you at the door by name, or at the checkout, and asks about your family, this is GREAT customer service? But if this process is automated it is an outrage that you have been recognized?

When you walk about in public, there is no reasonable expectation that no one will recognize you. The reality is that, as Scott McNealy of Sun Microsystems said a decade ago, “You have no privacy—get over it!” That “reality” is closer to the truth than society’s general welcoming of the embrace of the benefits of technology, but abhorrence of technology’s ubiquity, perceptiveness, permanent memory, and exploding ability to manage big, BIG data.

Society (and government) is an ass when it attempts to control the genie that has long since left the bottle. Part of the problem is that those parties don’t even understand the futility of what they are doing—much like retailers banning cameras in stores, for decades. Whatever happened to that? Did you get the memo that outlined the change in policy? Me neither!

The policy was changed by the reality that a very large share of shoppers are carrying phones with embedded cameras. Go ahead, Mr. Retailer, spit in the wind and try to stop the use of phones in your stores. Very likely those very same phones (and their cameras) will become integral to massive increases in the efficiency and sales in your stores, in the years ahead.

Don’t understand this? Not to worry, millions of some of the brightest minds on the planet are working day and night to accelerate smartphone technology at retail, and that cart is unstoppable. (Along with the ability of all those cameras to look back at you and your operations, Mr. Retailer.) That “looking back” is part of the “reverse big brother phenomena.”

There used to be a lot of talk about the danger of “big brother” watching you, which generally referred to government as big brother. But with the advanced government/industrial complex, interestingly, it is people observing—even photographing—government and industry, and communicating same to EVERYONE on the internet, that is driving the largest changes.

I know this is a bit broad for people who prefer to see issues at the other end of a long pipe. But facial recognition is only one small, but very important cog in the inexorable turning of the wheel of progress. You can impede it (like the banning of cameras—a surrogate for the human eye and mind—but by and by, history will chew your obstructionism up.

Martin Mehalchin
Martin Mehalchin
11 years ago

Considering that many retailers are still struggling to get their traffic counting systems to work, I would be skeptical that the facial recognition systems can be successfully deployed and deliver the advertised features and benefits. We’re probably super-early in the adoption cycle for this one.

Janet Dorenkott
Janet Dorenkott
11 years ago

Most consumers will not like the idea, but eventually it will be accepted. My skepticism is more related to the low cost of $800/month. I have a hard time believing that technology will really be available in the US for that inexpensive price. My skepticism also comes from what types of stores will be using it and how they will train their associates to use it. Do you focus on the people who come in all the time? You may be ignoring a wealthy person who is not a regular and is there buying gifts. I’m just not sure the value will be there.

Matt Schmitt
Matt Schmitt
11 years ago

Retailers are generally going to steer clear of facial recognition technologies for the foreseeable future.

However, as Dave Haynes points out, there is a critical difference between facial recognition and anonymous video analytics (AVA)technologies.

The use of facial analysis systems to determine number of shoppers, age and gender can go a long way in helping a retailer better understand and serve their shopper, without actually crossing the line of privacy invasion. AVA technologies are anonymous in nature, and don’t try to match a face to an actual personal profile.

A valid point is raised about the firehose of data being generated by retailers. How does a retailer prioritize and implement video analysis data, personal or otherwise, into their big data projects and strategies?

Kenneth Leung
Kenneth Leung
11 years ago

I think the issue is really the ROI of getting and acting on facial recognition data. Unless there is an action and offer to consumers, you are just generating more data and potentially creating privacy concerns with shoppers.

John Karolefski
John Karolefski
11 years ago

Knowing who is in the store via facial recognition can supplement the study of shopper behavior; in other words, “who” is doing what? If shoppers find out about this technology, they might find it creepy at first. But they’ll get over it.

Bernice Hurst
Bernice Hurst
11 years ago

Here’s one British newspaper’s take on the idea….

Mike Osorio
Mike Osorio
11 years ago

Leading edge, customer-focused retailers will be able to utilize the technology in ways that add value to the consumer. Unfortunately, a few non-customer focused retailers will jump in and probably not gain from the consumer insights, and may even hinder the customer experience.

As always, the customer focused retailers will utilzie the latest technologies to provide real value and improved experiences and their customers will love them for it.
Nordstrom says they will not use it…not yet. But I expect they will as the technology gains acceptance and customers demand the bennefits that come from it.

Carl Gohringer
Carl Gohringer
11 years ago

Whilst the benefits to business are clear and seductively tantalising, it has been impossible to ignore the increasing murmurs of discontent amongst the wider population.

Face Recognition in Retail: Privacy, Profits and Ethics