Macy’s, Bloomingdale’s and a dozen other large retailers are quietly using controversial facial-recognition technology to fight a rise in smash-and-grab robberies and other coordinated attacks.
It’s part of a larger push to turn retailers’ existing security cameras — those ubiquitous black gadgets that produce thousands of hours of mostly useless footage — into a more sophisticated artificial-intelligence surveillance system, capable of automatically identifying people, license plates and other information that can be used to alert store employees of unfolding threats and ultimately prosecute offenders.
“There’s a big play to use AI right now,” said Read Hayes, director of the Loss Prevention Research Council. “Retailers have got cameras all over the place anyways.”
The technology has new appeal at a time when theft and violence is on the rise, particularly from organized crime groups that steal millions of dollars in merchandise and then sell the products online. Organized retail crime has risen by 60% since 2015, according to the National Retail Federation, with nearly 70% of retailers reporting an increase in 2021. As much as $69 billion worth of products are stolen from the nation’s retailers each year, or 1.5% of sales, according to estimates from the Retail Industry Leaders Association and the Buy Safe America Coalition. A record 523 people were killed during robberies and other violent retail incidents in the U.S. in 2020, including 256 customers and 139 employees.
This month, the federal government charged 29 people with stealing $10 million in over-the-counter medicine and other items from Walmart, Costco, CVS, GNC and others, then reselling the products on sites like Amazon and eBay.
“What retailers are really trying to do is give police more evidence of who is doing this,” said Adrian Beck, a professor at the University of Leicester whose research focuses on ways to combat retail loss.
Facial-recognition technology is controversial because research has shown it’s often inaccurate when identifying people of color and women. The worst technology has an error rate of up to 35% when scanning darker-skinned women, but less than 1% with lighter-skinned men, according to one research report. The reason? Early versions of the algorithms were trained using images of celebrities that skewed white and male.
Companies have been working to address the racial bias, and between 2014 and 2018, facial-recognition software got 20 times better at searching a database to find a matching photograph, according to the National Institute of Standards and Technology. However, as of 2019, the government found that some software still misidentified African-American and Asian people 10 to 100 times more often than white men.
“People have been wrongly arrested because of this technology,” said Jay Stanley, a senior policy analyst for the speech, privacy and technology project at the American Civil Liberties Union. “It’s really not ready for prime time.”
There’s been a push to curb the use of facial recognition by law enforcement, with cities like San Francisco, Minneapolis and Boston barring police officers from using it. Companies like Amazon and Microsoft have stopped selling the tech to police.
However, facial-recognition companies actively market their technology to retailers, who use it to help gather evidence on repeat offenders before ultimately sharing it with local law enforcement.
“There’s been a shift to where the retailer is doing some of this work themselves,” said Tony Sheppard, director of loss prevention solutions at ThinkLP. Law enforcement resources are sometimes thin, he added.
FaceFirst, which said it works with one-quarter of the largest retailers in North America, compiles time-stamped incidents and a calculation of past losses packaged for law enforcement. “When you send this to police, it’s a really strong storyboard,” FaceFirst president Dara Riordan said at a recent industry conference.
Retailers must create their own watchlists, a manual process that involves an employee identifying a person on video footage after an incident and asking the software to alert them the next time the same person walks into one of their locations. It won’t help the first time someone steals, but it can catch repeat offenders.
“Demand for our product is exploding,” said Dan Merkle, CEO of FaceFirst, which now runs more than 12 trillion face comparisons a day for its customers, up from 100 million in 2017. It claims its technology has a 99.7% accuracy rate, with no difference by gender or race, and can reduce theft from non-employees by 34% and in-store violence by 91%.
It’s helped one retailer catch “The Philly Fanatic,” a thief with a reputation for donning Philadelphia Phillies gear, not to be confused with the baseball team’s mascot of the same name. He’d been hitting the same retailer every one to two days with different accomplices, surreptitiously filling a cabinet with $2,000 or so in goods, closing it with double-sided tape and purchasing the cabinet like normal. The goods were then turned over to a “fence,” in charge of reselling the goods. The software helped identify the suspect and send a real-time alert to the retailer, who then apprehended him and turned him over to police, says FaceFirst.
U.K.-based Facewatch said business has doubled during the pandemic, with particularly strong demand from convenience stores. Its biggest deployment is at a retailer with over 100 locations.
Retail is also a growing customer segment for New York-based Oosto, according to chief marketing officer Dean Nicholls. Both regional and national retail chains use its technology, and some of them have deployed it at hundreds of stores. The 150-person company has raised $350 million in funding.
The technology suffered a setback at the beginning of the pandemic, when everyone suddenly began wearing masks. Accuracy rates have been improving as companies train their algorithms on masked faces, but their success rates still vary widely, ranging from 60% to 99%, according to a report from the National Institute of Standards and Technology published this month. Facewatch says it’s now operating at about a 90% accuracy rate and has a human employee check each image before sending an alert to a retailer.
“In 2020, it was debilitating to everybody,” said Merkle. “Now we can function at very high accuracies even with a mask on.”
Retailers are also trying to figure out when to communicate to employees that an offender has walked in the store and how to instruct them to act without putting them in danger. “What do you say to the poor checkout employee on minimum wage?” said Beck.
Few stores want to admit they’re involved with facial recognition, fearful of backlash from shoppers. Rite Aid used the technology for nearly a decade at hundreds of stores, but stopped after an investigation from Reuters pointed out that it used the software in largely low-income, non-white neighborhoods. Others, like Target and Home Depot, have tested facial recognition in the past, but now say they don’t use it.
Retailers are also using other types of video technology. For instance, virtual security guards have become increasingly popular during the pandemic. Guards are stationed remotely and can respond to various alerts — like motion detected by a security camera in the parking lot at 1 a.m. — by speaking directly to the suspected intruder and alerting law enforcement. It’s easier than hiring in-person guards during the current labor shortage and certainly cheaper, running 8% to 12% of the cost, said Sean Foley, an executive at Interface, which provides virtual guards to Zale’s, Dollar General, Big Lots and Gamestop.
Body cameras, like the ones worn by police offers, are also being adopted by retailers. Typically outfitted on security guards, two-thirds of the retailers surveyed by the ECR Retail Loss Group were using the cameras in some capacity and reported a 45% decrease in the number of violent and verbal abuse incidents. They can help cover blind spots left by security cameras, said Scott Thomas, national director of sales for signature brands at Genetec, which sells security system software.
Security cameras are also increasingly capable of identifying objects, like license plates, as well as behavior. Oosto is developing technology that could someday identify weapons, like a gun, or pick up on unusual motion, like multiple people running towards a store, and quickly send an alert to the store to put employees on notice.