Amazon reveals how it’s using AI to transform the retail experience

0 today showcased the myriad ways AI-powered machine learning and computer vision algorithms are combined with synthetic data to improve key retail automation technologies such as Just Walk Out, Amazon One and Amazon Dash Cart.

In a keynote at Amazon’s re:MARS event today in Las Vegas, Dilip Kumar, vice president of physical retail and technology at Amazon, Explain how computer vision plays a critical role in enabling these technologies. For example, Just Walk Out, a technology that allows shoppers to skip the line, has benefited from continued innovations in areas such as sensors, optics and machine vision algorithms.

These advancements have allowed Amazon to reduce the number of cameras required in Just Walk Out-enabled stores, Kumar said, making them more cost-effective and able to run their algorithms locally.

“Our sensors and algorithms have evolved to detect a wide range of products and differences in shopping behavior across large grocery stores, while ensuring an effortless customer experience,” Kumar said. “We’ve also increased the diversity of environments our algorithms need to consider when deploying Just Walk Out technology at third-party retailers.”

Meanwhile, the company is using computer vision and sensor fusion algorithms to scale its Dash Cart service, which lets customers skip checkout at Amazon Fresh stores in the United States. Most important, Kumar said, is that the company has developed more robust algorithms that can detect moving objects and capture both their weight and quantity.

“Machine vision algorithms also have strict latency budgets because we keep track of a customer’s receipt in real time,” he added.

AI also helps provide better recommendations to customers. For example, shoppers at Amazon Style, the company’s physical clothing store, will be treated to a diverse list of recommended items based on the products they scan as they browse the store.

“The system also generates complementary selections, such as a shirt to match with a pair of jeans to create a complete outfit,” Kumar said. “We went to great lengths to keep shopping fun while improving the experience through machine learning algorithms.”

However, none of this would be possible without the use of synthetic data. As Kumar explained, Amazon was challenged by the lack of diverse training data needed to train these algorithms. To compensate, Amazon researchers set out to create massive sets of photorealistic synthetic or machine-generated data, which could be used to perfect its algorithms.

In the case of Just Walk Out, Amazon had to create synthetic data sets to mimic realistic shopping scenarios, Kumar said. For example, synthetic data was used to create variations in lighting conditions to account for differences in sunlight in various stores. Additionally, Amazon created crowds of synthetic shoppers in order to train its algorithms to handle many customers at once.

Synthetic data has even been used to create palm prints to form Amazon One, which is a service that lets people use their palm to pay in a store or enter physical locations. Authentic palm prints are hard to find, Kumar explained, but Amazon had to train Amazon One’s algorithms to recognize different demographics, age groups, temperatures and even variations such as calluses and wrinkles. So he opted to create a huge volume of diverse and realistic synthetic palm tree images instead.

A final challenge Amazon has faced is that shopper behavior tends to change as the company develops its retail technology. Just Walk Out was first rolled out to smaller Amazon Go stores that typically only cover 1,800 square feet, but has since rolled out to much larger grocery stores of 40,000 square feet or more, Kumar explained.

Amazon faced an unexpected challenge as it quickly realized shoppers behaved differently at these stores. At an Amazon Fresh grocery store, for example, people tend to search for items like the freshest produce, whereas at an Amazon Go they’re more likely to just grab a sandwich, for example. Likewise, expanding Just Walk Out to businesses such as travel retailers and sports venues presented unique challenges.

“All of these scenarios increase the complexity of our algorithms, and my team continues to innovate to meet the demands of our customers and retailers.” Kumar said.

Pictures: Amazon

Show your support for our mission by joining our Cube Club and our Cube Event community of experts. Join the community that includes Amazon Web Services and CEO Andy Jassy, ​​Dell Technologies Founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many other luminaries and experts.


Comments are closed.