Software Development

This company is using AR headsets to offer detailed insights on how customers shop 

Tupelo, new from Pittsburgh consulting firm Rivers Agile, is attracting attention from major product manufacturers.

Software Consultant Mark Westerlund demos Rivers Agile's new platform Tupelo (Alice Crow/ Technical.ly)

A 16-year Pittsburgh consulting firm is pivoting to tech by launching a custom platform with a lofty goal: to revolutionize focus groups. 

Founded in 2008 to provide quality assurance and software testing, North Shore-based Rivers Agile hasn’t shared much about what it’s been up to over the past couple of years. Founder and CEO Ben Wilson is excited to change that with the Robotics Discovery Day launch. 

“We spent most of our 16 years of existence building great products for other people,” Wilson told Technical.ly, “and now we’re rolling out our own.”

That product is an augmented reality (AR) research tool called Tupleo. It’s software that can be installed on a VR headset customers wear while shopping for or interacting with a company’s product. It tracks how they react — verbally or through physical cues — to gather feedback for a company, similar to how a focus group works, but resulting in more granular, quantifiable data.

Rivers Agile recently joined the Pittsburgh Robotics Network (PRN), and its approach has been “very Pittsburgh,” according to PRN Executive Director Jennifer Apicella. 

“Between AI, computer vision and augmented reality, they are really maximizing advanced technology,” Apicella said. “Innovation first, innovation forward.”

Rivers Agile community involvement, especially in AI, goes back even further. In 2019, it announced a workshop to support local, early-stage AI, machine learning and fintech companies with its knowledge of the tech from its time consulting.

Building on existing tech for a smooth transition into the field 

Rivers Agile has been developing Tupelo for the past 15 months. 

It was initially designed for a large beverage manufacturer that wanted to analyze best practices for end caps, those displays at the end of supermarket or department store aisles that showcase high-demand items. The beverage company sought data that would help it determine the best placement for its most profitable products. 

The Rivers Agile team decided to use existing tech like Meta’s Oculus headset and the Unity programming platform to develop Tupelo, which uses AR to overlay digital content onto the real-world environment. 

A man sits at a fake steering wheel wearing a VR headset, while another points at what he is seeing on a computer.
Rivers Agile Founder and CEO Ben Wilson points out the arrows that indicate user eye tracking on the Tupelo platform (Alice Crow/Technical.ly)

It then combines eye tracking, mood detection and image recognition technology to gather in-depth user feedback. 

“We are using hardware and repurposing it in a way it was not necessarily intended,” said Wilson.

Tupelo tracks the user’s eye movements and marks their focus on specific objects as “events.” These actions are displayed as an arrow on the recording, indicating exactly what the user is looking at. This allows the platform to track customer behavior by recording data like how often a participant looks at a particular product or whether certain items go unnoticed entirely. 

AI programmed to understand how humans express themselves

The Rivers Agile team also added a sentiment analysis feature powered by AI. 

Software Consultant Mark
Software Consultant Mark Westerlund demos Rivers Agile’s new platform Tupelo (Alice Crow/ Technical.ly)

The program listens to what users say, transferring speech to text, and an algorithm interprets the feedback. Sometimes it can be simple, like a user saying, “I hate this” and the program registering the feedback as obviously negative, but it can also pick up on more subtle user signals using the sensors on the headset. 

The team associated ranging human emotions with subtle expressive cues, or the physical expressions of feedback. The headset senses those movements and is programmed to understand whether it’s positive or negative input.

“These headsets have a ton of sensors and some of them are exterior, so when you smile, your cheekbones go up,” Wilson said. “There’s no denying it, it’s a fact of being a human being. Understanding a number of other human emotions and defining those was a model that had to be created.” 

What’s next? Applying the tech to new industries, like robotics

Seven staff members at startup Rivers Agile pose for a group photo.
Staff members of the Rivers Agile team at their office, located on the North Shore (Alice Crow/ Technical.ly)

Rivers Agile is now considering other use cases for AR to track human feedback. 

For example, the software could be used in store cameras to learn what customers are looking at or, it could help automakers determine how drivers interact with car features. 

“With each application to a different industry we’re going to find small pockets of new functionality to be developed,” Wilson said.

Looking ahead, Wilson said Rivers Agile is focused on forging new partnerships with interested companies and expanding Tupelo’s capabilities. 

One area it’s particularly excited about is remote troubleshooting, which would allow two people in different locations to wear VR headsets and guide each other through complex tasks, such as fixing a car or learning how to fly a plane.

“In the future, we’ll be placing further emphasis on deep tech,” Wilson said, “wrapping our arms around the robotics community and continuing to serve them.” 

Companies: Pittsburgh Robotics Network

Before you go...

Please consider supporting Technical.ly to keep our independent journalism strong. Unlike most business-focused media outlets, we don’t have a paywall. Instead, we count on your personal and organizational support.

3 ways to support our work:
  • Contribute to the Journalism Fund. Charitable giving ensures our information remains free and accessible for residents to discover workforce programs and entrepreneurship pathways. This includes philanthropic grants and individual tax-deductible donations from readers like you.
  • Use our Preferred Partners. Our directory of vetted providers offers high-quality recommendations for services our readers need, and each referral supports our journalism.
  • Use our services. If you need entrepreneurs and tech leaders to buy your services, are seeking technologists to hire or want more professionals to know about your ecosystem, Technical.ly has the biggest and most engaged audience in the mid-Atlantic. We help companies tell their stories and answer big questions to meet and serve our community.
The journalism fund Preferred partners Our services
Engagement

Join our growing Slack community

Join 5,000 tech professionals and entrepreneurs in our community Slack today!

Trending

The person charged in the UnitedHealthcare CEO shooting had a ton of tech connections

From rejection to innovation: How I built a tool to beat AI hiring algorithms at their own game

Where are the country’s most vibrant tech and startup communities?

The looming TikTok ban doesn’t strike financial fear into the hearts of creators — it’s community they’re worried about

Technically Media