Did 3 Penn freshmen build the future of vision at PennApps X?

“What if a piece of software could tell you what exactly was in front of you with one simple command?” was the pitch that won ThirdEye a spot in the top 10 at last month’s PennApps X.

ThirdEye is an image recognition system for the visually impaired that takes a picture using Google Glass, recognizes what it is and tells you about it through an earpiece or speaker.

The hackers behind the innovative Google Glass app were first-time hackathon participants: Ben Sandler, Joe Cappadona and Rajat Bhageria. They were also the only all-freshmen team to make it to the top 10.

With Cappadona programming since 8th grade, Sandler having interned at a couple of tech startups and Bhageria having build his own media startup, CaféMocha, which he describes as the SoundCloud for creative writers, these three computer science majors were all excitedly anticipating their first hackathon.

Forming the ThirdEye team

Bhageria, Cappadona and Sandler weren’t initially part of the same team. However, a few hours before the competition started, Bhageria and Cappadona’s team members backed out, so they were looking for more team members. This led to them wandering around and running into Sandler who wanted to work on a hack for the Google Glass.

Remembering the struggles that his grandfather faced as a blind man, Cappadona was hit with an app idea they could work on — an app that would allow Google Glass to recognize whatever was in front of it.

“Blind people want to be independent. They don’t want others to think that they are physically inept because they lost their sight,” Cappadona said. “They’re still fully capable of most things. This really inspired our project.”

The team now had an idea, the only problem: they barely had experience building an Android app, let alone integrating it with Google Glass.

This is where the hackathon fun came in.

The first eight hours resulted in three lines of code. When they finally got Google Glass to take pictures, a bug hit their program and they had to start from scratch.

Not wanting to just give up, the trio did something every programmer does: Google the problem and start reading.

After hours going over Stack Overflow and previous APIs, Bhageria, Cappadona and Sandler started high fiving each other as they finally got their Google Glass to take pictures and recognize what was in front of it.

“PennApps really taught me the beauty in software, The barrier to entry is so low. It’s easy to build something quickly,” Sandler said. “Moreover, building something, not just for a class or for your job, but because you want to is really special. Thanks to the mentors, you could do so much more than you thought you could.”

Selling the vision

With only 36 hours to work on their hack, Bhageria, Cappadona and Sandler realized that a big part of doing well in this hackathon would actually be selling their vision and marketing their project.

“You don’t really have much time to build a perfect product,” Bhageria said. “A big part of PennApps is having to convince others that your hack is better than others.”

In hopes of getting people interested in ThirdEye, Rajat spent roughly 10 hours creating a logo, building a website to market the app, and coming up with a video to show people how the app worked.

Moreover, the team also handed out fliers before the Sunday morning demos began. In no time, people were talking about “ThirdEye” and the innovative hack the team came up with.

“We barely had a team, we didn’t have an idea. We didn’t know if it was going to work. But we just got the ball rolling,” said Cappadona. “So when people started telling us we did a great job, it felt really fulfilling. It just feels great to be able to contribute, to be able to help a few people get their sight back.”

Hackathon Experience

“All these people will change the world,” Bhageria said of his fellow PennApps participants. “They’re going to do something special. I met so many amazing people: recruiters, sponsors, mentors, and hackers; it was really inspiring.”

With PennApps being the world’s largest and most prestigious student-run hackathon with a roughly 10 percent acceptance rate, Bhageria, Cappadona and Sandler were extremely excited when they found out they had made it to the top 10.

“Every student should take advantage of this opportunity regardless of what their major is,” Cappadona said. “It’s just an amazing event in general. You get free food, free shirts, goodies and you’re just surrounded by all these people doing awesome things.”

Looking forward, Bhageria, Cappadona and Sandler each can’t wait to compete in another hackathon. But for now, they can’t help but think of the possible impact that their ThirdEye application could have for people with visually impairments.

The ThirdEye app is currently open sourced, and the team hopes to continue developing it and getting feedback.