AR / Funding / Health / Resources / Universities

NYU is building a real-life holodeck

For what? All sorts of stuff. A $2.9 million grant from the National Science Foundation will get things going.

Could Star Trek's holodeck actually become a real thing? (Photo via Memory Alpha user Aatrek)

The prospect of sci-fi turning into real life is pretty familiar these days. In the past couple of years, several projects have claimed to point the way toward a real-life version of the holodeck from Star Trek: basically, an immersive virtual-reality experience in which participants can freely interact with their surroundings. For now, it’s still more hype than substance. But NYU got a $2.9 million grant from the National Science Foundation to make it happen.
When Winslow Burleson, director of the NYU-X Lab at the College of Nursing, announced the grant at the NYC Media Lab Summit last month, quite a few people in the audience, including this reporter, were incredulous. (“I’m still trying to wrap my head around that holodeck presentation,” one demo participant said, hours later.) So we tracked Burleson down for more details. Does the project actually intend to bring Star Trek to life? And besides the obvious cool factor, what will be the eventual payoff?

We very much believe in this as a large-scale open innovation endeavor.

The project involves several schools and departments within NYU, including the Media and Game Network (MAGNET), which is based at NYU Tandon in Brooklyn. Brooklyn-affiliated faculty on the project include computer science professor Ken Perlin, who is a member of MAGNET, and composer and Tandon associate professor R. Luke DuBois. In addition, NYU is partnering with 18 other universities, including MIT and UC Berkeley, as well as companies such as IBM and Sennheiser.
The National Science Foundation grant covers a five-year period, with the first two years designated for bringing the technology to initial maturity: in other words, one that other universities could investigate. Burleson said he expects that an initial prototype will be built by the end of 2017.

The NYU Holodeck isn’t a novel technology, per se, but a combination of many technologies under one platform: virtual and augmented reality, wearable devices, brain-computer interaction, 3D printing, robots, advanced sound systems and game controllers, all underpinned by IBM’s Watson. What’s novel, Burleson said, is how these technologies will work together. Interacting with a virtual reality display in the NYU Holodeck, for instance, could summon a social robot to perform a task.
“It’s like using the Force to bring it to you,” he said.
(Yes, we know that’s a Star Wars reference. Apologies for the mixed sci-fi metaphors.)
Burleson has previously worked on several of the technologies being incorporated into the NYU Holodeck. As a doctoral researcher at the MIT Media Lab, he developed a “multimodal affective learning companion” — in other words, a software-generated character that responds to users’ emotions in order to help them learn new skills. In his current role at the College of Nursing, he has worked on virtual-reality simulators that enable students to train for medical procedures.
With this project, the university hopes to expand upon its educational and research possibilities. One key aspect of the NYU Holodeck is that its platform will be distributed, so it won’t be confined to NYU’s campus. Instead, its partner institutions will be running experiments at the same time, and their results will be captured and shared on the platform. As a result, Burleson said, the Holodeck will become “smarter” in a sense, by having more simulations and virtual scenarios to draw upon as it undergoes testing. And, of course, universities and other institutions will be able to collaborate more efficiently, by using the same shared resource.
“We very much believe in this as a large-scale open innovation endeavor,” he said. “We hope it will pave the way for many other Holodeck nodes to be built.”
So how much does this line up with the Star Trek holodeck that inspired it — or, at least, its name? Note that there’s no mention of holographs here: one key feature of the original, which other supposed “real-life holodecks” have attempted to replicate. Burleson acknowledged as much.
“The current thinking is that’s further out than our project,” he said.
But, he added, with the distributed model, the NYU Holodeck may be even more sophisticated than the sci-fi version, at least in one respect.
Overall, he said, “We think that we’re spiritually in the ball park.”

Companies: NYU Tandon School of Engineering
Series: Brooklyn

Before you go...

Please consider supporting Technical.ly to keep our independent journalism strong. Unlike most business-focused media outlets, we don’t have a paywall. Instead, we count on your personal and organizational support.

3 ways to support our work:
  • Contribute to the Journalism Fund. Charitable giving ensures our information remains free and accessible for residents to discover workforce programs and entrepreneurship pathways. This includes philanthropic grants and individual tax-deductible donations from readers like you.
  • Use our Preferred Partners. Our directory of vetted providers offers high-quality recommendations for services our readers need, and each referral supports our journalism.
  • Use our services. If you need entrepreneurs and tech leaders to buy your services, are seeking technologists to hire or want more professionals to know about your ecosystem, Technical.ly has the biggest and most engaged audience in the mid-Atlantic. We help companies tell their stories and answer big questions to meet and serve our community.
The journalism fund Preferred partners Our services

Join our growing Slack community

Join 5,000 tech professionals and entrepreneurs in our community Slack today!


Get a snapshot of Baltimore's latest housing, job and business trends with this new easy-access tool

Technically Media