AR / Arts / Media

A real-life walkthrough of how to make cool things with augmented reality

Brooklyn's Jane Friedhoff documented the problems and solutions of making a music video in AR.

Google Creative Lab made its own AR music video. (Image via YouTube)

We’ve been hearing a lot about augmented and virtual reality in the last few years but not seeing a ton of it in the real world. Now, for all the tinkerers and creatives out there, is a little guide on the problems and fixes that Brooklyn’s Jane Friedhoff encountered (and solved) while working on an AR prototyping project for the Google Creative Lab.

In a new post on Medium, Friedhoff talks about what was easy and what was hard in making an AR video experience of pop singer Grace VanderWaal’s song “Moonlight” with the lyrics written out in augmented reality that unfurl on the viewer’s screen as they move through the world.

“We came across Grace VanderWaal’s lyric videos and loved their style,” Friedhoff wrote. “Her videos frequently include her handwriting, and it made us wonder: could we make a handwritten lyric video, but explode it out into AR? What if the song wrote itself out, as it was being sung, all around you ?”

In the original video, the 14-year-old pop star VanderWaal, who came to fame competing on America’s Got Talent, walks and sings down and around what seems to be Kent Avenue on the Williamsburg waterfront. Friedhoff’s team at Google took the song and added hand-written neon lyrics to their own video shot in the perspective of walking around as well. In Friedhoff’s version, the viewer can look at, around, and through the lyrics on the screen as they move the screen.

Friedhoff, who is also a game developer, found a way to use game engine Unity to show the 2D handwriting in 3D space. Friedhoff’s previous creative work includes a favorite game of ours she released for free online called Handväska!, the point of which is to whack fascists with your purse.

Friedhoff’s whole post is worth a read if you’re curious about this stuff and shouts out another Technical.ly fave, Chris Whong, for learning how to get the handwriting right in CSS.

Snapshots from the video look cool. Maybe we’ll start watching some videos in AR sooner rather than later.

Read the full story
Series: Brooklyn

Knowledge is power!

Subscribe for free today and stay up to date with news and tips you need to grow your career and connect with our vibrant tech community.


When entering the STEM industry, keep your options open

Technically Media