Shop Talk: Orpheus Media Research music analysis tool Clio

Dr. Greg Wilder says that he’s created software that can dissect and analyze music so deeply and powerfully that music copyright litigation, discovery and automated composition will experience a paradigm shift.
He’s beaten researchers at MIT and Stanford who have been researching similar technologies for years, he says. And he was able to beat them because they’re computer scientists with an interest in music.
He, on the other hand, is a classically trained musician with an interest in computer science.
Wilder, who is the co-founder and chief science officer of Orpheus Media Research, received his piano performance and composition doctorate from Eastman School of Music, a top nationally-ranked music conservatory in Rochester, New York. He says he’s been playing piano since he was three.
“I spoke music before I spoke English,” he says. “Computer stuff, at this point, is out there for the taking. The music stuff is not.”

The company’s patent-pending software, called Clio, was vetted, peer-reviewed, published by and presented at the 2008 International Computer Music Conference in Belfast, Ireland.
The company just recently decided to form a corporate structure, with two new partners, including Chief Technology Officer Jason Allum, who we’ve covered for his recently-sold Macintosh-based RipIt software, and co-founder Dennis Slovak, a financial industry veteran who sold stock market analysis software on Wall Street.
The software, which Allum has been improving, once took one hour and 20 minutes to analyze a single 12-minute performance. Now, the same piece runs in under a 100th of a second, most of which the company credits to a development shift from Java to Objective-C.
It’s come a long way since Wilder first envisioned it as a way to automatically compose jingles for commercial music.
After graduating from Eastman, Wilder moved to Philadelphia where he began teaching at West Chester University and Dickinson College in Carlisle and became involved with more music theater production gigs and commercial song writing opportunities that required him to think beyond his classical training and develop and write pop songs.
That’s where the foundation for Orpheus Media was built.
“I began to analyze my thought process. What are the essential elements I need to replicate to emulate this band without crossing the plagiarism line and how little information do I need in order to imitate it accurately?”
He created Web software, a collection of PERL scripts that stitched together audio clips, that was like an “intelligent Garage Band,” he says.
A user would log-in, input the length of the clip he or she needed, choose a genre – like gospel for instance – decide the instruments, and the software would assemble the proper-length clip. His customers were made up of corporate training video editors and interactive Web producers looking for affordable, customizable song clips.
But he saw an opportunity for something more. Something more intelligent.
“If you want real power, you have to know what’s going on inside the language of music. What is that bass line doing, how does it relate to the guitar part, how does the vocalist know when to come in,” Wilder asks.
“Those things require detailed knowledge of music, and that was my expertise.”
Holed up in his studio apartment, Wilder began pulling apart generic pre-recorded music style snippets, piecing together the elements to see how they are created. He realized something crucial to Clio early on: researchers in the past shaped their algorithms around a particular style.
“It might work really well for that country ballad style they were analyzing,” he says. “Next thing they know, they try to apply it to a samba style or whatever, and it doesn’t work at all. The rules have all changed.”
“I knew I couldn’t set up a set of rules that would conform to a style. Depending on the context, it changed totally the flavor of what was happening.”
Clio takes every event and builds context from note to note to note. The software has no associations, no knowledge of music, genre or style. What it does know is theoretically technical models like the Implication-Realization model, published in the early 90s, which says that listeners of music – note to note, event to event – have expectations that are realized or denied.
Hum the first two notes of the Jaws theme: Duh-Duh.
The notes mean nothing on their own. But begin to hum them in a increasing succession (Duh-Duh, Duh-Duh, Duh-Duh, Duh-Duh) and humans recognize the dramatic implication immediately.
Using other such theoretical models of universal language, Clio is able to analyze music without bias.
Watch a video of Wilder explaining how Clio extrapolates musical language below.
[tech]R_D1xBbQQSo[/tech]
Wilder says the company has several potential business models. Wilder hopes to someday use the software to automatically create music, as was originally intended. It could be used in music discovery engines, like Pandora and Last.FM. Finally, there’s legal research, where the company is now focused.
He says that the technology can be used in copyright litigation that protects intellectual property, like Coldplay and Joe Satriani’s recent dispute, settled last week, concerning “Viva La Vida”‘s melodic similarity to Satriani’s “If I Could Fly.”
“It can pull out the most important parts of the melodies. Clio can determine ‘this is the most important chunk of the melody.’ This is the part you walk away humming, the part you like, the hook.”
It can be potentially used by lawyers to research comparisons and record publishers and advertisers to mitigate risk. It could be even more powerful when paired with a database of music to run songs against, which Wilder says the company is assembling now.
Wilder says that the business model built around legal issues is the most immediate way to bootstrap the company, since music companies are risking tens or hundreds of millions of dollars per year to defend and prosecute such lawsuits.
If it takes off with lawyers, it’s got a lot of room for growth in other fields as well.
Music discovery services like Pandora, which help you get through the work day by choosing songs it thinks you will like – based on parameters you feed it – is currently based on audio fingerprinting technologies, which can identify only limited aspects of music files, like tempo rate in songs with obvious drum rhythms.
Clio’s base technologies could make discovery far more powerful, Wilder says. He uses a library metaphor to compare audio fingerprinting with his software.
“I want you to go into a library with 20 million books in it and classify them. By the way, everything is written in Russian. If a book kind of has a dictionary format, you can assume it’s a dictionary. If you see pictures of food, it’s probably a cookbook. But that’s about it. You can’t separate mystery from romance.”
“What Clio is doing, is saying that you can pick up a book, read it and understand the characters. If you’re going to read Moby Dick, what Clio says is ‘hey, the main character’s name is Captain Ahab, he’s got a nemesis, this is what he likes to do, this is the time he’s living in,'” he says.
“It’s a very deep, integrated, powerful process.”
-30-
Every Wednesday, Shop Talk shows you what goes into a tech product, organization or business in the Philadelphia region. See others here.