In partnership with

The music industry is a fascinating place full of existential challenges and incredible opportunities.

Today we meet a startup that wants to open up one of those opportunities to many more artists.

Read on for all about Unit1. And as usual, Premium subscribers get the FULL story, and there’s a lot more to tell, with this one!

– Martin

Get the investor view on AI in customer experience

Customer experience is undergoing a seismic shift, and Gladly is leading the charge with The Gladly Brief.

It’s a monthly breakdown of market insights, brand data, and investor-level analysis on how AI and CX are converging.

Learn why short-term cost plays are eroding lifetime value, and how Gladly’s approach is creating compounding returns for brands and investors alike.

Join the readership of founders, analysts, and operators tracking the next phase of CX innovation.

Subscribe here to stay informed and ahead of the curve.

Subscribe now

Unit1 wants to take music fans on ABBA-style voyages with artists old and new

The world of live music is in a state of flux right now, with small venues closing and touring costs rising. But the other end of the market is thriving. Think of the big, must-see arena shows from world-famous artists.

The biggest spectacle of them all isn’t a live performance at all. ABBA Voyage, the London attraction featuring digital avatars of the legendary group performing their hits, was credited with generating a £1.4 billion boost to the UK economy in its first two years.

Now a startup called Unit1 is looking to bring a similar experience to many more artists. And while you might expect them to be targeting other stars of ABBA’s status, there is potential to make avatar technology relevant to a much greater chunk of the music industry, too.

“What we've done is merged VFX and game production workflow to make the production of avatar concerts much quicker and much more efficient,” says Pavle Mihajlovic, technical director at Unit1.

This is a technical achievement, Mihajlovic says, because of all the things that can go wrong with digital avatars of live performers.

“Anything that you get wrong will immediately break the illusion that there's a person on stage. In a lot of cases, you have digital assets being displayed literally right next to… things like lights and stage props. You're layering real things with digital assets, so it’s incredibly challenging,” he says.

“And you have very long shot lengths, which are basically unheard of in the world of VFX. Even in a Marvel movie where everything looks amazing, their longest shots end up being like 10 seconds, because they'll just cut between cameras. And every time you cut, you get a chance to reset all your simulations, to restart the renders. We get none of those advantages.”

As a demonstration that it has beaten those challenges, Unit1 has created a working avatar performance using the singer-songwriter KT Tunstall.

Mihajlovic says they worked with Tunstall for one day back in April of this year, and by September they had a working demo.

The process for creating an avatar performance can vary. If you’re trying to capture a performer who has retired, you might have to rely on video of them at their peak. Or as with the case of ABBA, you might be able to get a performance from the band but you’ll have to de-age them during the production process.

Tunstall’s case is different. She’s an actively touring artist who wanted to be presented as she is today.

Mihajlovic explains that Tunstall performed a live set wearing a motion capture suit. It was very much intended to capture the experience of one of her shows, in which she builds songs up on the fly with a loop pedal. She also provided some chat between songs.

The motion capture data was then combined with high-quality reference photos to create a digital avatar of KT Tunstall. From here the avatar performance was created, drawing on multiple takes recorded on the day.

Of course, an avatar show is a live experience, and so the stage needs to look the part. Unit1 worked with a professional set designer and a high-end stage lighting company to create a fitting stage environment.

This presents some challenges, Mihajlovic says.

“We have lights in the real world that are pointed towards a digital asset, and have to believably light that digital asset without interfering with the display technology that we're using [and] the sound needs to appear to be coming directly from the artist.”

Because the tech can work with archive footage or aging performers who want to look like fans remember them at their peak of fame, Mihajlovic says the tech is incredibly flexible:

“We’ve created a pipeline that gives us a tremendous amount of manual control over the outputs… Artists are careful about their image, and if it's a rights holder for an artist who's deceased, even more so. We want to be incredibly respectful and incredibly careful.

“With KT's demo, we were tweaking eyelines, because we recorded before we knew the shape of the venue. Once we had the shape, we had to change where she was looking so she's looking at the audience. We can tweak small expressions. If KT wasn't quite happy with any specific element of it, we can tweak it, we can always change it.

“We have an amazing amount of control, and that's super important for this, because it needs to be perfect, both for the rights holders and for the audience, such that they're fully immersed. Every single detail needs to be right.”

The story so far

Mihajlovic began his career in the games industry, as a developer for Peter Molyneux’s company 22cans. He went on to co-found his own games company which created live-action video interactive experiences.

Then, around a year ago, he was introduced to Barney Wragg, a longstanding music industry executive who was keen to explore opportunities in the market for avatar performances.

“What he explained to me is that there's still some work to be done in democratising avatar shows, making it so that other artists and rights holders can also put on avatar shows.” Mihajlovic says.

“His thesis was that you could use a lot of tech from other fields other than just VFX, to make these shows easier to produce, quicker to produce, cheaper to produce. And obviously, games tech comes in big there."

The two duly teamed up alongside creative director Jonathan Butterell and have developed tech to underpin Wragg’s vision, with patent applications filed for some of their innovations.

With the KT Tunstall demo now complete, they’re working to take the next commercial steps for the business.

“We’re showing the demo to rights holders, to artists, to other people in the industry, and looking to take the next step, with an actual avatar show or multiple avatar shows.”

There’s some interesting potential here. We explore that in detail in the ‘Vision’ section below.

Subscribe to Premium to read the rest.