Demystifying Virtual Production Technology

Transcript:

Elise Orlowski:

I’m Elise Orlowski, a senior video director here at Cramer.

Tripp Underwood:

And I’m Tripp Underwood, a creative director at Cramer.

Elise Orlowski:

And at Cramer, we work with so many incredibly fascinating people from all over multiple industries.

Tripp Underwood:

We have so many great conversations, many that are just too good to keep to ourselves. So now we’re sharing them with the world.

Elise Orlowski:

Right here from Cramer Studios.

Tripp Underwood:

This is Pivot Points.

Elise Orlowski:

Cut.

Mark Wilson:

Welcome to another episode of Pivot Points. We’re changing things up a little bit today. I’m not Tripp. I’m Mark Wilson, the executive creative director here at Cramer. And I am joined by my co-host, Pat Drumm, who heads up all things technology with regard to broadcast at Cramer. He’s our senior engineer of broadcast technology.

Pat Drumm:

Thank you, Mark. Today, we are speaking with Brian Olson, Pixotope’s VP of Sales from the North America division.

Mark Wilson:

I personally am very excited to have Brian here because we actually employ the technology, and he is a wizard and a genius. He’s going to tell us all kinds of really amazing, cool stories about where it’s headed. But a little background on Brian. He’s got more than 20 years of experience in virtual technology and graphics. And at Pixotope, Brian and his team are bringing this technology…it’s an award-winning, global, international technology, and he is bringing this software to his clients to help them better tell their stories using augmented reality. So thank you so much, Brian, for joining us. We really appreciate you being here.

Brian Olson:

Hey, guys. Great to be here. Thanks for having me.

Pat Drumm:

All right. Thanks for coming, Brian. So, Brian, for those that may not be familiar, what is Pixotope? Has this technology been used anywhere that our audience may have seen it?

Brian Olson:

So, Pixotope’s a virtual production platform. We utilize the Unreal Engine. We actually incorporate that into our software, and that’s what does the actual rendering. But we’re focused on really designing software that’s for the broadcast and media production industry. So while Unreal is a gaming engine, we are taking that in and adding our own user interfaces and workflows to simplify the process for media creators to do production so they can focus on getting the work done and not messing with the technology so much.

We also recently acquired a tracking company, TrackMen, and we’re integrating that with our software. So ultimately, it’ll be a single user interface that the user deals with, both for rendering and for tracking. So again, the goal is to simplify the experience, to sort of democratize virtual production, make it more accessible for more people so it’s not just the very high-end that are doing it anymore. So more markets, more use cases, because we really believe at Pixotope that ultimately all production is going to be virtual as broadcast and feature films and gaming all sort of merge into just a common media experience.

Pat Drumm:

You talk about accessibility from the user standpoint, and we definitely witnessed that when we were able to create a little iPhone gooey. That one of our guys was able to walk around the studio, and when we wanted something to come in, literally just take his iPhone and touch go in, and the animation would come up and pop up and stuff like that is the kind of thing that will set you apart, for sure, from other kinds of competition when the end user has that sort of simplistic interface.

Brian Olson:

Yeah. And I think it’s even since you guys have had it now, we actually have an HTML5 interface now. So you don’t even need to build an app or use our app. All you need is a webpage to trigger things, which could be on an iPhone, could be on iPad, whatever you want to do, and could be remote as well.

Mark Wilson:

So I love the term to “democratize virtual production.” It seems to me that it’s not just about cost and complexity, but it’s about being able to utilize existing hardware so it’s not so specialized that it’s inaccessible to people who don’t have a ton of budget to spend. Can you talk a little bit more about that technology and where it’s headed? And you mentioned trackerless technology, which I think we’re all pretty excited about because that takes care of a lot of the headache, to be honest.

Pat Drumm:

Yes, it does.

Mark Wilson:

Talk about that accessibility as in availability and where it’s headed.

Brian Olson:

So a couple things. So we don’t have proprietary hardware, which is a barrier, and there’s usually some more cost associated with very proprietary hardware. So it’s COTS hardware, off-the-shelf, that people can use as long as it meets the specs. And really, any kind of a robust gaming PC or a laptop will work right now. So we want to eliminate that factor. We’ve actually been able to run in the cloud without completely virtualized, so either private or public cloud, which is going to create some new workflows. We haven’t commercialized that yet, but we do have that running.

As far as tracking goes, we have a number of ways to track, but the one that’s probably got the most attention right now is what we call through the lens tracking where you’re operating without sensors. It’s really AI. It’s all computer-based AI. You basically take your camera and you learn the environment through the AI. So it looks for reference points like edges of buildings, or walls, or sets and just picks out those points in 3D space. And once it learns the 3D environment, then you can go place things in it, augmented reality items, and they just stick.

One place that you’ve probably seen it a lot lately is in drone tracking that we’ve been doing. So, that was used on the NFL Draft. Some of the sports networks are using it. And that’s really exciting when you can actually fly a drone and then stick logos and different pieces of creative to racetracks, to football stadiums, float above venues. It adds a whole new level of production value, I think, and I think it helps viewer engagement. It also ultimately opens up some sponsorship opportunities too for advertising. Right now, it’s been sort of informational or entertainment-based in a lot of cases that we’ve seen it, both in broadcast and in venue, but I think more and more you’ll see it used for advertising.

Pat Drumm:

Well, I feel like one of the most famous examples of augmented reality is the 1st and ten line, right, when we’re watching football games. And I don’t think I can imagine watching a football game anymore without having that in there. Even when I go to a game, I’m like, “Where’s the line? I need the line.”

Mark Wilson:

Right. First best use of AR.

Pat Drumm:

Yeah. And that technology’s been around for a long time. And I remember-

Brian Olson:

Long time.

Pat Drumm:

… a couple of times that I’ve had to help integrate it for some college football that I was doing, and just massive machines. It creates delay and all this other stuff, and it becomes this production headache that I’m hoping we’re going to start seeing or not start seeing…it sounds like it’s already starting to go away and things are going to start being a lot simpler and easier.

Brian Olson:

It’s much easier than it used to be. And a lot of things can be done optically now instead of with sensors as far as tracking that stuff. Also, the technology to do the keying, I don’t know if you’ve ever tried to watch a football game when it was snowing, trying to keep the 1st and ten going and they’re sort of dynamically adjusting it as more snow falls. But there’s-

Pat Drumm:

Right. The grass is changing color.

Brian Olson:

Exactly. So now there’s stuff besides basic chroma keying, AI keyers that look at difference, that look at movement, that really can do some amazing keying now that wasn’t possible before. So I think that’s changing things and making it a bit easier to do.

Pat Drumm:

Well, you certainly have done a good job at eliminating a lot of the barriers to entry for this kind of technology because beforehand in the broadcast world you’re, like you said, buying dedicated hardware, you’re buying motion trackers, which we’ve invested a lot of that in, but now you’re able just to take any drone off-the-shelf, fly it in the sky, map a couple of points, and all of a sudden you’re putting advertising that you can make money off of it, which is pretty fantastic. And then even again for our standpoint, any off-the-shelf hardware that can do the processing, we’re using hardware that we have anyways that our animators are using to create and generate graphics, and now we’re able to put people in virtual worlds with it.

Mark Wilson:

Those are-

Brian Olson:

Exactly.

Mark Wilson:

… great examples. You’re just making me think. We’re always looking for, what can we do inside a ballroom, outside a ballroom, experiential broadcast. But if we’re able to, in a relatively inexpensive manner, put a drone in the sky and do something for the time period when we have audiences moving from one venue to another venue, when we’re doing things at night, if we’re running a large festival, that just opens things up to us in a pretty dramatic way. I hadn’t really thought about that as a use for Pixotope, but that’s pretty inventive. Because, I mean, we’re all in the business of what’s the next great idea? How can we make the experiences even more robust, even more interesting? And I think having conversations with you and people like you is only going to help us and help everybody who watches this podcast to come up with those ideas.

So what have you seen, what have you done that you’re really excited about in terms of how people are applying Pixotope? I know how we’re using it here in the studio in a couple of different ways, but what really gets you going? It doesn’t have to be so far out there, but just like really cool stuff. We’ve seen your demo page, and you talked earlier when we were warming up about the Ravens, and I love that because if you haven’t seen it and maybe at the end you can give a URL so people can go look at your samples, but it’s a lot of fun. So hit us, what do you like? What have you got?

Brian Olson:

So I mean, actually, we’ve been talking about it. The drone tracking is what’s really exciting for me where I can actually go, I’m in a bar, and I look at a TV and I see drone tracking, and I go, “Yeah, that’s pretty cool.” But I think for venues, I think the mixed reality stuff, the augmented reality in venues where you’ve got something like a raven for the Baltimore Ravens or a panther for the Carolina Panthers running around, jumping from the press box down to the field and jumping through the goal posts. I mean, then you’re getting into feature film-like animation that’s happening in real time for the people in venue.

I think from a production standpoint, you’re probably aware that, at least in a lot of cinematic production now, they’re starting to do stuff with large LED volumes, LED panels. And in lieu of shooting in the field, they basically have camera tracking. And as they move the camera, the point of view changes on the background as it would in the real world. We’ve had some estimates. It may or may not be true that you can save up to 60% on a production by shooting XR, extended reality, versus shooting in the field. Obviously, there’s the initial investment.

Mark Wilson:

Well, I think this podcast right now is an excellent example of that. It looks like we’re at our bar, Red’s, but-

Brian Olson:

I thought you guys were in a real studio there. It’s going to be-

Mark Wilson:

We are in a real studio. We’re just not at Red’s, which we should be.

Pat Drumm:

We should be at Red’s. Yeah.

Mark Wilson:

But I think your democratization, virtual production, you just talked about Hollywood, that’s something that we’re doing here. This is a simple example of using a backdrop to create a little bit of an illusion of being in a space, but it’s also something that we’re playing with constantly. Mandalorian is always the example everybody uses of the virtual production process, but I think one of the things I’m sure you’re seeing is how people are using the Pixotope technology and Unreal Engine to both create backgrounds that give you the ability to parallax with camera pretty easily but put you in an environment, and you add a few practical elements and people and, man, that’s an illusion that looks fantastic. You sell the illusion so much easier.

Brian Olson:

Especially if you have shallow depth of field where the background’s pretty blurred anyway. You can sell it big time. And if they actually mix a shooting in the field with shooting with XR, then it takes it to another level too. And the average person just isn’t going to know. I mean, it’s that good right now. I think that’s super exciting. And things that are not possible practically, they’re going to be able to do virtually now in some cases without green screen if they’ve got the big XR stages. And I think that that sells even better than green screen does.

Mark Wilson:

And I like the fact that we’re talking about doing it economically, small, medium, large, extra large. We’re talking about big venues like stadiums and arenas, but we also want to talk about ballrooms and theaters, and then we want to talk about pure production.

Pat Drumm:

Yeah. I think a lot of that is just demystifying this technology and just figuring out, “All right, how does it apply to me? How can I use it in my own production world?” and just being able to do that sort of thing.

Mark Wilson:

A question that we were talking about before the show is, who’s leading out there? So are there sectors of the industry that you’re seeing that are really pushing this along, that are trying more, that are adopting it more readily and more quickly? What are you seeing out there in terms of industry leaders and industry sectors?

Brian Olson:

So there’s the industry sectors and then there’s also the geographies. So with industry sectors right now, most of what we’re doing, or a good share of what we’re doing, is events, events in sports. And they’re using augmented reality, mixed reality to add another layer to the experience for the viewers, for the fans. That’s where we see a lot of it. And as far as virtual production where you’re using virtual sets, set extensions, that type of thing, in North America we’ve probably been the slowest to adopt of any region with respect to that, and I would say Asia is probably the fastest. So I expect to see a lot more virtual production beyond just augmented reality in North America going forward as it becomes more commonplace to do production this way.

Pat Drumm:

So we operate in a world where I feel like a lot of our people, our techs, our freelancers, they like to trust in the things that they know. They have a show that they have to make air or to please a client, whatever. And you’re only as good as your last show. So what would you say to the people who need convincing that this is worth trying and going and taking a risk for?

Brian Olson:

I mean, that’s the tough thing, whether somebody’s been burned by virtual production in the past or it’s just completely new and totally foreign and scary. But what I would say to an average customer is given the use cases of where this is being used and the things that are possible, and we as a company try to support our customers not just in pre-sales but after the fact as much as we possibly can and help them grow, it’s usually baby steps. Let’s start small and work our way up. Let’s not build Star Wars on our first production here. Let’s do a little bit of augmented reality or something. And then as they get more comfortable with it, they’ll do more with it and start to see the possibilities.

Mark Wilson:

So if I were going to summarize, your message really is come back, try it again. Because I think the message here is set your imagination free. Because anything that you can do in Unreal, anything that you can think of, I mean quite literally we can now bring to the screen and we can do it quickly, and we can make edits to it, and we can do it in a way that just really, it’s practical, it’s functional, and it’s pretty awesome. I mean, what we can do now is very, very powerful, especially compared to what we used to be able to do. So let’s all have some fun together because it’s certainly where we’re at.

Brian Olson:

The operative word is fun.

Mark Wilson:

And, Brian, we’re coming to the end of the program here. It’s really all the time that we have. I want to just say thanks again so much for joining us, we really appreciate it, Pat, your expertise. Thanks, everybody. And this has been another episode of Pivot Points.