AnsweRED Podcast Episode 29 — Capturing Movement: Performance and Motion Capture [transcript included]

Paweł B: Hello and welcome to the AnsweRED Podcast. People sometimes say you should eat a mic, and I will eat a mic made of sugar. This is gonna take a while. Hello and welcome to the AnsweRED Podcast. My name is Paweł Burza and I am joined by Kelpie today and of course by Paweł Mielniczuk, my friend and co-host, the Art Director for Project Hadar. Hello, Kelpie.

Paweł M: Today we'll dive into the magical world of motion capture, together with Paulina Lendzion, Motion Capture Producer and Tomasz Kowalczyk, Motion Capture Lead.

Paweł B: It's going to be a great episode. Can't wait. So let's meet our guests. Paulina, Tomek, welcome to the podcast officially! This is going to be one of my favorite episodes, I can tell you that already because we're talking about motion capture. Could you guys introduce yourselves?

Paulina: Hi everyone, I'm Paulina. I've been at CDPR for almost four years now and together with Tomek, we are in charge of the Mocap team.

Tomasz: So hello everyone, I'm Tomasz and I'm Motion Capture Lead, and I've been working with people in funny suits for around 15 years.

Paweł B: Wow, 15 years. That puts it into perspective.

Paweł M: He's a dinosaur.

Paweł B: Yeah, that's what we call old REDs here. Like they're dinosaurs; they remember everything. 

Paweł B: So yeah, let's talk about the stage, the setup that you have right now. Of course, without going into trade secrets and stuff like that, because I know each motion capture studio does things their own way. Let's talk about the general setup and the day-to-day operations or shoots that you do for specific titles and all that jazz.

Tomasz: So I can start about technicalities and later maybe you can explain how it goes during shooting. We are talking about a motion capture stage, mocap, and so on. So let's explain what that is. We are talking about a kind of big space. It's a big room with a truss around with 74 cameras. And as you mentioned, we are recording everything at the same time. This is not like we are changing cameras — "Oh, and right now, this is this one, we are recording from this one and probably in a second you change it." We are recording every single place, every single point of the environment at the same time. So we can really, really easily change the cameras and adjust it later because we are recording 3D animations all at once. Our current volume — we call the place where the actor is visible and covered by the cameras the "volume" — is around 50 meters by 7 meters. So this is the place where we can record walking, jumping, running, and all kinds of cutscenes, like — spoiler alert — when Jackie dies. Everything we can record in that place. So it's a kind of big room, definitely bigger than the current toilets, and we can definitely do a lot of things. Everything for Cyberpunk was recorded in the current stage.

Paweł M: But it's not only the room, you also need all those tools and elements to build scenographies during the session, right? So what is the facility around the mocap studio that allows us to do that?

Tomasz: So okay, we have a changing room for actors, that's obvious, and a green room, which is a place where actors can spend time when they're not on stage.

Paweł M: Why is it called a green room?

Tomasz: It's called a green room or a lounge room.

Paulina: A break room. Like in TV and stuff like that.

Paweł B: I know the color is not green, but green room is the official naming.

Tomasz: Yes, there is no green screen inside, no, no. This is also an important thing: on our stage, there is no green screen. We are not rotoscoping anyone, we are not changing the environment; we are recording people.

Paweł B: It's very gray. Yeah, actually it's very gray.

Tomasz: But to continue about the facility and the most funny part: we have a huge workshop with a lot of workbenches and 3D printers.

Paulina: I would say it's a playground for people who actually like to play with tools.

Tomasz: Definitely, yes!

Paulina: Yeah, we actually have a part of our team which is responsible for creating props, so it's kind of like their own playground. They just spend whole days there just creating stuff.

Paweł M: So what, for example, would they build?

Paulina: Whatever you want!

Tomasz: Okay, I think a good example is maybe that famous tank for Cyberpunk, because we had to record a whole sequence inside. We checked the construction because it had to be really stable — all the romance scene inside had to be there, and it had to be safe. So we prepared everything and checked that it was safe and stable. It couldn't interrupt the actors because they are in really intimate scene positions, and things like a piece of wood or additional foam here or there might be problematic for them. So we had to check it.

Paweł B: So you have to construct things in terms of a space, but you also create things that actors can use in terms of holding them, right?

Paweł M: I imagine those things need to perfectly match what will be visible in the game, like the geometry in the game. But at the same time, you need to make sure that all the cameras still see those people, right? So it's not the real tank.

Paulina: I would say it's a challenge to build all of the closed spaces. If you think, for example, about a lift or an elevator, and you have a bunch of people there, it's really tricky to build because the markers on the body need to be visible by all of the cameras around, right? So it's very challenging to come up with how we should build this and what materials we should use so they are not reflecting or blocking light, allowing the cameras to actually see the markers. It is very challenging sometimes to come up with something that is useful but also very safe and is a representation of the model that is in the game. As you were saying, we are kind of discovering 3D printing right now.

Tomasz: Super cool! Yeah.

Paulina: So probably Tomek can say more about it. But yeah, we've checked and it's proving to work. It's actually pretty cool because you can actually print stuff that is basically the same prop or asset that you see in the game. So if you're drinking from some cup or using some other tool, it actually looks the same. It is also easier for the actors to imagine, "Okay, I am using this thing," rather than having a piece of a box or wood that they need to imagine is an axe or something else. It's actually a real thing.

Paweł B: Or at least a replica.

Paulina: A replica, yeah, yeah, yeah.

Tomasz: And it's exactly from the game, so we can print things like runic signs or something like that. It's the actual thing, the proper size, and more or less the proper weight sometimes. But as I said — safety first, I say many times safety first — sometimes we prefer to reduce the weight just to be sure that everything is fine and safe, and stunt performers or actors are not harming themselves.

Paulina: I would say what can be interesting sometimes is that we actually have a creative power to design some of the assets in a game because it happens that we record some of the animations in advance. Since some of the assets are not really designed yet, we have situations — not often, but sometimes — when an asset artist reaches out to the Mocap team and says, "Hey, what did you use for recording this? Because now I have to design it." So it's kind of the other way around than it should be, but you know.

Tomasz: An example, of course, is the spider bot from the quest with Maelstrom. We got information that we had to record a machine in a box. We asked, "What kind of box?" and they said, "We don't know yet, it will be designed in the future." So we bought a bucket in one of the stores and said, "Okay, it's more or less like that." We recorded everything. After half a year, they said, "Okay, this is the time we are preparing the proper box. Do you have the bucket? Because we will measure it and recreate everything." So that was another way.

Paweł M: I guess there's a lot of creative process where you have to instantly figure something out. Does that often happen that you have to act quickly? Like, something's missing, some prop is missing, or there's some kind of unforeseen scene you have to record and you have to come up with props on the fly?

Paulina: I would say it depends. Usually we get a list of props that we need to prepare in advance. But sometimes, if you're actually playing with each other performing a scene, some stuff may come up that you weren't ready for, and the director can say, "Hey, I really like this, let's use something else." That is a moment when you actually need to figure out stuff on the fly, and then yeah, we need to act fast.

Paweł M: Is there room for improvisation during the recording?

Paulina: Oh yeah. Definitely. Definitely.

Paweł B: I think we should bring up the wok story from Cyberpunk. You guys needed a wok to make food in, right?

Tomasz: Yes! There was a case where we had an additional half an hour or something like that after recording everything. We said, "Okay, we have some spare time and we have one additional animation we would like to record — a vendor somewhere on the market who is preparing food in a wok." We said, "Nice! And... what?" They said, "And we need a wok." But how? This is not like a cube or a regular pan that we can just borrow from our canteen. It's a wok! We had half an hour, and we started running around looking for something similar. We realized that our lamp had a similar shape, so we removed all unnecessary elements and said, "Okay, is it good enough?" They said, "It's perfect! This is exactly the shape we need! So yeah, feel free!" I mean, thank you very much!

Paweł B: Think of a half-sphere that just covers the lamp to make the light a little bit less hardcore. We just took it apart and used it, and actually, if you think about it, it does look like a wok.

Paweł B: So yeah, let's talk about the stage, the setup that you have right now. Of course, without going into trade secrets and stuff like that, because I know each motion capture studio does things their own way. Let's talk about the general setup and the day-to-day operations or shoots that you do for specific titles and all that jazz.

Tomasz: So I can start about technicalities and later maybe you can explain how it goes during shooting. We are talking about a motion capture stage, mocap, and so on. So let's explain what that is. We are talking about a kind of big space. It's a big room with a truss around with 74 cameras. And as you mentioned, we are recording everything at the same time. This is not like we are changing cameras — "Oh, and right now, this is this one, we are recording from this one and probably in a second you change it." We are recording every single place, every single point of the environment at the same time. So we can really, really easily change the cameras and adjust it later because we are recording 3D animations all at once. Our current volume — we call the place where the actor is visible and covered by the cameras the "volume" — is around 50 meters by 7 meters. So this is the place where we can record walking, jumping, running, and all kinds of cutscenes, like — spoiler alert — when Jackie dies. Everything we can record in that place. So it's a kind of big room, definitely bigger than the current toilets, and we can definitely do a lot of things. Everything for Cyberpunk was recorded in the current stage.

Paweł M: But it's not only the room, you also need all those tools and elements to build scenographies during the session, right? So what is the facility around the mocap studio that allows us to do that?

Tomasz: So okay, we have a changing room for actors, that's obvious, and a green room, which is a place where actors can spend time when they're not on stage.

Paweł M: Why is it called a green room?

Tomasz: It's called a green room or a lounge room.

Paulina: A break room. Like in TV and stuff like that.

Paweł B: I know the color is not green, but green room is the official naming.

Tomasz: Yes, there is no green screen inside, no, no. This is also an important thing: on our stage, there is no green screen. We are not rotoscoping anyone, we are not changing the environment; we are recording people.

Paweł B: It's very gray. Yeah, actually it's very gray.

Tomasz: But to continue about the facility and the most funny part: we have a huge workshop with a lot of workbenches and 3D printers.

Paulina: I would say it's a playground for people who actually like to play with tools.

Tomasz: Definitely, yes!

Paulina: Yeah, we actually have a part of our team which is responsible for creating props, so it's kind of like their own playground. They just spend whole days there just creating stuff.

Paweł M: So what, for example, would they build?

Paulina: Whatever you want!

Tomasz: Okay, I think a good example is maybe that famous tank for Cyberpunk, because we had to record a whole sequence inside. We checked the construction because it had to be really stable — all the romance scene inside had to be there, and it had to be safe. So we prepared everything and checked that it was safe and stable. It couldn't interrupt the actors because they are in really intimate scene positions, and things like a piece of wood or additional foam here or there might be problematic for them. So we had to check it.

Paweł B: So you have to construct things in terms of a space, but you also create things that actors can use in terms of holding them, right?

Paweł M: I imagine those things need to perfectly match what will be visible in the game, like the geometry in the game. But at the same time, you need to make sure that all the cameras still see those people, right? So it's not the real tank.

Paulina: I would say it's a challenge to build all of the closed spaces. If you think, for example, about a lift or an elevator, and you have a bunch of people there, it's really tricky to build because the markers on the body need to be visible by all of the cameras around, right? So it's very challenging to come up with how we should build this and what materials we should use so they are not reflecting or blocking light, allowing the cameras to actually see the markers. It is very challenging sometimes to come up with something that is useful but also very safe and is a representation of the model that is in the game. As you were saying, we are kind of discovering 3D printing right now.

Tomasz: Super cool! Yeah.

Paulina: So probably Tomek can say more about it. But yeah, we've checked and it's proving to work. It's actually pretty cool because you can actually print stuff that is basically the same prop or asset that you see in the game. So if you're drinking from some cup or using some other tool, it actually looks the same. It is also easier for the actors to imagine, "Okay, I am using this thing," rather than having a piece of a box or wood that they need to imagine is an axe or something else. It's actually a real thing.

Paweł B: Or at least a replica.

Paulina: A replica, yeah, yeah, yeah.

Tomasz: And it's exactly from the game, so we can print things like runic signs or something like that. It's the actual thing, the proper size, and more or less the proper weight sometimes. But as I said — safety first, I say many times safety first — sometimes we prefer to reduce the weight just to be sure that everything is fine and safe, and stunt performers or actors are not harming themselves.

Paulina: I would say what can be interesting sometimes is that we actually have a creative power to design some of the assets in a game because it happens that we record some of the animations in advance. Since some of the assets are not really designed yet, we have situations — not often, but sometimes — when an asset artist reaches out to the Mocap team and says, "Hey, what did you use for recording this? Because now I have to design it." So it's kind of the other way around than it should be, but you know.

Tomasz: An example, of course, is the spider bot from the quest with Maelstrom. We got information that we had to record a machine in a box. We asked, "What kind of box?" and they said, "We don't know yet, it will be designed in the future." So we bought a bucket in one of the stores and said, "Okay, it's more or less like that." We recorded everything. After half a year, they said, "Okay, this is the time we are preparing the proper box. Do you have the bucket? Because we will measure it and recreate everything." So that was another way.

Paweł M: I guess there's a lot of creative process where you have to instantly figure something out. Does that often happen that you have to act quickly? Like, something's missing, some prop is missing, or there's some kind of unforeseen scene you have to record and you have to come up with props on the fly?

Paulina: I would say it depends. Usually we get a list of props that we need to prepare in advance. But sometimes, if you're actually playing with each other performing a scene, some stuff may come up that you weren't ready for, and the director can say, "Hey, I really like this, let's use something else." That is a moment when you actually need to figure out stuff on the fly, and then yeah, we need to act fast.

Paweł M: Is there room for improvisation during the recording?

Paulina: Oh yeah. Definitely. Definitely.

Paweł B: I think we should bring up the wok story from Cyberpunk. You guys needed a wok to make food in, right?

Tomasz: Yes! There was a case where we had an additional half an hour or something like that after recording everything. We said, "Okay, we have some spare time and we have one additional animation we would like to record — a vendor somewhere on the market who is preparing food in a wok." We said, "Nice! And... what?" They said, "And we need a wok." But how? This is not like a cube or a regular pan that we can just borrow from our canteen. It's a wok! We had half an hour, and we started running around looking for something similar. We realized that our lamp had a similar shape, so we removed all unnecessary elements and said, "Okay, is it good enough?" They said, "It's perfect! This is exactly the shape we need! So yeah, feel free!" I mean, thank you very much!

Paweł B: Think of a half-sphere that just covers the lamp to make the light a little bit less hardcore. We just took it apart and used it, and actually, if you think about it, it does look like a wok.

Paweł M: But I guess this is a rare case, right? Usually the preparation goes way back before the session starts. How does that work? How do you prepare, collect all the data, build props, and decide who to invite to the session?

Paulina: I would say it's a lot of cooperation with other teams, especially with Animation, Cinematic Design, and gameplay teams, of course, because in the beginning you prototype a lot. You record combat-style animations, check your ideas, see if you need to improve something, or maybe record something else. In the beginning, it's basically about hearing the expectations and listening to what they actually want to record. It starts with a meeting and the other team requesting the animation. They come to us, or we reach out to them sometimes if we know that the project needs animation. Then we talk: "What needs to be recorded? Is it gameplay animation? Is it open world? Is it a cinematic scene?" Then we start talking and get the list of props that we need to prepare and the list of animations later in the process. First of all, we decide, "Is it stunt material? Is it actors' material? Should we maybe work with some other crew to prepare some of the stuff?"

One of the examples is the scene presented in the Unreal tech demo that we recorded. It was the scene in the beginning when they're in the carriage with the merchant and—

Paweł B: With the merchant and the, as Tomek would call it, washing machine. Hamster in a washing machine!

Paulina: It was kind of like that because we actually worked with an external crew to work on the whole piece of that thing. We were shaking the carriage rig to record the real movement of that. In the beginning, we knew this is the kind of stuff we need to record, and that it's very heavy with a lot of prep time that we need to put into it. Of course, the other thing is if you actually have this time for preparation, but you know, that's on the side.

Paweł M: What kind of advance do they need to book the mocap studio? How much sooner does the team have to reach out to you to let you know?

Paweł B: Because you guys are busy!

Paulina: We are! Yeah, I should probably answer that because it was different when we worked on one project, but since we switched to multi-project development, it's been a little bit busier. That's why we also try to invest more in the Mocap team, and as Tomek said, some new things are coming. So yeah, a few months in advance for sure. We are talking about months.

Paweł B: Because I think we also should put it into perspective. If you think about how many recordings you did — because you recorded everything in this mocap studio for Cyberpunk — how many recording days is that? Let's put it into people's heads so they know the large scale and volume of this thing.

Tomasz: When we are talking about Cyberpunk, that was more than 300 — to be more precise, that was actually 311 shooting days. Yes, I was counting.

Paweł B: That's almost a full year of shooting, pretty much, if you take that into account.

Tomasz: And that was only one project. Also, we had a chance to record some things for GWENT, that was also part of our job, and right now, Ciri and Witcher and other projects. You can multiply this 311 days by the other projects, which is why we are fully packed.

Paulina: I feel like at this moment, I need to tell you about Tomek's superpower, because Tomek remembers almost all of the stuff that we recorded on every shoot. He's basically a bank of recorded animations.

Paweł B: Mr. Library. Yeah, exactly!

Tomasz: Actually, yes. I remember my first shooting day. But yeah, that is helpful because sometimes different teams want to record animations like, "Yeah, we need this or that," and I say, "Give me two seconds because I'm pretty sure we recorded it somewhere, and I can give you the data." So yeah, I use that superpower, hopefully in a good way.

Paweł M: Going back to the shooting days, is it like on Monday you're recording Cyberpunk, on Tuesday you're recording Witcher, etc.? Is every day a different session? Because building these props and setting up the studio must take some additional time.

Tomasz: It might be. We can record the whole week for Witcher, or do Cyberpunk on the first day, then Witcher, Cyberpunk, Witcher, or something for marketing purposes, and so on. We are fully flexible and we can adjust, so this is not a big deal for us and we can prepare ourselves. Also, a really important thing is that because we are recording Witcher, we need a lot of swords for a Witcher game, so we have them. For Cyberpunk, we had a lot of different guns. Some props are prepared in advance because we know we'll have a blacksmith, so we need an anvil and a hammer. We will have someone who is fishing, so probably a rod will be necessary, so we can do it in advance. If they say, "Yeah, we need to record a blacksmith," we can do that easily because we've prepared it before.

Paweł B: Sorry to cut you off. Whenever you visit mocap you can tell more or less what they're working on because before you enter the stage, if you look to the right, you see that there are actually different things. During Cyberpunk times, it was just full of weapons — it was just guns, different guns, knives, katanas, stuff like that. Now if you enter, it's totally different. You see a lot of swords, you see us having shields there, a rake, for example, stuff like that. There is a lot of different things, and also things which don't resemble anything — it can be anything, like mannequins or animal parts.

Paweł M: I remember those crazy legs for... I think a long time ago in The Witcher 3, we used this additional leg setup where you put your leg—

Paulina: Leg extensions. Leg extensions.

Tomasz: Arm extensions — we still have them. It's crazy.

Paweł M: That's why when you enter this place, you feel like Alice in Wonderland. All the toys.

Paweł B: Everything's pretty much possible. But what I also really like is the props which help actors do certain motions which they wouldn't be able to do otherwise. Think, for example, of a sword without a blade — just a hilt — so you can actually stab someone and it looks realistic. Of course, you later recreate that in-engine and it looks like a whole sword, but you're just using the hilt to pierce through someone. I feel like you need to be thinking in a very creative way about how you do things because you have to improvise, like, "How do we make this look realistic? Well, we need to cut the blade off and then it's going to work," right? But there are probably more examples of stuff like that.

Tomasz: Yes. We have things like... we call it "a thing," which can be literally anything.

Paweł B: If you could imagine a heavy object that you hold in your hand that is taped up all around and has these funny markers on it so the cameras can see it. There is a smaller thing and there is a bigger thing.

Tomasz: And it can be a monster's heart, a Witcher's grenade, a rock, or a Witcher's trophy of some kind.

Paulina: Whatever you imagine.

Tomasz: And it is actually the thing because it has a shape — but what kind of shape? It's kind of round, but not exactly. The most important part is that it's heavy and you can grab it in one hand, so it can be anything. Later, we can put the proper model on that. So as I said, that can be a grenade, a heart, and so on.

Paweł M: All right. Okay. So someone reaches out to you because they need some animations, you have a list of animations, a list of props, and all the setup you need, and then you need people. Who are the actors performing? You already mentioned that it depends on whether that's gameplay or cinematic. Where do you get those people?

Paulina: Basically, we usually... it depends on whether we are recording mocap or if we are recording performance capture, right?

Paweł M: What's the difference?

Paulina: For performance capture, we are recording voice at the same time as we are recording the animation. For motion capture, we are only recording animation — so only the movement, right? It's not that important to be in a quiet environment, quiet setup, and all of that. For performance capture, we usually work with English natives, so right now we work mostly with a UK cast. We're probably going to talk about it later a little bit more.

When it comes to a regular shoot, which is a motion capture shoot, we usually work with Polish actors because it's basically the easiest. Most of them live in Warsaw, so it doesn't take a lot of time for them to travel to the studio. There are people who have worked with us for a very long time — I would say some of them have worked with us for like 15 years already, or even more. We usually know who is good at what, so if we are recording some open world, we know in advance who we would like to work with. If there is a specific need — like some sessions that are unique and require specific performers — we look for them. A good example would be the Polish sumo team that we worked with, or the parkour athletes and gymnasts. Depending on what you need, we try to look for the performer that would be good at it, or someone we want to try a specific solution with. We usually work with agencies or basically with actors we've worked with before, and sometimes people just come to the studio and leave their portfolio or send it to our email address.

Tomasz: And I think we have to remember also about stunt performers because they are a very important part of the production. In that case, we are not casting or working with a crew from the UK or US because we have our own friends who have been working with us for more than 15 years. They know exactly what's going on, and we don't have to explain the technicalities. We just say, "Okay, you're Geralt. Be Geralt for the whole shooting day." That is definitely helpful when we are doing a lot of animations; they know the drill.

Paweł M: And having in mind all those animations you recorded, is it possible that you look at an animation, a scene in the game, or just an open-world animation, and you can tell which person did the mocap for that?

Tomasz: Um... yeah. Actually, yes.

Paweł M: Because people do have a very unique way of moving, right?

Tomasz: Yes, yes.

Paweł M: Yeah.

Tomasz: Yes. And also, this is the reason why we try to have a performer for a specific character and role. At the beginning of Cyberpunk, we recorded Jackie with three different actors, and still, I am able to see who was playing when and in which cases. In those moments, you can see the inconsistency where in one scene Jackie is heavy and focused, and in another, he's energetic and so on. So right now, we would like to have one performer just to keep that style and mood consistent.

Paweł M: Repetitive performance.

Tomasz: Exactly, exactly. So I'm able to see it, and I'm sure not only me, but gamers can see it too, like, "Why is he so hyped?"

Paweł B: Yeah! "Why is he moving differently?"

Tomasz: Exactly!

Paweł M: Like, "Why is he jumping all around?"

Paweł B: Yeah, "Too energetic. That's not him for sure."

Tomasz: So I can see that, and I know the reason, but gamers do not always have that kind of knowledge.

Paweł M: Does it sometimes happen that actors are not aware of what exactly they're recording for? Like they don't know specifics about the projects?

Paweł B: Or do they just know the motion they're supposed to show, like, "Oh, do two somersaults and end it with, I don't know, something," and you don't tell them what it is specifically for? Or do you tell them specifically that this is for this game and this scene? How much insight do you give them?

Paulina: Yeah, a lot, I would say. Basically, usually at the stage of casting, we don't tell them what project we are casting for or what they are going to be actually performing. We also sometimes use code names for the characters because we just don't want people to talk about it or know that CDPR is casting for a specific project. So we use code names. But when we meet with them in person, then we tell them, "Okay, this is what we are casting for, or this is what we are recording for."

Paweł B: After signing an NDA?

Paulina: Of course, after signing an NDA. I just feel — and I guess my team would also agree — that it's just fair, because how can someone do a good job if they don't know who they are playing or performing as? So they usually know the context. Sometimes, of course, we ourselves don't know the whole setup or the whole scene because it's being iterated, rewritten, and reviewed, but we know the intention we are going for. We tell them, "This is the character you're playing, and we expect you to perform this or that," so we try to share as much as we can.

Paweł M: And does it always require an actor, or do sometimes people from the team perform as well?

Paulina: Tomek himself! We just wanted to tell you guys that Tomek was actually Geralt, yeah.

Tomasz: No, I wasn't! But I had a chance to play Jackie in a few scenes—

Paweł M: Which Jackie?

Paweł B: That's why he's been mentioning Jackie all the time! He loves that character, he soaked it up.

Paweł M: But which version of Jackie? You know, like—

Tomasz: Alive version of Jackie! But we decide to hire professionals, and that's actually our main focus. Sometimes, when we need to prototype something really fast to be sure whether a character should be bigger or smaller, we ask people from the studio to do it, but that is mostly for pre-visualization. For the final production data, we focus on hiring stunt performers, actors, or people who can do weird things, or who have special abilities like sumo fighters or an extreme—

Paweł M: Swordmaster.

Tomasz: Swordmaster. Exactly, that's a great example. So if someone from the studio is a swordmaster, then yeah, feel free! You're invited! If someone from the studio has a special ability.

Paulina: I mean, we have worked a little bit with people from CDPR. Usually, when we look for something specific, climbing can be a good example. Because, what, we’re going to reach out to a climbing club or something like that? When we search for some specific skill we put a message to our folks: "Hey, is anyone interested or would like to join the session?"

Tomasz: But in regular cinematics or gameplay, we focus on working with professional stunt performers and actors.

Paweł B: If you ever need someone riding a bike, let me know.

Tomasz: Sure!

Paweł B: Yeah, a cyclist somewhere. Yeah, that'd be cool.

Paweł M: And the animal actors?

Paweł B: Yeah, I was going to say, what are the weird or out-of-the-ordinary things that you recorded? Because we know humans — we have athletes, stunt actors, normal actors — but you've also done some animals, if I'm not mistaken.

Paulina: Yeah. Tomek, you wanna?

Tomasz: Yes! We recorded a dog as our first animal.

Paweł B: A well-trained dog.

Tomasz: An extremely well-trained dog. That was a pleasure, literally, to work with this dog. And later we had the chance to record a horse. Actually, that was not just a chance, that was a full, almost one-week shoot with a horse for the upcoming Witcher game.

Paweł M: Great. Did it happen in our studio?

Tomasz: Not in our studio, but with our cameras and our gear.

Paweł M: Oh, right.

Tomasz: We had to disassemble the whole studio, pack it up, go to the stables, reassemble everything, calibrate, prepare the horses, record everything in a few days, and then once again disassemble everything, pack it, bring it back to the studio, clean everything, and reassemble. A really easy process!

Paweł M: And calibrate. I heard that calibration is not the easy part there.

Tomasz: Yes, actually, this is the magical word because if we ever have to explain something, it's always, "Yeah, we have to calibrate."

Paweł M: Lunch break!

Paweł B: "Yeah, we're calibrating right now." Yes.

Tomasz: So it was a kind of big thing with the horse, but we are really proud of it.

Paweł M: And what we saw so far on Unreal Fest, that was—

Tomasz: Yes.

Paweł M: —amazing.

Paulina: That was a real horse recorded, yes. No harm was caused, which is worth mentioning.

Paweł B: I think that's the most important thing. We could not bring the horse in because we have stairs here and horses don't really do stairs, so taking everything to the stable was a big achievement. I remember everybody was gossiping in the corridors of the studio, like, "You know what Mocap did?" "What?" "They took the cameras to a stable and they're recording a horse." "A horse, really? For what?" "For Witcher 4." We were like, "Oh my God, they're recording a horse!" It's something out of the ordinary that you don't even think is possible, but in the end, it actually is.

Tomasz: For us, that was also a big deal, and as I said, the whole team was really, really proud of that.

Paweł B: Yeah.

Tomasz: Yeah.

Paulina: I would say we put a lot of preparation into making that happen. As you mentioned, the horse was actually here at the campus because we wanted to do some initial tests to check if the ideas we had and the way we imagined it would work were actually going to work. We wanted to check and test it before we spent more money on organizing the full session.

Paweł M: I guess the horse was not wearing a suit?

Tomasz: No, but that was our Plan A, and—

Paulina: We actually have a horse suit!

Paweł B: Wow! 

Paulina: Yeah, yeah, yeah.

Tomasz: And it looked like Batman, but a horse.

Paweł B: Bat-Horse!

Paweł M: You put two people inside?

Tomasz: No! But, you know, it was pink, so we decided it didn't look good in the camera. We prepared everything with veterinary Kinesiotape; everything was checked by vets to ensure safety first. Secondly, if it's designed for a horse, then probably it will work.

Paulina: It's going to work. Yeah. As you were asking at the beginning about how much time in advance we need to prepare something: this is the thing. For the horse session, it took us at least one year of prep. It was initiated with, "Okay, we want to record a horse animation, start thinking about it." Then we started thinking about it, looking for a space, and figuring out how we wanted to attach the markers to the animal and what kind of animal we wanted to have. We worked with trained horses that perform on movie sets in Poland, and we had a trained crew. It took us a lot of time to come up with all of this stuff and iterate on the ideas.

Tomasz: And how to recreate the studio—

Paulina: In a harsh environment.

Tomasz: Exactly, exactly.

Paweł M: Because the cameras are usually placed on frames so that vibration from the walls doesn't affect them, right?

Tomasz: Exactly. The cameras are just hanging and they are in one place — please don't touch them, everything should be safe. And we are talking about stables, so a lot of animals—

Paulina: A lot of dust.

Tomasz: Dust and so on. Because people are riding horses, surprisingly! So, how to design everything to protect the cameras, and from the other side, how to record the horse because the horse should feel—

Paweł B: Comfortable in its own environment.

Paulina: Natural.

Tomasz: Exactly.

Paweł B: We talk a lot about the suits, but could you explain to someone what a typical mocap suit looks like? Because it's also very interesting; we can totally tell in the studio when you guys are recording because when the actors have a lunch break, they come in wearing these funny suits and eat lunch with us. Could you explain how that works and how the camera actually sees it?

Tomasz: Okay, I can explain. The easiest way to explain is that it looks like a really tight pajama made entirely of Velcro so we can easily attach the markers. We've mentioned this word "marker" a few times. What is a marker? A marker is a spherical piece of rubber covered with reflective tape. That's it, really. They are placed on a performer's body — for example, on the top of the head, on arms, or on legs — just to inform the system where the actual animation bones of the actor are.

Paweł M: Joints.

Tomasz: Joints, thank you. This is kind of weird to explain, it's much easier when you can see it. Every marker is responsible for giving us info about the rotation of the joint and the length of the bone. A combination of markers can explain the length of, for example, the whole arm. One marker is just a point, but three markers can tell us, yeah, this is a hand, this part is a head, and this part is responsible for being a foot. So we need some markers on the suit.

Paweł M: And the cameras are probably infrared, right?

Tomasz: Yes.

Paweł M: They see only the markers, and it works like GPS, basically. So you need at least three cameras to see one point to triangulate the position. But why so many cameras, then?

Tomasz: Yeah, because actually, you need only two cameras. Because we, as human beings, have only two eyes and we can navigate more or less in 3D. In edge cases, we can use only two cameras, but three cameras can focus on a small space. If you want to record runs, different cameras are responsible for different parts of the environment. For example, three or four cameras are responsible for the entrance part of the stage, and they are not focusing on the other side of the studio. We need plenty of them to be sure that every point in the environment is covered by a camera, even if we are preparing environments like tables, chairs, or cars. Even if some part of the environment is covered by scenography, other cameras can capture it.

Paweł M: Like the tank.

Tomasz: Exactly, like in the tank.

Paweł M: Okay, all right. So that's for the body. What about complex objects like hands or mocapping the face? We all know the pictures from Avatar and the recording of people with dots on their faces, cameras, and helmets.

Paweł B: I think it's also very popular now in games. Everybody talks about performance capture, and they always show these actors with this blue light on them with markers and stuff like that. How about complex things, like Paweł said?

Tomasz: Paulina mentioned performance capture, and you also asked about that. Yes, we have HMCs (head-mounted cameras) — helmets with cameras — and we are recording faces. That's why we record with English natives, because we want to record their bodies, their faces, and their voices all together in sync. When you record everything in sync, it is much easier to get the proper animation and you don't have to focus on combining different inputs like voice, face, and body. We are recording that, and we also have gloves. This is a slightly different system from different providers, but you have a special glove with magnetic sensors, and you can record the changing of fingers' positions. Sometimes it's easy to synchronize, sometimes not, but also we are working on that to get the full performance capture inputs.

Paweł M: Okay, okay. Because I remember usually when you see the mocap data, the hands are stiff, right? But it's because we have, for example, some overlay animations for animating the hands, or we have animators who make the hand performance separately later in production. Or, if you're using the "thing" prop and you don't have the real prop, the animator has to adjust the hands and the fingers to whatever the character will actually be holding later in the process.

Tomasz: Yes. Previously, the solution which worked pretty well with a full body sometimes had issues with fingers. That's why we have a separate system only for fingers. Sometimes we use it when it's necessary, and when it's not, we can record faster without focusing on fingers, knowing that the animator will probably have to fix it anyway. So we end up with "shovel hands," as we call it.

Paweł B: I like that you asked about hands because for an artist, they always talk about how to capture perfect hands so they look lifelike. It's the same with animating that; it's very hard to animate real life sometimes, and actually get the complex movements that we have in our hands.

I have to ask this question because it always comes up: what about A-listers and stuff like that? How do we do people like Keanu, like Idris, and all that stuff? That feels like the glamorous part of capturing actors.

Paulina: We cannot tell.

Paweł B: Awww!

Tomasz: I can tell you one thing: we are right now in Poland, so recording Keanu Reeves in Poland or spotting him at the Warsaw airport—

Paulina: On the campus.

Tomasz: —or on the campus, someone would for sure recognize him and the whole reveal would be pointless. To avoid such cases, we try to find other solutions, but we have it in mind and maybe one day we'll be able to record them here in Warsaw. Yeah, because as I said, this is cool.

Paweł B: I think everybody in the studio would go crazy.

Tomasz: Oh, absolutely.

Paulina: I would tell you that probably more preparation goes into these kinds of shoots because you don't want to waste anybody's time. Because the calendar is so packed, you don't want to — and often can't — book another day with people of this profile. Often, we use stunt doubles or doubles in general, iterate on the material we record, and then once we have the final version of the scene, we go and record with the A-listers. But that's, I believe, all we can say.

Tomasz: Who is an A-lister? No idea!

Paweł M: Yeah, but on not wasting a moment, that's also a very important thing. Our studio is a very creative place — people running around, walking, talking, dogs barking.

Paweł B: Yeah, there's a lot going on.

Paweł M: Chaotic, crazy stuff going on there. When you enter a mocap session, it's like you are in this magnificent, beautiful, fascinating place, but there are all these rules around: "Don't touch this, don't go in there, don't say that, be quiet," etc. It's also very clean, right?

Paulina: Thank you!

Paweł M: It's like a NASA facility or some military facility. I feel like that, you know? It's because we don't want to waste a minute, right?

Paulina: I would say so. As a producer, I need to say it all comes down to the budget. With so many people joining the session and being there, you want to get the job done and record what you planned to record. Sometimes we are able to record even more than we planned, sometimes not really. So it comes down to following the rules and following the plan. We put as much effort as we can to record everything, and then you see the outcome. It is a creative process, so it doesn't always go as planned, but everyone here tries their best to deliver as much as possible. I wouldn't say I see it as a NASA lab, especially when you're in the middle of it and responsible, and you know there is a delay or the tech doesn't work. You need to have these rules to make your work easier. If people follow the rules and something doesn't work, it is much easier to spot the problem and bring it back to life or fix it.

Tomasz: And I will add that a shooting day in our studio is a moment when a lot of people are involved at the same time. Someone is responsible for preparing actors, someone else for preparing the system, someone else is checking the scripts, and so on. You need the rules to set boundaries and direction so everyone is responsible for the success of the shooting day. If everyone is just like, "Okay, let's do something!" then everyone will do their own thing, but we don't know what's behind that. If we have a rule that we have 20 minutes to prepare actors, then other teams know they have 20 minutes to brief them or prepare materials. Everyone knows where they are in the pipeline, which is really helpful to manage and get what we want.

Paulina: It is much harder to manage the crew when you have a bigger shoot, like performance capture. You have lots of people coming from abroad and you need to explain things — sometimes it's their first time on a mocap stage. Tomek is actually the person responsible for introducing everyone to these rules. Sometimes you have a shoot that is very easy with only one performer, especially when it comes to gameplay stuff. The team knows each other so well because they've worked together for ten years, and there is not really much input needed because the rules are already in your blood.

Paweł B: How about the feedback loop during a session? For example, an actor is doing something and you're like, "No, let's do it differently. Let's do it this way." Is it instantaneous, or do you have one director on set directing the actors? Is the actor getting multiple inputs from different people, or how does it look?

Paulina: It shouldn't look like that! You want to have one person responsible for giving feedback to the cast and executing the vision. If you want to record something, you want one point of contact that gives feedback or advice—

Paweł B: POC, that's military talk, by the way. Point of contact. You were on to something, yeah, yeah.

Paulina: Yeah, you probably want to have one director responsible for the session. The other people — usually animators or designers — can come and advise, but one person is preferable.

Paweł B: Yeah, because the crew is always getting bigger and bigger, and when you are creating a scene, you have input from multiple departments. Does all that input go to one person who passes it to the actor, who then recreates it?

Paulina: Yeah, I'm not going to lie, it's challenging. Let's say you have 20 people on stage together with the talent. Sometimes there is a camera operator or an AD who wants to introduce a change or give feedback, so it can get a little bit chaotic. But we are trying to go in a good direction and introduce the rules more and more to manage this.

Paweł M: It all sounds like a movie set, basically.

Tomasz: Exactly, yeah.

Paweł B: So what's in the future for mocap? What is going to change, or is nothing going to change? Is the scope just going to get bigger because you need to cater to multiple IPs and games we are working on? How do you see the future for yourself?

Tomasz: You can start.

Paulina: I would say the multi-project approach is going to be a challenge. It is a bit of a challenge now, as we see. We try to introduce new rules on the fly and deal with priorities and talk to the projects more often. We are in a more convenient environment than outsource studios because we are in the same environment as the project is.

Paweł B: Yeah, you're part of the DNA of the studio, and you also understand the language everyone is speaking. Working with outsource, it's always, "How do we calibrate and understand each other so we speak the same language?" Sorry, I cut you off.

Paulina: Yeah, I would recommend studios have their own mocap sets because you can record more stuff than you probably would with an outsource since it's cheaper. From a production perspective, the future is to make it work so we can provide mocap services to all of our projects. Since the studio is booked months in advance, it's going to be a challenge, but we'll have all hands on deck.

Tomasz: In the future, it will definitely be a busy time, I hope! On the technical side, we are testing different systems — even coming back to the weird suits with a PC on the back.

Paweł M: Oh, really?

Paweł B: Oh, wow!

Tomasz: Right now, they have definitely smaller computers on the back. This gives animators a chance to record ideas and pre-visualizations at their desks, make fast iterations, and record only the final decisions on the big stage. We want to record only the cases that we know are important for the game, not all variations. Thanks to that, we can reduce the scope and give other projects space to record their stuff. That is the main idea.

Paweł B: So each animator gets a mech suit with a computer that lets them record animations at their desk? That would be really cool! No, I can totally see it.

Paulina: We would be unemployed!

Paweł B: No, the final shoot still needs to be done on the stage!

Paweł M: It is a draft.

Tomasz: I think you are going too far. Slightly too far! No promises from my side, but we are testing different solutions just to improve the process because it's a never-ending story of trying to do things better. We try to feedback ourselves and find what we can improve. Can we do it by hand right now, or do we need other stuff? Maybe we have to invest in something, or maybe we can work more efficiently without changing the pipelines. In the future, we have to focus on the multi-project approach to help deliver multiple games.

Paweł B: What type of skills does a person need to have in order to work in mocap? On the production side, I imagine you need to understand how a set and pipeline work, how game development works, and how to collaborate. But what about working on stage? It's a very specific set of tools and experience.

Tomasz: As we mentioned, we prepare scenography, so someone who can work with wood — I know that's weird for games — or someone who can work with tools or prepare costumes.

Paweł B: Totally like a movie set. Everything I'm hearing is like, "Yeah, this is the film industry."

Tomasz: And on the other side, we have programmers. If we did everything by hand, the projects would have to wait until 2077 for the materials and data! What skills? Everything! Literally everything. From manual things to digital knowledge of how to model, because we need to print props and sometimes modify them to add weights. The answer is generic, but while one person doesn't need to know everything, there is always a place in our team for very unusual skills. We can find a job for such a person.

Paweł B: Perfect. Awesome. I think we have it, unless you have any more questions, Paweł. I think we should end on a favorite story from the mocap set, or your favorite cinematic that you did, or something crazy that happened. Just give me one story. We already had some examples, like the hamster in the washing machine for the tech demo. I know you did some weird monsters too, like people walking on all four limbs. What else is there? We talked about the horse and the dog. Any other examples?

Paulina: To be honest, I would say the horse and the dog are pretty memorable because they are different. I also remember my first mocap session was a stunt session just a day after I joined CDPR, so it was pretty fun to watch all of the fights. Open world is also fun to watch because you always have some NPC doing a weird thing that you actually need to record. But yeah, probably the horse session for me.

Tomasz: And for me, as I mentioned, the romance scene in the tank.

Paweł M: That one.

Tomasz: Yes, it was really challenging to prepare that, so that is my top one. The horse, of course, but it was mentioned already. And I think I have some others, but maybe one day I will reveal them in my diaries. There are a lot of shooting days where even the boring ones were encouraging because someone is doing something cool in the background that gives value to the game.

Paweł B: I'm surprised you haven't mentioned how we record swimming — having someone on a chair without a backrest, being suspended, and swimming without really swimming!

Paulina: We should have a separate block for, "Okay, ideas on how to record specific mocap," right?

Paweł B: Yeah, that's super cool. That's super interesting.

Paweł M: Pure creativity. Awesome!

Tomasz: Yes, so I think that will be in my diaries in the future. But yeah, about swimming — disclaimer — we tested on ropes, but actually, an office chair without a backrest is much better than swimming in ropes.

Paulina: Swimming in ropes.

Tomasz: Fortunately, unfortunately.

Paweł B: All right, before we finish, one more thing because I know the Mocap team is watching. Do we want to send our regards in some way, form, or fashion?

Paulina: Absolutely. Tomek?

Tomasz: Yes, definitely. Because you know, they are waiting just behind the door. We promised we would say something like, "Hi Mom, I'm on TV!" So, guys, you're doing a great job, and thank you very much.

Paulina: Yeah, thank you, guys. Nothing would happen without you, so exactly. Yeah.

Paweł B: Wholesome, perfect. All right, and we're off now.

Paweł M: Thank you. Thank you.

Tomasz: See ya!

Paweł M: Thank you for tuning in and listening to our podcast. I hope you heard a lot of interesting information and insights about how the mocap sessions are performed here in CDPR.

Paweł B: And as always, don't forget to comment, like, subscribe — all that jazz — and let us know what you're thinking about the episodes. Of course, we'll see you on the next one. Bye!