AnsweRED Podcast Episode 28 — More Than Testing: The Truth About Game QA

Paweł B: Hello and welcome to the AnsweRED Podcast. My name is Paweł and I am joined by Paweł Melniczuk, the Art Director for Project Hadar. Paweł, who do we have today?

Paweł M: Today our topic will be about quality assurance and our guests will be Monika Rokita, Narrative QA lead, and Piotr Tyszko, Associate QA director.

Paweł B: It's gonna be an incredible episode and I can't wait to meet our guests — so let's meet 'em! So, Monika, Piotr, welcome to the podcast. Officially, QA, one of my favorite topics. I'm surprised we haven't done this earlier, to be honest, because it's a foundation, I would say. I know, we'll just blame it on Kalemba. It's a foundation. But before we get into all that, could you guys just introduce yourselves for the people listening so they can hear your voice and associate it with the person and, and the job position that you have here at RED?

Monika: I'll go first. I'm Monika Rokita. I am a Narrative QA Lead working in RED since 2013 with a short maternity break leave and always in QA. Started in QA. Started in narration. Still in QA. Still in narration after all those years. Because I like it, obviously. And that's it, I guess.

Piotr: All right. I'm Piotr Tyszko, Associate QA director. It's now over seven years in CD PROJEKT in general. However, it was over five years in GOG and almost two in RED today.

Paweł B: Perfect. I'm happy that he said "Pro-yekt" not "Project" because—

Piotr: I tried. I tried my best.

Paweł B: That's how you say it properly. Perfect. So guys, let's start from the basics. 

Paweł B: I think when people listen or watch this, they think QA, for sure these are the people who just sit down, play the game all day. I could do that, right? Anyone can do that because it's easy. But I feel like it's a big misconception. So what are your takes as people who actually work in QA? How does your day to day work? And is it just sitting down and playing the game over and over?

Monika: So I think I can take this one because this is kind of my day to day. Um, yes. That's true. This is the most common misconceptions about the QA. That's true. And yes, we play the game, but not in the context that you would think of it. We do it not only for fun, we do it for fun as well, because we have to make sure that the game is fun, but we also analyze it a lot. And there are many, many aspects of playing this game. You know, the analysis is based on many things that we need to check. And there are like subjective and objective stuff. So we do play the game, but it's not the only thing that we do. And we are not the only ones doing that. Basically everyone in the company should play the game at least once. So yeah.

Piotr: Yeah, I think that it very often looks like we're playing the game just as you said, right? But Um a lot of things happen behind the scenes that you don't see. I sometimes use this metaphor that this is like, you know, solving the puzzles under the microscope. And it can look like we stare at the same door for 40 minutes, but actually it values. Right? This is QA also.

Paweł B: So it's more than just looking at a door, but— There's more depth into it.

Piotr: Sometimes. But there are little pieces you don't see yourself, but they're important. 

Monika: You're looking at the door, like from a different angle and with a different context in mind, but this is the same door all the time.

Paweł M: I think the other misconception could be that QA is the last step, right? So we make the game and then we send it to you guys and you just test it and it's done, right?

Piotr: It's totally the opposite. It's totally not. I think that the approach we are now having and like we're looking forward to have in the future is to be early in the development, right? So this is this terminology from software QA, shift-left, where basically we try to be as early as possible. So, you know, it's always easier to move a chair or this door I mentioned on paper than later, once the house is built around the chair, you know what I mean? So it's better to be earlier to prevent costly and big bugs and problems in the future, right?

Paweł M: I remember from Phantom Liberty because previously I was mostly engaged in the characters. And since Phantom Liberty, I was also working in environments. And I did not realize, for example, how many elements connect to the environment. So the scene placement, dialogs, you know, unique points, you know, all the stuff are—

Monika: Many systems are overlapping, many features are overlapping. And we are the people—

Paweł M: Having the good QA there with the team from the very beginning, with the content team, with level designers, environment artists, we've seen designers just taking, you know, a hold on top of all these moving elements was incredibly important.

Monika: But we are even earlier in the process. Not only when the game is already implemented, the puzzles are being done. We are even in the design phase. Of course, because we also have our other side, we are the guardians of quality. QA means quality assurance, so we have to guard the quality and support it in many different ways, not only when the game is done, but also when there is an idea of the game and an idea of the features of the system, and everything needs to be working together. We are there to be the voice of ideas, of risks, of many other elements that we see already at this point. So we are there like from the beginning, basically. Till the end.

Piotr: It's like sometimes people associate QA with just finding bugs, right? It's not this, only this, this is really important of course, but beyond bugs, bugs are visible, but beyond bugs, there is like, I think that the main goal of QA is to protect and prevent from a bad player experience. So we protect player experience and we try to spot, identify, and reduce possible risks. Sometimes we need to jump to the fire. Sometimes it's better to go where the fire will be in the future. It's always cheaper, you know.

Paweł B: Anticipating where the fire will actually happen—

Piotr: Than fighting the fire. Right. So that's what we do. And what Monika said about like, you know, making sure that like, there is a quality in the game. That's our role. I try to explain it to the people in QA and also outside QA in our company, that QA is the one accountable for quality, but QA cannot be the only one responsible for quality. It's like a common effort. It's teamwork, right? So there are producers, designers, artists, engineers, all the development teams and folks who collectively work on the quality, but we drive that quality aspect, right? We make sure they see the risky areas. Like, we identify the problems. We try to address them as early as possible. That's our job. Right? And one more thing. Sorry, I'm talking too much but like—

Paweł B: No, that's perfect.

Piotr: One more thing about QA that I really like to show to the people, like there are two sides of this coin. There is objective QA, I would say, it's more about functional testing and making sure that the game, the product works, but there is also subjective QA, which is more about the experience. How does it feel to play the game? Does it match the general game direction and vision that we decided we want to have in this particular game? And it's important to blend both of them. It's like once we have the car, we make sure that the car has brakes, right? But then we need to check if it feels— Not like, you know, not like, sorry. It doesn't feel like a shopping cart going downhill, you know?

Paweł B: Yeah.

Piotr: It has to be like a smooth drive, right?

Paweł B: There are some cars that are that way. Beaters.

Piotr: I know.

Monika: But it also has to have a pretty color to make sure that the people looking at it also— You know, there are many emotions connected to that. There are things like checking pacing of the game, checking motivations of the characters, many things that are like zero one things not technical wise, like not only about working, but also about being fun. Again, the fun thing. But yeah, this is important.

Paweł M: Also, I think what will help is how I think about QA is there's two things that support this. Like, first of all, they're the most experienced players, gamers. They are walking libraries of all the games—

Monika: We have to play the games, we have to compare the stuff. We have to know what is going on in the market, what people want, what the player wants.

Paweł B: I would also say the best players gameplay-wise, yeah.

Paweł M: So that's that's all this, you know, this experience, they can share this experience from different games. Other games. External games. And tell the developers who are not always, you know, as experienced gamers what to do. But the second thing is people in the company who know the most about the game from all the layers from engine design, you know, to the final experience to narrative. Because a lot of people in departments, they're very specialized and they're just locked inside their own, you know, I like the word silos, of course, but they are very focused on their own part of the work. And QA is something like, you know, that's the glue.

Monika: We're the collective.

Piotr: So just wanted to say that I fully align with what you said. And yeah, we are the first audience and very often the first critics of the game. And to be these critics in a meaningful and healthy way, we need to be experienced with the games, right? So, for example, if we want to have some touch of, I don't know, soulslike in the game, we need to know those soulslikes. We need to play them. Someone has to be really experienced in that. Because we are not a trustworthy partner if we don't. If we want to have an opinion, this has to be based on our experience. So yeah, QA has to play a lot of games.

Paweł B: What about specializing? Because we kind of touched upon it. So if you could tell us more or less, if you look at QA as a whole, you said you're mainly narrative, but there's different parts of QA that focus on specific parts of the game. So how, what, what's the division pretty much?

Piotr: It's complex. Maybe you can start with your perspective.

Monika: It is complex. But let me paint a picture. On the projects that I work on, because every project can be different. QA is adjusting to the project and to the needs and to the areas that are created on the project. So for our projects in RED, we have those 5, 4 or 5 main areas. It's narration, it's gameplay, it's tech and it's art and open world as a fifth one. QA are adjusting to those areas and are more focused on what those areas are creating, implementing. So we have this deeper knowledge about stuff because what's important, we work as analysts and analysts need to know the tools. They need to know the systems, the process. We need to know everything that the designers know, sometimes even more. And we have to have like this broader picture on everything, this holistic view on everything, because the systems also are overlapping with each other. And you know, we are just specializing in something. We are better in something, but that doesn't mean that we are only in this thing that if I'm in narration and I work mainly with quests and cinematics and story, the things that are building the narration most, that doesn't mean that I don't work with level design or localization, because those are also ingredients. There is a metaphor that I like to use. This is the point that I use my soup metaphor because I think QA is like tasting critics, like a restaurant critic, we are tasting the dish that somebody else prepared, and we are checking if the ingredients work with each other, if there is enough salt from level design, enough sweet from story side and stuff like that, and being specialized helps us to do it better, basically. But it also doesn't mean that we all need to have a broader view.

Paweł B: It can't be tunnel vision on one thing.

Piotr: We have to be careful with this because we also don't want to be silent, you know, in our disciplines or areas. And especially when now we are developing multiple projects, I really encourage ourselves to be more flexible and adaptive to different scenarios. So yeah, this differentiation that Monika said is like in place, but also we need to remember to be adaptive, right? One more thing that I want to say, which is really important, this division, this is how we, for example, form our team in Polaris, Witcher 4 game, but we can't forget about the rest of the QA, which is also disciplines like localization QA, audio QA, QA engineering, which is part of tech QA, but also, recently formed, really, really important team, publishing QA who's supporting our external projects where we have different type of work because we are not like a dev QA team embedded to the content teams in-house project, but we kind of verify the work of the developers, external developers who work on the projects for us. Because we became a publisher as a company as you know. So there is a big spectrum.

Paweł M: Making sure that the external projects are matching our own quality and—

Piotr: Exactly. So yeah, once we decided, okay, let's publish the games that are developed by someone else, but let's make sure this meets the same level of player experience. We used our players, right? So we don't screw it. This is super important. And actually so far, so good. But we'll see in the future right.

Paweł M: And does the QA adapt to the phases of the project? How does the structure of the team change over the production?

Monika: It does and it doesn't. This is a complicated question because— again, I'm speaking only from my perspective on the project on the bigger one, our inside project. Those areas are stable. They are there and we need to work with those areas. But the needs change, like the needs from these areas change. And obviously there are different phases of the project and different needs during those phases. As you know, at the beginning is the design. So there is a lower need for QA. But if you go further into the game, there are things like compliance that stepped in. So, you know, more things are being checked.

Piotr: Compliance is very important. Compliance QA is a QA that verifies and checks if the game and the features in the game are in line with the certification requirements for each platform. So for example, you need to publish, you are about to publish the game, launch the game on PlayStation 5, Xbox, or Nintendo. And each of these platforms has different requirements. We need to meet all of them. I'm really proud of our compliance QA team. It's part of our publishing QA. We have one man army now, but supported by external partners who support us in testing. So far, every certification we have had since like 2024 was successful, right? So in the first attempt, which is like a big success in my opinion, and this is important to have someone in the studio in QA who is aware of where we need and how we need to get prepared for this certification.

Paweł M: It sounds like the last thing we do, but during the development we prepare ourselves.

Piotr: That's the last thing we do. But in early development, like in pre-production, you need to sync with the engineers about how the game will be built to meet this requirements at the very late stage of the project. Right?

Monika: We can act earlier than just waiting for this moment. The certification moment.

Piotr: But you need to start thinking early. I think this is the essence of QA. You need to identify some important issues early because later on in the late stage of the project, like, I don't know, advanced production. It could be too late. Right? Or too costly.

Paweł M: I think this could be one of the mistakes that the new studios are doing. For example, they're thinking about QA, introducing QA too late in the process, right? Thinking this is the last step, no.

Monika: When the game is done. Basically see it and test it? No, no.

Piotr: I think it used to be like that for years in the industry. It's changing now. It is. It changed a long time ago probably in the US. It's changing also in Europe. And I'm glad this is happening because we see the benefits of this.

Paweł B: It needs to be an iterative process. Like with anything that is within game dev, it needs to happen at the early stages of the game. And then just going through different iterations, testing and playing and kind of, you know, finding out what's the next thing that should be done.

Piotr: I think that, uh, just one more thing. It's like with QA, like sometimes you don't see— it's like with producers, you don't see sometimes the value of production or QA unless you don't have it, unless you miss it, right? So yeah.

Paweł B: They're like, yeah, we're screwed pretty much. You also mentioned that you use external QA partners. So how often do we actually reach out to external partners with help and how does the collaboration work?

Piotr: Quite often, very often I would say.

Monika: I would say all the time, all the projects that we did.

Piotr: Most of the projects, maybe not all of the projects, like at the moment, we are a multi-project studio. So we work on very small projects sometimes, and then it's okay to have just a couple of our analysts inside. But with the majority of our projects, we use external partners who are responsible for functional tests, mainly sometimes compliance tests. These are testers. This is the difference. They are testers. We are analysts inside our studio. And you know, within the, like the complexity of the project and the later stages, we need to scale up our testing and then we have more testers, right? So we use testers. We're really glad that we have them. So thank you. Thank you our partners that we have you on board.

Monika: Because both roles are important. Like there are different roles. Analysts analyze more testers play more like functional tests. But both roles are important and obviously working with each other. Of course.

Piotr: I think that very often external test testers are the main source of knowledge about the state of the game and its functionality. Sometimes there are supplements, sometimes we really trust the value we take and the data we have thanks to their work and we try to use it to make the best decisions about potential risks or about how we develop the game in general, right?

Paweł M: So let's now maybe dive deeper into what the actual work looks like. Monika, can you tell us more about this narrative QA? That's interesting for me because, you know, I think people might have different imagination. What does it actually mean to be a narrative QA?

Monika: What does it mean? This is a philosophical question.

Piotr: It means a lot.

Monika: I already mentioned that the narration works with specific teams. The core of our job is working with the teams that create narration. So basically quests, story writers, and cinematics. But we work with everybody. The teams are, Piotrek mentioned, being embedded. We didn't explain that. But this is important actually in this situation, because the specialization also means for us in RED that we are embedded. That means we are working close with those groups, with those teams, with those developers. And when I mean close, I mean very close, like we are on the meetings, on the planning, on all other things that are being decided, we are there.

Paweł B: Super important.

Monika: Maybe we are not the decisive people. We are more of a supporters group, but we are still there and we can still have the voice about it. So in narration we obviously work on quests, so we play the game and our most important aspect is playability of the game. So you can start it and finish it, and you can move through the quests and you know, pick your own choices and finish it. And there is no blocker on the way. So this is the most important aspect for narration. And there are many, many things going into this playability things. And again, the soup thing, so many ingredients coming into the soup and we are testing it and feeling if the story works well, if the motivation of the characters works well, if there is a continuity, if we are doing multiple titles one by one.

Paweł M: And also between quests, right? Because yes, each quest has different quest designers and sometimes maybe they can get lost in their own work and forgetting about interconnecting.

Monika: Exactly. We need to take this holistic view of the game and know if the quests are— we are comparing a lot of stuff. The pacing thing that I mentioned is checked along the full game. Like each scene can have their own pacing, but also each quest can have this, but also the whole game has their own pacing. So we have this whole level thing of checking this playability stuff from the beginning, from the smaller parts till the end. And what's important is that in our work, we also touch a lot of topics from other themes. Because again, everybody is implementing everything into this one big jar with the soup. So we are the last people looking at it. So not only the narration thing, but also if other features, gameplay features worked well with this narration.

Piotr: So you are like a cook tasting the final product, right? Each ingredient could taste delicious. But together, not always, right?

Paweł B: Yeah, they need to work together.

Piotr: I think that the really important thing about your side of QA is like narrative. Our games are narrative-driven games, right? So this is like super important for us to have this aspect and have focus on this, not only this, but this kind of—

Paweł M: Needs to be perfect.

Piotr: Yes. This is like something that makes us unique basically.

Monika: We like to do many, many parts and many, many choices. So QA needs to feel the consequences in the game.

Paweł M: Does the work start even you said before, like on the paper. So I know that quest designs, they usually start with this massive document with a lot of, you know, A, B option. If you do that, this happens. It's really hard to read. There's a lot of, I guess—

Monika: Multiple branches thing.

Paweł M: Going through all of that, making sure this logical connections are working, etc.

Monika: This is our job. There is a thing called path testing, the player path testing. And this is also the most important thing next to the playability for narration. So we make sure that all choices that the player take among the game, like all the variables that you can take, are finishing well, like are not blocked or not stupid, basically. So you do something and there is a consequence that doesn't fit your action. So we play the game, but we play it in a context with like 100 steps to check if every variable, if you take a car right now or just, you know, fast travel to some place, if the scene is going to work. It sounds very monotonous. It kind of is because you have so many steps that you have to follow, and there are so many possibilities of those paths that we cannot even cover all of those. There are thousands, even millions of those possibilities that the player can take, but we try to cover as much as we can. And yeah, and we do that for the longest time on the project, like the game, if it's ready, we're checking if all the paths, all the possibilities and variations are correct, fun, again and working, basically.

Paweł B: That must be super difficult because we work on massive RPGs and the games are pretty huge. Like if you look at Witcher 3, 100 hours or more easily. So are there any shortcuts or any things that you can do in order to speed up some things? So kind of, I don't know, move faster, traverse faster. Like you have a big open world to kind of traverse like, are there any tips and tricks that you can pretty much do in order to make the process a little bit faster?

Monika: Yes, yes. But they're based more on what the engine can do for you. So we in our studio, we prepare the stuff inside. So if there is a need to skip something, we have the engineers or the people creating this stuff for us. So there are many debugs actually, this word didn't come up today. So debugs are the things that we are, um, we are going deeper into the stuff. We are not only just, you know, analyzing what is on the shallow ground, just looking at the back, this is not working and that's it. We are going deeper and debugging why it's not working. So, you know, there is a lot of things to cover. I was actually the person that was preparing the paths, for example, for Cyberpunk thing and for Phantom Liberty. And actually I was working on the paths for Witcher 3, so I'm a good person to ask about it. And there were many variables that we needed to check. And actually, my favorite thing that I like to mention from Witcher 3 is that when we wanted to check all the possibilities that the designers prepared, we didn't have that much time to check it, obviously, because there could be thousands of variations. And the QA, we as narrative QA wanted to check it in pairs. And one person played during the day and the other during the night. So we had the night shifts. It isn't normal. We don't have the night shift normally. It was only our idea to cover all the possibilities, and it was worth it. As we saw, for example, the Baron thing in Witcher 3, all the possibilities that you can, you know, play this quest. Thanks to Paweł Sasko. Hi Paweł.

Piotr: Hello Paweł Sasko.

Paweł B: Has to be mentioned.

Monika: So you know, we needed to check all those things and those massive games. We like to do massive games. We like to give people the choice. We like to make people roleplay in our games. So we need to make sure that this is working.

Paweł B: Giving them the freedom I think is a big thing.

Monika: But it's also fun for us to play the game and check what the difference is.

Piotr: Just one more thing to add here. Of course our games, as you mentioned, are very massive big games. So we are not able to play them all the time. You know, like to run the game from A to Z. So we also do speedruns, right?

Monika: Oh yeah. But because there are different type of paths that you can take, we're also doing that. There is a completionist path. The biggest one that you do when you do everything or you know, the Skellige question mark.

Paweł B: Gotta get that platinum. I mean, come on.

Monika: Yeah yeah yeah. I'm also a platinum gamer. I do the platinum things on the PlayStation. So yeah. So I love that. So we made those completionist paths to make sure that you do everything. But also there are speedrun paths to make sure that you can skip everything. There are also the paths on the way because you can fail the quest. So again, many different possibilities that we need to take, think about and many different tools to do that, because there is a lot of data to take into account, to randomize it to, you know, make sure that you didn't miss anything. And for that one more thing, there is also a thing called Golden Path because this is the most popular, so it's supposed to be the most popular path for the players. When we want to show some stuff on the way, we are making sure that like the biggest percentage of the player will play this path. So we are making sure that this is working. This is the priority.

Piotr: It has to be perfect.

Monika: And all the other things are like—

Paweł M: It's usually the main quests, right?

Monika: Yes. The main path through the quest.

Paweł M: If it branches, what then? We have two golden paths.

Monika: We have more... If it branches, we have more paths. And it's our problem to check all of those.

Piotr: Actually, this branching thing is so common for us in our games. We need to be prepared for that.

Paweł M: It's hard right? I wonder how many times you finished the game?

Monika: You want a number? I don't remember. Many, many times. We need to finish it many, many times because you can—

Piotr: It's hard to count it.

Monika: In a different way. And the truth is we are not always finishing it. You can play the prologue for 100 times before you finish the game, because—

Piotr: First you need to have the game, right?

Monika: Yeah.

Paweł M: But here I also wonder, because I know how hard it is for regular developers to just play a draft of the quest, for example. They're super rough, you know, it's all gray. The characters are not talking or making some silly moves. There are no animations. It's not a pleasant experience. So you need to have a lot of imagination just to imagine yourself the final product, rather than just focusing on what doesn't work, because basically nothing works at some point, right? I mean working, I mean, maybe it looks horrible, you know, and just you have to do a lot of imagination to imagine like, okay, like I'm playing, I'm focusing on this narrative structure and I'm paying attention to all those crazy things happening onscreen.

Monika: The truth is, when the game is in that state, what you are describing, we are focusing on that state more, not on the final vision, because things may change during this time. So we have to focus on what is there right now. So If it's working, if the fundamentals are working so we can make it beautiful later and polish it later. So it's kind of, we cannot step into the future that much with our mind.

Paweł M: I just mean it's very difficult to just focus on this actual thing. You have to evaluate that at that point.

Monika: But I think it's difficult for everybody on the project to imagine the full game, how it's going to be with everything coming from every department, because you don't know that much at this point. Once there is a time in this process of creating a game called feature complete, um, it's called different, it can have different names.

Paweł B: Content complete?

Monika: Exactly. But after this moment, this is the moment to like look at the full game. I mean, we should be talking about risk before because after this moment, it is hard to introduce any changes. But this is the moment when you see the full picture, like very, very full picture with the details and everything with all the audio sounds, because the post-processing is coming up a bit later than the quest thing. We don't work like all the teams are working at the same time. Some teams need to prepare something for other teams. So it's...

Piotr: It's I think worth mentioning that this is also the moment when the content is complete, where we shift a lot of our resources from like, embedded work to we call it like a global game where we play the game, we just play the game. Whether these are analysts or testers, they play the game and just, they find the findings right.

Monika: This is the moment for the paths actually.

Piotr: Yes.

Monika: Yeah. So yeah.

Piotr: And this happens rather first of all, we need to have the content complete, and it's rather the more advanced stage of the development. So advanced production.

Paweł B: Having such a big knowledge of the game do you feel like it kind of changed your approach when playing games in your free time?

Monika: Unfortunately, yes.

Piotr: Oh, yeah. I recently started to play the game just to, you know, identify maybe not the bugs, but like how these games are designed, what we can learn about them in our games. So this is kind of, I love it because this is like, it's about I don't feel guilty if I play these games because I learn and this is for my work, but also pleasure, right? Sometimes if it's like more pleasure, I stay in the game and try to beat it. Sometimes I play just like 10 hours or 6 hours just to have the feeling about the game and see what we can learn from this. The successes they provided the developers to us, or whether the failures we want to avoid in our games.

Paweł B: We have an analytical approach to it. You kind of see what's happening?

Piotr: That's me. But I think that many people do the same.

Monika: That's true. We know what's under the hood at this moment. So we know how the game works, not only like enjoying it, but we are also enjoying it. But not only. So we are, I think me personally, I appreciate more what the other creators are doing, but I know how it's working under the hood. So for example, when I see that somebody uses a camera in a creative way in a game and I know how it's made, I really appreciate that. I know this and I appreciate that the people are doing that.

Paweł M: You know how hard it is to make it.

Monika: Exactly.

Piotr: Yeah, absolutely. I think that you have more empathy to the developers because you know how hard it is. This is the miracle. I think that Seba said that in some previous episode, our game director of Witcher 4 said that we make, we make miracles, right?

Paweł B: Yeah.

Piotr: And it's difficult to make them right. Sometimes it doesn't work well. So yeah, I have a lot of like, empathy and appreciation to the developers.

Paweł B: I'm happy that you said that because I was thinking it's going to be the other way around, that it's going to be like, oh, no, it's very hard to enjoy games now because I know how everything works and I know where they kind of cut corners. This could have been better. I know what they did here.

Monika: There is this side but I didn't want to speak about that. But there is this side. When you see that things could be done better, if you know how they're done and you see that um, there is an example that I like to take, like when there is a scene and you are a player at the scene and the scene ends and you see that you are spawned somewhere outside, like far, far away. I'm thinking, why can you be spawned at the same point? Like I'm—

Paweł B: Breaking immersion in that case.

Monika: Breaking immersion. Beautiful words, but yeah, this is the side.

Piotr: I think that there are things in games that I observe that when I play the game, that irritates me, because I know that this could be done better. We have our own perspective. We develop AAA narrative-driven games. So it's different than indie, right? And sometimes I'm irritated that the, you know, the character I play jumps in a weird way, but later on playing the game, I know that it doesn't really matter because the overall experience is great. So it doesn't matter that the character jumps in a weird way, right? So these are the kind of experiences we have.

Monika: We have mixed feelings right now. Basically, it depends on the game and how the creators did it. But yeah, there is this something in your brain working differently when you know how it's done.

Paweł B: So what's the most challenging thing you would say in terms of testing an RPG system, quest logic, or branching narratives. And also I want to touch somewhere on content lock and kind of getting all the submissions and things done and then pushing it out. And because that always is, I feel like in the studio, there's always, we always talk about, content lock is coming up. We gotta like put as many things and ship it.

Paweł M: And what happens when the content lock happens like the day before or night before all the submits come. So in the morning, you open the floodgates. You don't recognize the game.

Monika: Actually, these are two different answers. Because what's the hardest thing about testing those massive games? I think it's to cover everything because we're just people and we are also getting tired of stuff. And we also work only eight hours a day. And to cover everything that the players, that the thousands and millions of players can do, it's not possible. So we are trying to cover a percentage of it. Like the most popular thing that I mentioned, the prioritized thing. And we would want to cover everything and make sure that there are no problems in the release product. Like we cannot do it. The other thing is the time. We would have to have a lot of time to make sure that the game is tested, like very, very properly. And we are doing what we can in the time that we have. And the other thing, the locks, this is kind of a different thing. The locks are part of the process for people that don't know, there are moments in the process that you need to lock the submissions to make sure that you control what's coming into the build to control the submits, to check all the submits.

Piotr: It's like, I love this metaphor. I know I'm using the metaphor again, but like.

Monika: We like metaphors.

Piotr: You know, it's like when we build something in the early stage of the project. Then later on, for example, we stop people who want to move the wall. And this wall is at the same time being painted by someone else, right? So this is kind of a metaphor.

Monika: Putting a picture on it.

Piotr: Yeah. So that's why we do content lock. So no walls moving because we paint them, right?

Monika: We're stopping everything just to check where we are, how the game is, how the build is, and everything that is coming in needs to be checked by QA. Yeah. Triaged and checked and controlled and even before it's implemented into the game. So yeah, there are moments that nobody likes to be honest, the people that are stopped. And we are also like, this is a heavy burden on us because everybody is waiting for us to check.

Paweł M: A bottleneck.

Monika: Yeah. The bottleneck thing. We are becoming, we try not to because we don't want to, we don't want to stop anyone. And, you know, we have a counted number of days to do something. So we have deadlines. So we have to fit into deadlines. So we need to be smart about the locks. And there can be different rules about it, but it depends on the moments in the process. Depends on what is the state of the game. The lock can also be only on one team, not on everybody. It's different. So there are many, many details to this process, but basically it's about control and making sure that everything is stable and the game is stabilized.

Paweł M: So basically when this lock happens and a new change needs to be introduced and someone is doing the change, could be a small one, but could be the whole system.

Monika: You never know.

Paweł M: It's being sent to you. So nobody else in the studio actually can download it in the engine. You have to test, does it work? Does it fill the requirements? Does the whole game work? Does it fit compliance? Does it break the whole game? Yeah, exactly. This doesn't break the editor, the game, the paths, etc. And once you validate it, it enters and everybody in the studio can download it, right?

Monika: Yeah. And at the end, we have all those submits and the build should be still stable. But being prettier, let's say, and working better. And this is hard because, you know, again, the bottleneck thing. So people are waiting for the submit. It takes time to check because—

Piotr: Yeah, we, we're the bottleneck and we stop the game, the development, but we do it for the good of the game.

Monika: We have good intentions. This is not to make anybody—

Piotr: We have, we feel this pressure. We need to be lean and effective in what we do. So we don't stop the development because this costs.

Paweł M: I remember the famous boat from The Witcher 3. It was like you change something completely unrelated, one thing in a quest or something, and then the boat breaks.

Monika: This is the thing where you need to QA because if the designer is working on one thing, he's testing only this thing in his own environment. And we are testing it in the environment of the game, which is very important because it can affect anything, even the things that even you didn't think about it. It happens and that's why we're there.

Paweł M: So we have bugs, right? How are they communicated to the team? What are the means of communicating all these problems and bugs to the development team?

Piotr: There are many ways of communicating and means of communication. There is obviously, very like ad hoc communication. We use our Slack channel, for example, our communicator, there are reports we prepare and send, there is Jira reporting.

Monika: We use Jira for bug tracking. Uh, and we use many, many fields to let people know if this is high priority with severity. There are like assignees, themes, and stuff. So they should know, they should follow this as well.

Piotr: Like, yeah, reporting is one thing, but this reports needs to catch the attention and explain swiftly what's the problem. If it takes ten minutes to understand the bug, something's wrong. You need to improve your reporting, basically. Right?

Monika: We have some best practices that come to the bugs to make, again, things effective. We are crazy about effectiveness.

Piotr: But actually this communication is also very difficult because we need to be honest and direct, but at the same time not to be assholes. Sometimes we look like assholes, but that's not our intention.

Paweł M: Guardians of the quality.

Piotr: This is radical candor, right? So you are direct.

Paweł M: "Hey, come on, this one centimeter in this wall, we'll move just a little."

Paweł B: Won't change anything.

Paweł M: Yeah, it won't break anything.

Monika: We have to be very assertive in our work because we communicate, we work with many, many different people, teams,and characters. So different people communicate in a different way. So assertiveness is very important because we want to relay the message about the data. We don't want to criticize anybody's work. We are doing that, unfortunately, because we are pointing out mistakes in the work. But we don't want to be harsh. We are doing this with a good intention.

Piotr: I think this is like we try to be kind but not nice, right? We say how it is, but we do it for the good of the developers. And what's most important, the good of the game.

Paweł B: Brutally honest. Always the way to go.

Piotr: Yeah. Be honest.

Paweł B: I'm happy that you mentioned reporting. I remember that whenever we are before, like hitting a big release, we always get reports with amounts of bugs that are happening, the stability. And it's always like, what's the number? What's the number? How close are we? Especially if you work on communication, you want to know before the release how close you are to a product, which would seem that is like, you know, tested, looks perfect and that you're ready to kind of, you know, work on the publishing side of it and the marketing side and the communication, the PR, and all that jazz that, you know, you're seeing also how much work is actually put in by QA team. And you're seeing like the stats going down because we get graphs, we get everything. So it's like a fully blown awesome presentation that is easy to digest. And you're like, yeah, there's really a lot of work going in. And also you know, in the last days, in the last months and, you know, and you see the whole process, which is super interesting from my point of view, for example. I love the data.

Monika: I love gathering data.

Paweł M: How many bugs there could be?

Monika: Well, many. It depends on the phase, depends on if you are thinking about all the bugs or maybe bugs directed to one team. But there can be thousands.

Piotr: Thousands. But like what matters is—

Monika: They should be closed, obviously not open thousands.

Piotr: What are the bugs that really matter? Of course we try to avoid all the bugs, right? But we also prioritize and set the severity for the bugs. We try to avoid those. And first of all, those that really impact the game in the wrong way. You can't avoid all of the bugs, no matter how you test the game, no matter what kind of tests you apply, the game will always surprise you at the end and the players will surprise you, right? But our aim is to minimize this surprise, right? So where there's this development pipeline, we try to make sure that the outcome of the pipeline is not a big surprise.

Paweł M: And what are those levels of severity of the bugs?

Piotr: I think it depends on how we agree on production. Right?

Monika: There are three levels, the most high one, medium, and lowest.

Paweł M: What's a P0 alert? Priority zero if you don't know what P0 is. So it's like the highest. It's like red lights everywhere. When you got a P0 bug, then—

Piotr: Everybody, every hand on deck.

Monika: Yeah.

Piotr: So for example the game doesn't work. The build is not like—

Monika: Starting.

Piotr: Gamebreaking. You can't really kick it off. So then like, yeah. It's what we try to achieve, always working game. So we try to have the game that is always working, right? The build that is always like able to start, right?

Monika: On all platforms.

Piotr: If it's not happening, it's like a blocker. It's P0.

Paweł M: And you get an email.

Piotr: Yes, we have special channels. We have—

Monika: We have our own. Uh, yeah, let's not talk about it because people are listening and we have our, I think good methods for that. And yeah, depends.

Paweł M: So there's gamebreaking, like important, for example, visual bugs or functionality breaking. Then there are probably some cosmetics probably.

Monika: That really depends also on the specialization of the team, because each team can have their own priorities inside.

Piotr: Uh, and also the P0 thing changes over the development. So I guess... It's different in the later stage of the project and in the very early stage of the project, sometimes you don't pay attention to the things that are critical in later stage when you were in the beginning of the project, right?

Monika: Yeah, I think it's kind of at the beginning, we don't report everything because there are not enough things to be reported. Like we report all the blocking bugs, the groundbreaking things. Then there is this gate open flow thing, report everything moment when we filter out everything. And then again, we're closing this filter to the most important things because the time is running out, unfortunately. But just to say this, there is a big chance that if you see a bug after playing after the release, the bug in the game, it was seen and reported, but for some reasons, we didn't have the time or the chance to fix it. We try to report everything, but we don't have enough fixers, for example. And that's why the patches after the release, we try to address that after. So yeah, we're trying to cover everything. But again, we're only humans.

Piotr: But like one more thing about the bugs, like of course, P0, P1, and P2 probably these are the things that we need to address immediately, but like there are good examples of games you can check in the world that they had bugs, but overall they were brilliant, right? Because the overall experience was great. So we can't really be too focused only on bugs and like go mad because of bugs. We need to balance this objective side and player experience feeling with the bugs. Sometimes it's okay to have little things. What matters is if the overall experience is good. It feels good, right?

Paweł B: Yeah. I'm happy that you also mentioned post-release because there's always this thing that, like you mentioned, you can test things for so much time. And still when you give a game to a million players or more, they will eventually find something, right? So how, how do you manage this? Because sometimes players do some of the weirdest shit you could ever imagine and you're like, I would never think that like, you need to have a special weird weapon and something together and go instead of going like the main path. Go around from the back and do something.

Monika: Yeah, the repro steps are very complex. And yeah, so how do we—

Paweł B: So reproducing actually what the player did.

Paweł M: Based on what? YouTube videos, for example, like Twitch streams or just you are getting the information from the players.

Monika: Exactly. After the release you're asking. Yeah. So after the release, the main focus on let's say bugs and problems with the game is shifting to customer support. And customer support is like the group that is gathering all the info. So if you have any problems, reach out to customer support. They will reach out to us.

Piotr: They are very active in looking for this because this is not only them like, receiving the tickets from our players in our customer support portal, but they go to the forums, right? They read about the problems or they— Check the social media portals and stuff like that. And if they report something, yes, then they let us know, right?

Monika: They have their own Jira and their own project. And they're reporting their own tickets from all the information that they could gather. And it's sent to us to try to reproduce, try to debug it, try to analyze it and find the conclusion. So this is for all the people that's sending those issues. The more information you get us, it's the easiest for us to find it. So, you know, if you have, save files, crash files, anything you can think of the repro steps before you did it. It's kind of a QA job to prepare a ticket like that. But the more you tell us what you did, there is a bigger chance that we will fix this issue. If we only have info like, this thing doesn't work. We are not sure why. What's the reason? We can try, it works for us. And there's the saying in Polish language: it's working for me, not working for you. So yeah.

Paweł B: Like what do you mean, everything's fine.

Monika: Works on my PC, exactly.

Piotr: Just restart your computer.

Paweł M: It's the same quote for developers. Some developers are also sometimes reporting bugs to you. Right?

Monika: That's true.

Paweł M: It doesn't work for me, but also—

Monika: Or they're reporting the bugs for themselves. But we are helping find the solution and investigating.

Paweł M: There's always this, you know, when your editor crashes, people tend to send just okay, just kill it and restart it. But it's better just to summarize what you did so it crashed. So you have more information and you can make better-

Monika: We work on data and you know, the bigger data that we get, the more data the better.

Piotr: As Monika said, the more information, the more context we have, the better and easier for us to fix it or address the fix with the developers soon.

Paweł M: Like with doctors. I'm in pain. Like what?

Monika: The game is an organism. This is also a metaphor that I use. This is a living organism that each change, each environment, even if you are using different headphones that we used at work, it can break something, it can change something, and we don't know about it and we can look for the solution. Not seeing anything breaking for us because of this one small thing. So yeah, the more data, the better and we can help.

Piotr: And in this post-launch stage, I really appreciate the fact that the players come to us and tell about the problems. Of course, we want to avoid the situation. The less interaction with players sending us bugs better. But at the same time, this is how we listen to them and how we improve the games for them, right? So I truly believe that our games are as good as they are because we listen to their feedback and we just incorporate changes based on that, right?

Monika: We try.

Paweł B: Yeah. And we've always been open with players. I always like that. Because I remember, I always tell this story before I joined CDPR. I was playing Witcher 3 at home and I had one bug before patch 1.08, something like that. And it was, Lambert in Kaer Morhen would fall under the ground and I couldn't interact with him to progress the main quest. And instantly I was asked to send a log of what happened. And then I had a community manager reach out to me and say that, okay, this is going to be handled in the upcoming update. Told me when the update is going to happen. The update happened, the thing was fixed, and I got all this information. Well, sometimes I've always been in a position that I would have problems within games, and either it was very hard to talk to someone that is actually from the studio, ask me for all these things and then have the reassurance that these things are going to be fixed. So I've always liked that we do that. And I also love that we never kind of say, no, like, you know, in the sense that, okay, we know that, for example, the game has a lot of bugs, but we'll be going and going and improving, improving and fixing things until we are satisfied with it and we see that players are satisfied with it and kind of like, you know, the resilience of never stopping. And I think that's super cool. And that's mainly thanks to QA, right?

Paweł M: Does it happen that we cannot reproduce something? There's some super rare bug that's happening and nobody can find it.

Monika: Yeah, yeah, yeah. Mostly when we don't have the data, how it happens and we cannot reproduce it. Usually when we have a save file or something like that, it's easier. But if we don't know what happened, like what the steps before it may work for everybody, or it's about the equipment because sometimes the equipment, the hardware, can also do something. So we just need to know about stuff and we hopefully can reproduce it. But also there are situations that we cannot, but we try, we try to do it.

Paweł B: Must be easier for sure with consoles because everybody's using the more or less same hardware, but also depends which patch are you on? Some people don't download specific patches or they play like super vanilla versions.

Monika: Mods are kind of a pain in the ass for us because they're breaking the original environment of the game. So you know, if you have mods and you have problems, we may not help you the way that you think we will. So yes.

Paweł B: That's why whenever we do updates, we tell players to like, hey, if you have any mods, you better get rid of those and then reinstall them and then see if it works. But you need to kind of like once you update to a new version, those most likely won't work. And then the modders need to kind of update their mods to fit the new version of the game. So it's like, it's a whole process.

Monika: The game is a very delicate organism.

Paweł M: It also could happen that the drivers, for example, are— So the game is broken because of totally external software.

Monika: Even for us, we have to refresh our drivers from time to time when there are new ones.

Piotr: So that's why we also always recommend to, you know, make sure that you have the minimal requirements.

Monika: Like the requirements are also important if you're not playing on at least minimal ones.

Paweł M: We can't guarantee.

Monika: Exactly, we can't guarantee that we can help you. Again, we can try, but there is no guarantee if you are not playing by our rules, there is no guarantee that we can help.

Paweł B: How do you guys keep morale going? Because I feel like you're covering a wide aspect of everything. And like you said, the game is a living organism. It's huge. Before release, I know that everybody's like, you know, stakes are high. Anticipation is incredible. Everybody just wants to get the game out the door. There's a lot of testing happening. There's a lot of bug fixing happening. How do you keep everybody going? Because I know that QA is one of the hardest working parts of a gaming studio, and if someone tells me it's not true, I'll tell them it is true because you can tell how many things are getting fixed super quickly. But how do you keep people going and kind of keep morale going in the team?

Piotr: I will start with this, you know, you're right. Sometimes this morale can drop faster than anywhere else because we also see the negative side of the project, of the game. We see bugs, we see problems. And there are sometimes many of them, which is natural. But if you see the problems everyday, you start thinking like there are only problems, right? My approach in this is to try to show people the bigger picture. We do it for the bigger thing, right? For the greater good, how do we say it, right? For the good of the game. And sometimes the morale boosts when you achieve something, right? A little thing, for example, the build doesn't kick off, it looks very bad, but you meet with the developers, you brainstorm, you go to the logs and suddenly, yeah, it works. Let's celebrate. I don't say we always celebrate. We should celebrate more, but I see people really have the spark in their eyes, if they think if they manage to fix or find a way to fix a little thing, like if you look at from very high perspective at this little thing, nothing, a peanut, right? For some people, this is a lot and we celebrate this and this kind of thing probably, in my opinion boosts the morale up. I don't know about—

Monika: We are problem solvers when we solve the problem. This is a success for us.

Paweł B: That's the dopamine hit. Yeah, we did it. Boom.

Piotr: Because obviously it's great to be patient and strong to wait till the release. But you know, release is like sometimes years of work in trenches, right? You need to find—

Monika: But the satisfaction after the release. It's great.

Piotr: It's huge. If everything is as we expected, it's probably huge. Not probably. It's huge. But you need to find a way to find the satisfaction in the pipeline. Like when the game is not ready to be launched, right? From the other side I would add that we also, we gain our boost from appreciation of others. Like if other people in the company, the developers appreciate our job. And you know, not doing that basically, it's like we feel that we are doing the right work. If somebody says, I don't want your bug, you are not correct. This is working for me. This is not a bug. I will do it my own way. Your ideas are shit or something like that. This is not a good thing. But the appreciation for what we do, especially if we're not the decisive people. We're supporters. So we stand in the shadows, let's say, with the ninjas in the shadows, making the game working and the quality working. And again, what I said, working with different characters, it's hard. So sometimes we get different reactions. Uh, the better the appreciation ones making us work better. And the morale boost is coming.

Piotr: No matter how hard it is, people— that's the nature of human beings. Like, we want to hear that our job was valid, right? And if someone was happy about that. This is a little thing. Doesn't cost too much. If you hear that from time to time, it's great. For sure.

Monika: Simple things. It's, yeah.

Piotr: Simple things, little things. But we shouldn't forget about them.

Paweł B: Yeah. Too many times when I've been to any type of demo we were doing or hands on, there were so many times that a QA person would just solve things on the go. Like downloading the latest build like an hour before we go on stage and still some, you know, final checks and stuff like that. But I always see QA people as just fixers. They pretty much get stuff done. They provide you with the build they tested, they know it's solid and then you just go and everything works, which is incredible.

Paweł B: Two more questions before we wrap up because QA probably, at least in my opinion, has all the bloopers in the world. Like you guys and gals probably collect everything. You have a million videos. Is that true or false?

Monika: It's true, it's true.

Piotr: Yeah, it is.

Monika: Since I was working, we collect those in our folders and we do this for us obviously to have like an archive of things— So many players would want access to that vault.

Monika: After Witcher 3, there was this video.

Paweł B: There was a blooper.

Monika: And this was the video actually created from our bloopers, from our Jira. So it's also useful outside of our stories. Yeah, yeah. So yes, we are collecting that. Obviously, the whole bug tracker thing is like collecting all the issues, but we're collecting the most bizarre ones that you know, the ones that are unique, basically.

Paweł M: Do you recall the craziest bug, or your favorite?

Monika: My own no, because it was so long ago that I was reporting bugs. I now work more with the people than with the game, unfortunately. I love working with the people, but I don't have time to work with the game and report the bugs. But I asked my team. I usually ask them, what are the bugs that you're reporting, which do you remember. There is no one bug because there are like categories that I will tell you the funniest one. So like things being stretched among the whole level, like part of it, you know, one finger being stretched among the whole level, among the whole game. Or problems with textures were like those, uh, blue and red thing, disco thing.

Paweł M: Or faces, you know just breaking completely.

Monika: Faces, NPCs being bald, more visual ones are the funniest ones. Obviously all the animations being crooked and stuff like that.

Paweł B: Wonky.

Monika: Yeah. Being wonky. There is nothing that I can say like this one thing was the most funniest one because we have thousands of them. But for example, there are things, those were the visual ones. But we also use sometimes weird naming convention. For example, there was a bug when the lamp was called pillar of light. So it's also hard to find those issues. Pillar of light is... it is pillar of light, but—-

Paweł B: It creates a pillar of light.

Monika: It's harder to find. There are also the bugs that we did for example, Cyberpunk connected to the sex scenes because we wanted to make it realistic. And there are different sex scenes, so there are bugs explaining what the sex scenes should look like in the real version. There is like a lot of stuff about the physics of the body, for example, as well. I don't remember the bug, but I can say that I was very heartily an advocate for Takemura romance in Cyberpunk. So there is much feedback from me about it. Not a bug. So yeah, this is something that I remember. And still waiting. Still waiting. Maybe it will happen.

Paweł B: There's a lot of people. There's a group of people that is advocating for that.

Monika: I wanted to make sure that they will get what they want before that. But sorry guys.

Piotr: My favorite, you mentioned physics. My favorite category of funny bugs is like physics. When we learned like, there are new rules of physics in the world, right? Destructive tests.

Piotr: So levitating characters, stuff like that. It's good if you see it only during development, right? But sometimes you see it after the development.

Monika: It's not so good.

Piotr: Not in our games.

Paweł M: Not so funny.

Piotr: That's a funny thing.

Paweł B: What about for the last question, what about for people wanting to get into QA? Like what set of skills do you need to have? Because I feel like you need to kind of cover almost everything. And could you also talk about kind of QA being the, you know, the first step that you can take and then from there go to different departments within a company and kind of, you know, work in different parts of development.

Piotr: I was waiting for this question. Let's answer it together. I can start. I think that there is this concept of QA being the entry level for game dev, which is true in my opinion, and which is great. But of course, we try to make sure that people stay in QA, right? If they don't stay in QA, it's also okay because I really love to see designers, artists, or producers who are coming from QA because they already have this foundation, it's easier to work with us if they have this background.

Monika: Appreciate our work.

Piotr: And they appreciate it, they really know what it takes to be QA. And I think this is one of the misconceptions that not everyone could be a QA. You need to have like a certain like approach to—

Monika: Attention to detail, investigative like pattern recognition stuff. Patience, patience, patience. Resilience. Yeah. All those.

Piotr: Finding the way to communicate and confront. Your question was about—

Monika: What skills.

Paweł B: Skills you need.

Piotr: So of course there's this set of standard skills like attention to detail, being eager to investigate, having the analytic mindset. But I think that what else or maybe key nowadays is to be able to confront and question or maybe question and confront, because QA is about questioning things. And when you question things, you need to confront yourself with other people. It's always difficult for everyone, no matter what you do, no matter if you are QA or producer or whatever, you need to say that, no, I think it's different because facts, right? So be able to get prepared with the facts. Facts can also support your subjective opinion later on. Start with that. And I think that it also touches something that I love to say to the people I work with. This is the attitude, right? The attitude is the multiplier of all the skills and knowledge you have. If you have the right mindset and you are just easy to work with, so you are easy to communicate with, that's great. This attitude is also being proactive, being confident. Responsible. Accountable. Committed. We agree on something we commit to do, we do it no matter what. Sometimes it's difficult, right? And this attitude touches very much our values in my opinion. If you have to some extent, a portion of you is like our values, then maybe we can remind ourselves of our values at this point, right? So be ambitious, of course. Set the goal and persevere. I feel like I'm in school.

Monika: Be honest. We need to be honest because we work on data.

Piotr: Be kind and respectful to all around you.

Monika: And the gamers thing.

Paweł B: Always remember about gamers.

Piotr: Remember about the gamers.

Monika: Because QA should be a gamer. So people working in QA should be playing games in a different genre of games, not only focusing on one, but we need to compare. We need to be able—

Piotr: Able to support this and show it in your everyday action. Come, come to us.

Monika: Because you can learn. You can learn all the engine stuff. You can learn all the, you know, debugging, testing, different tests, kind of tests, but you cannot learn soft skills. You have those soft skills in you.

Piotr: And I think that nowadays, especially when we are in the environment of, you know, we as our company works on multiple projects and we grow in scale. QA will also grow in scale, but what's really important is to be flexible and adaptive. And everyone has to be open for learning new stuff. Switching gears, like for example, now in project X, you work as a QA on this and that, but be ready that within the new project, which is like a different project, different requirements, you will have to do a slightly different job, right? So be ready for that. Flexibility I think is—

Monika: Open to learning.

Piotr: Yeah, yeah yeah.

Paweł M: We know everything.

Paweł B: Well, I feel like there's—

Monika: We could be talking for hours. Yes, exactly. If people want, we can do it again. Additional questions.

Paweł B: Monika, Piotr. Thank you.

Piotr: Thank you.

Paweł B: This was really amazing.

Monika: Thank you for inviting us because there's not a lot of QA talks and discussions online.

Paweł B: Now there will be. Because it's a super interesting topic. And like I said, real heroes, in the game dev space.

Piotr: QA is a real hero.

Monika: That's true.

Piotr: Thank you.

Monika: Thank you.

Paweł M: Thank you so much for watching. Today, we cleared up a lot of misconceptions about the quality assurance team and understood better how this team works from the inside.

Paweł B: It was an incredible talk and I hope you guys enjoyed it. And as always, don't forget to comment, like, subscribe, all that jazz. Let us know what you're thinking about the episodes and we'll see you, of course, in the next one. Bye.