30th November 2020 – 4:44pm
Following the many setbacks at the release of patch 1.11, we wanted to contact the developers of Valorant to better understand what happened. We had the chance to chat with Joe Ziegler, Valorant Game Director at Riot Games. It was an opportunity to learn more about how patches are created, the problems encountered by different development teams and how they can be avoided in the future.
Note: These questions were asked at the beginning of November, but various scheduling issues delayed our exchange. Some points have already been discussed publicly by Riot Games, but Joe Ziegler gives us new details on various processes and upcoming projects.
Mandatory: How do you set your goals for each patch?
Joe Ziegler: Game development on VALORANT works in a lot of layers, and the convergence of a lot of those layers happens in patches. When we target a patch we’re usually folding in a lot of different goals, per team, into that patch. Some of those goals are ones we’ve set down months before and those often take the form of feature launches, feature updates, or even content releases like skins. Other goals are more emergent and recent, based on feedback we’ve seen from the live build and data we’ve analyzed against that feedback. Those goals often take the form of balance updates, fixes, and small feature updates.
Mandatory: What is the typical roadmap for a new patch, from a developer standpoint?
Joe Ziegler: At a baseline we try to release a patch every 2 weeks, , as that is about the minimum length we need to prepare a set of changes, test for stability, and distribute worldwide. Of course when there are game breaking emergencies, we will release hotfixes as soon as we can to ensure the game stays stable.
Mandatory: What happened with Patch 1.11?
Joe Ziegler: Patch 1.11 was an interesting case where we ran into a bunch of problems that came together at onceOne problem was that we had prepared a series of changes for before First Strike qualifiers, but communication got lost a bit on where it would land relative to First Strike, so we ended up releasing it after qualifiers had started.
This made us appropriately hyper sensitive to any disturbances it would cause, largely because we didn’t want to disrupt the qualifiers and make it harder for competitors who had been practicing so diligently to all of a sudden run into issues that would disrupt their practice.
The second problem came from defects that we were unable to catch in our test process. Our test team is insanely hard working and talented. They have helped ensure over 95% of the accidents that the different teams make never reach players. But at the scale we operate at, which is in the hundreds of people, and the scale that the audience plays at, which is in the tens of millions, there are new variables that expose things that we couldn’t catch internally. For example, with the ranked badge performance issues, we never caught these internally because our test accounts didn’t have full ranked badges. It was a condition we hadn’t met yet for internal test accounts, which are reset often, but when it went live it became something that innumerable amounts of players had. Every time we learn of a new variable, we try to incorporate that into our test process so that we can ensure we learn and grow from that mistake.
Mandatory: Why do Valorant patches tend to break the game, characters or maps?
Joe Ziegler: This is definitely a challenge with live service, and I believe any game, especially a game in it’s early life cycle being distributed and played at a worldwide scale is going to run into. Having said that there’s no doubt that VALORANT has run into many issues releasing patches. Some of this is the challenge of operating at the speed we need to to release content and updates to the live service. We have a lot of teams each pushing new things into the build daily as they work diligently to meet the needs of players, and the sheer volume of changes leads to a high chance of conflict between different pieces of the game.
Additionally, there is also the reality that dev and test environments operate somewhat differently than live environments. Players have an endlessly wide variety of hardware, software, and internet configurations that expose problems that we didn’t know existed in our internal development environments and set ups. Ultimately with all these factors, running a live service is about discovering these issues and learning from each one to ensure you can prevent these things from happening again, and if they do, identifying and fixing it faster.
Mandatory: Does working remotely have an impact on the patch quality and Valorant development?
Joe Ziegler: Absolutely, it does. In some ways working remotely slows things down and makes communication harder and this challenges our ability to operate efficiently. But just like any other challenge, we’re constantly trying to evolve to meet that challenge, and discovering new ways every week to improve our processes to account for this difficult work environment.
Mandatory: Do you think the release of a PBE Server will help with patch stability?
Joe Ziegler: I think having a PBE will definitely help to identify issues that have to do with hardware compatibility and software conflicts, as well as issues that have to do with operating in a live network environment. This will definitely help to make a large chunk of issues easier to identify before they go to the live build. Having said that, it’s important to remember there’s no single solution that will make all bugs go away. And so we’ll still have to remain diligent, learn from those mistakes we identify and evolve as we go along to ensure that we keep the game stable and healthy.
Mandatory: Will the PBE be open to European players?
Joe Ziegler: We’re still sorting out the details of the PBE and can’t promise anything, but it’s likely that Europe will be a place we do a lot of testing, given that we have so many passionate players there (and a shout out and thank you to all those players!!).
Mandatory: Have you considered slowing the release of patches, especially during competitions?
Joe Ziegler: Absolutely, we’ve actually done exactly that during First Strike. We’ve held back on some of the balance changes and additions while First Strike is going on, and are accumulating some changes we’re looking to make into a post-First Strike patch that will occur in December.
Mandatory: Are you considering closed servers dedicated to official competitions?
Joe Ziegler: Yes, in some form we’ve already done this for First Strike, with limited environments in each region participating. This is something we’ve set up as a somewhat temporary measure while we research how to set up a more permanent solution in tournament servers or something similar to that. The goal here, of course, being to ensure we can provide an environment that is more controlled and can selectively have a more closed and stable set of changes that can be fixed for a bigger tournament cycle, something like First Strike. We still don’t have an ETA for that tournament server, but we’re investigating what it could be right now.
Thanks again to Joe Ziegler and Riot Games for giving us their time to answer your questions.