QA Testing Environment Changes

Hi Everyone, This is Part 2 of a 3-part blog. If you haven’t read Part 1 yet, I recommend reading it first! You can find it [url=https://steamcommunity.com/games/1148810/announcements/detail/3708190655184969941]here[/url]. [img]https://cdn.akamai.steamstatic.com/steamcommunity/public/images/clans/36144418/ae5e276f1cceecbae6949655003f6633e41ce873.png[/img] I have a vision for Quality Assurance. I want to build a [i]task force[/i] for internal gameplay testing before an update reaches the live servers. To this end, I have requested a better Internal Testing Environment. It will also help other team members in creating their own QA teams. Up until now, our outdated QA systems meant that whenever we wanted to test a newer version of the game, the live servers had to go down. This was and is unacceptable and I would like to apologise. I hope it never happens again. Due to that limitation, we had limited opportunities to test the updates before they reached you, the community. Even when we did perform tests, we found only 1/10th of the issues that were reported to us after the released update (TFE 4.4). It’s crucial for us to be able to test with more players first and on a wider range of machines, before releasing it to the public. [img]https://cdn.akamai.steamstatic.com/steamcommunity/public/images/clans/36144418/1c3f4882b084a6394ee3eb5001bb37bb154230f1.png[/img] The team goal is to have an entirely separate master server for the testing environment and update that testing environment more often, with an automated process to pull and build from our project source so that any of us on the team and selected individuals can simply jump on in their own time to test changes and provide feedback internally. This will be especially helpful for our team, since members work different hours across multiple time zones from around the world. For me personally, the goal for the Testing Environment is to build a flexible team of community testers that I know will be able to provide very good critical and constructive feedback on the changes being brought forward for discussion. This will be a layer of QA testing before it reaches the public. I already have a list of a dozen or so individuals who would be suitable for such a task, who would potentially help me to reproduce gameplay issues and to provide feedback during the rougher stages, and I’ve already spoken about this with a few of our pre-alpha supporters in the past. I want to make this team a reality; it’s the first major request from the Design Team to the new Programming Team. If you’d like to volunteer your time, you think you’re a suitable candidate to have access to our internal test branches and would like to work with me by providing us with vital feedback in the pre-alpha stage of an update, send me a PM on Discord. I’ll take a look and evaluate your application. Should you ever want to make this your career and enter into the video game industry, do please tell us and we will consider taking your cooperation a step further in the future. [i]Join our official Discord server [url=https://discord.gg/redspear]here[/url]![/i] [img]https://cdn.akamai.steamstatic.com/steamcommunity/public/images/clans/36144418/362ee3df906263fd8f808cb5031448b3868d2b93.png[/img] The format of the QA process should be in this order: [list] [*] [b]Live Game[/b] [*] [b]PTR [i](Public Test Realm)[/i][/b] - [i]a downloadable and publicly accessible wider test branch to stress test the servers[/i] [*] [b]ITR [i](Internal Test Realm)[/i][/b] - [i]a standalone limited-access Testing Environment for selected individuals[/i] [*] [b]PIE [i](Play in Editor)[/i][/b] - [i]what we use to test basic functionalities before pushing changes, often providing different results than standalone servers[/i] [/list] One could argue that in its current state, TFE has the same function as the PTR. This will change after full release. Since Early Access is essentially our PTR, as some of our communities have mentioned, as TFE grows and as we expand our target audience, Early Access will become the Live Game, and our experiments will need to be tested first in ITR, followed by PTR. Therefore, the testing environment will be a vital piece of the puzzle to get absolutely right, that is if we want to increase and maintain/restore the quality of our updates, and be able to test our full builds more vigorously and robustly before they go onto the Live Game servers. [img]https://cdn.akamai.steamstatic.com/steamcommunity/public/images/clans/36144418/e4a40ad9c995cd6d085286e8070ac32843c309f1.png[/img] A rough outlook shows that we’ve got about half a year of work on the updated Unreal Engine version done. The progress is not lost; we have simply reverted the Live Game back to Unreal Engine 4.23 (TFE 4.3.1) so that we do not cut off the playability of our game. With that revert done, once we have a fully functional Testing Environment, we’ll be able to properly test the updates on the version of the codebase and Unreal Engine (4.25, 4.27+). These tests will hopefully allow us to catch any crashes or common issues for our programmers. In theory, this will let us continue to upgrade to newer versions without affecting the live playability of the game. We will also have a wider spectrum for testing large builds, such as the 4.4 build. Right now, the priority is to be able to compile and build the latest version of TFE builds on newer versions of Unreal Engine: to be able to play a few maps in a row without crashes! [i]-Demonic Stay tuned for Part 3 of Demonic’s development blog.[/i]