Learning to Identify Perceptual Bugs in 3D Video Games

Introduction

It is possible to identify a range of perceptual bugs using learning-based methods by making use of only the rendered game screen as seen by the player.

Automating software testing is a long standing problem in software development. The problem is however notably different in video games when compared to other software. Video games have attributes that most other software do not, they tend to present a narrative, have a rich presentation layer, and are highly interactive.

Hierarchy of video game bug identification showing automation difficulty. As usual, low level bugs, many internal engine and some game bugs may be identified by developer written conditions. However, perceptual and behavioural bugs require more sophisticated mechanisms for identification, the current method is to rely on human testers.

    Perceptual bugs are bugs that manifest in a visual or auditory manner. To give an example, the rendering of shadows is a procedure that relies on vast arrays of numbers (geometry, normals, light maps etc), it is wholly impractical to write criteria of the same form as for example the player position.

World of Bugs

    World of Bugs is a training and test platform for automated bug detection for both exploration and identification. It has been developed in Unity and makes use of the Unity ML-Agent package. The platform consists of a 3D game environment, support for developing explorative agents, and a collection of perceptual bugs that are game independent.

It was particularly successful in identifying missing textures, boundary holes, camera clipping and corrupted geometry, as well as the significantly easier black screen bug. But was less good at identifying some of the more subtle bugs, those that did not have a large impact on the observation or that look similar to normal experience. This is perhaps best highlighted by the geometry clipping bug, a bug where the player is able to walk through objects. Since the player is not supposed to be inside a geometrical object, there is no reason to texture the inside making everything look normal.

Bibliography

https://arxiv.org/pdf/2202.12884.pdf

.