User interface design in video games

User interface design in video games

By Anthony Stonehouse



User interface design in games differs from other UI design because it involves an additional element — fiction. The fiction involves an avatar of the actual user, or in this case, player. The player becomes an invisible, but key element to the story, much like a narrator in a novel or film.
This fiction can either be linked directly to the UI, partly, or not at all. There seems to be a growing debate with which is more immersive for the player. One side of the debate is that UI elements that sit within the game world, and therefore ‘viewable’ by both the player and avatar, are the more immersive and provide a more seem-less experience. However these elements are often clunky and inherit many restrictions. They need to strictly adhere to to the fictional narrative and adopt the visual design of the game’s art direction. They also sit within the geometry of the game’s perspective, often rendered on a 3D plane that can reduce their legibility.
Erik Fagerholt and Magnus Lorentzon explore these theories in their thesis for Chalmers University of Technology titled: Beyond the HUD — User Interfaces for Increased Player Immersion in FPS Games. They introduce the term diegetic interfaces for UI elements that exist within the game world that the player and avatar can interact with through visual, audible or haptic means.
I believe well executed diegetic UI elements do benefit the player by enhancing their experience of the narrative. However, they are more difficult to implement because the inherit restrictions.
Metro 2033 uses a complete diegetic UI with no HUD elements to help to support the game’s narrative. It does seem to run the risk of frustrating the player though slow response times but perhaps this forms part of the game mechanic, I have yet to play the game.

Metro 2033's diegetic interface

A well executed example that I am familiar with, and one that is often used, is the act of interacting with the phone in Grand Theft Auto 4. It mimics the real world interaction — you hear the ringing and there is a delay before you answer it, the act of answering itself is often an awkward process in reality. The game’s fiction adopts real world fiction so perhaps it makes diegetic elements easier to integrate.

Grand Theft Auto IV mobile phone interface element

UI elements can also exist in the game world and not be viewable by the player’s avatar, therefor avoiding the clunkiness of having to interact with the avatar (but also avoiding jumping out to menus) and the need to strictly conform to the game world’s art direction. They still need to follow the rules of the game’s fiction so they help immerse the player and provide a seamless experience.
These elements can either exist within the geometry of the word, or on the camera/screen for the player. Fagerholt and Lorentzon use the term Spatial to define elements of the UI that sit in the geometry and Meta to define elements that exist on the camera.
Fable 2’s dotted line that guides you to locations marked on your map is an example of a Spatial element, it exists in the game world but the avatar doesn’t interact with it. It ensures a more seamless experience, avoiding jumping out to a map screen as often.

Fable 2's glowing trail

Splinter Cell Convictions also adopts some nice Spatial elements in the form of projections that illustrate objectives within the game world. Their scale does seem to challenge the fiction slightly more than other examples.

Splinter Cell Conviction's objectives projected in the game world

A common example of a Meta UI element is blood the splatters on the screen as a form of health bar, as in the recently launched Modern Warfare 2.

Lastly there are the traditional non-diegetic UI elements. They have the freedom to be removed completely from the game’s fiction and geometry and can adopt their own visual treatment, though often influenced by the game’s art direction.
I think these elements are best used when the diegetic, meta and spatial forms provide restrictions that break the seamlessness, consistency or legibly of the UI element. World of Warcraft has a pure non-diegetic UI, that adds nothing to the fiction. It does allow the user to completely customise though, hopefully ensuring a familiar experience.

Here are some visual representations of these concepts that have been adapted from Fagerholt and Lorentzon’s thesis.

For further reading; Marcus Andrews from DICE examines recent games using these theories from Fagerholt and Lorentzon in this article posted on Gamasutra. Michael Grattan, a senior at the University of Souther California writes a response to the article on his blog. Dave Russell has also written an article on his blog.
New physical interaction methods make their way to the PS3 and 360 this year, called Move and Natal respectively. These technologies promise to challenge current best practice in game UI design and generate even more innovation in the area. As a designer currently working in web design I’m almost jealous of the levels of interaction plus the addition of audio and haptic elements available in game UI — while I’m still refined to users with an outdated keyboard and mouse. Web does offer other challenges and more variety in content delivery though.