A biannual magazine exploring light in all its forms
Tracy Fullerton is an experimental game designer, professor and director emeritus of the USC Games program. Her research centre, the Game Innovation Lab, has produced several influential independent games, including Cloud, flOw, Darfur is Dying, The Night Journey with artist Bill Viola, and Walden, a game-based simulation of Henry David Thoreau’s experiment at Walden Pond.
THE LIGHT OBSERVER : You are described as an experimental game designer. First, I was wondering what led you to video games? Secondly, what led you to experimentation and what sort of experiments are you interested in?
TRACY FULLERTON : I was actually trained as a filmmaker (first at UC Santa Cruz and then at USC Cinematic Arts), and was focused on experimental narrative, but I was also interested in digital technology. At the time I graduated school, there were a lot of opportunities in the emerging games and digital media field, and applying my ideas for experimental narrative structures, together with the interactivity of games, seemed like a fascinating challenge. At the time, it was unusual to find someone who had that combination of technical skills and cinematic skills, and so I found a lot of opportunities early on, quickly moving from an entry level position as a designer to a creative director. All of the projects I worked on early in my career were explorations, mixing the cinematic with the interactive, and my goal has always been to create experiences that evoke more subtle emotions than what we find in most commercial games.
The games you’ve worked on, such as Cloud and flOw, revolve around the idea that video gaming can be a relaxing and sensorial experience. What are the potentials of this rather young form in creating powerful and enriching experiences?
Games and play are aesthetic forms that are as old as storytelling in human culture, but until recently, we haven’t recognized their potential power as emotional, expressive experiences. The way that digital media can now be combined with traditional game structures has meant that games have evolved to include cinematic elements as well as playful elements. As we learn more about how these can be used together, we are seeing the potential of play to evoke rich story worlds and emotionally powerful interactive experiences. It’s important to realize though, that these are not entirely new aspects of play; play as meditation, as a spiritual practice are all part of the history of analog games in culture. Look at Go, for example, or even the early industrial game Mansion of Happiness, which are both examples of meditative or spiritual play. What we are doing in The Night Journey is a part of that history and evolution. When we first exhibited the game in 2007, there were virtually no other video games of this kind, but as the field has grown, there are more game artists exploring the varieties of meditative play.
Video games tend to borrow from cinema – there are obviously movies that become games and vice versa – but beyond that there is an affinity with the way stories are told; a beginning, a journey, an ending. From an aesthetic point of view many contemporary games seek the verisimilitude of the video camera. How do you feel about the influence and relationship between cinema and video games?
I would argue with the premise that all video games seek verisimilitude – that realistic aesthetic is part of a very particular genre of game: AAA first person shooter games. Very different aesthetics in games also exist, that reach for abstraction and conceptual artistry which are equally important to consider, especially when we are talking about the relationship between the cinematic and the ludic. One of the wonderful things about intertwining these things is that we can draw on all aspects of cinema – the surreal, the experimental, and, of course, the narrative aspects of the form. And when we merge these with the ludic, the playful, we find that we can design games that invite players into worlds that go beyond the limitations of the frame. One of the core design pillars for The Night Journey was that we were creating an ‘explorable video’, a world that had the emotional sensibility of the cinematic, but also the playful potential of a game world.
Bill Viola began working in video art in the early 1970s. Coming from a different field, what did he bring to you, and what did he learn from you (by you I mean the whole team working on The Night Journey)?
The really amazing thing about the process of making The Night Journey was the way the team was able to meld our different processes. Bill and Kira brought their artistic process – deep visual thinking and profound spiritual inspiration – to us, and we brought our game design process – prototyping and iteration of the experience – and we were each inspired by the other. I remember the first time we brought a paper prototype of the game to play with Bill. It was basically a board game that allowed us to simulate what it would be like to play before we had built any software, and I think he was surprised at the way we worked on paper to begin the design process. But soon, as we played that prototype and others after it, the team was able to have productive conversations about the nature of the project, how we could use game mechanics to explore the spiritual journey, and then later, as we built digital prototypes, to find the unique look and feel of the world as an extension of Bill’s prior work.
Video games are based on entertainment. Video art would be based rather on passive interaction and contemplation. How did you combine those two approaches?
Video games can be pure entertainment, that’s true, but so can video itself. To say that our current view of a medium determines the extent of what can be expressed with it seems very reductive to me. Video art shows that video can be used for more than pure entertainment. And video game art shows the same for this medium. Play itself is not constrained to any particular tone or mood – we are playful when we look for patterns in the clouds, or when we make rhymes with language. Games can express many different tones and moods of human playfulness, and that includes contemplation and reflective thought. Just as walking allows time and space for our thoughts to wander and play, so too can exploration in a game world allow us that same opportunity for flow and thoughtfulness. Along with the concept of explorable video, we also strove to create a sense of embodied movement in The Night Journey. Years before the term ‘walking simulator’ was coined to describe games where walking was the core of the experience, we were creating a simulation of a pilgrimage in our game. We looked at footage from Bill’s prior work, where he wore the camera and was exploring with it, finding compelling visual patterns and moments in the landscape, and we modeled the player movement in the game on this first person footage. Unlike most first person games, where the camera moves robotically, smoothly, and very quickly through space, we created a purposeful bob and sway to our game character that made it feel like the view of a real person, moving at the speed of an actual body in space. This sense of walking and thinking, exploring and reflecting, formed the core mechanics of the gameplay.
The Night Journey team began to work on its concept in 2005. It took about ten years to finish the project. Can you describe the journey it took to make this video game?
We created an exhibit version of the game that was released in 2007, and that was shown worldwide in a number of venues, including the ZKM in Karlsruhe, Germany; the Museum of Design in Atlanta; the Nam June Paik Art Center in Seoul; the XXI Triennale International Exhibition in Milan; the IndieCade Festival of Independent Games; the Museum of the Moving Image, and more. For just over a decade, the game was only available to play as an exhibit piece. But in 2018, we created home versions of the game for PlayStation, PC, and Mac and released it widely. Creating the home version of the game was a labor of love on the part of the team, as our original funding from the National Endowment for the Arts and the Annenberg Foundation only covered the exhibit version. The team here at the Game Innovation Lab was committed to bringing the game to a wider audience, though, and so we worked on it in our spare time as we were able to do so. Recently, during COVID, we were able to localize the game for PlayStation in Europe and we have just released that this month, which is exciting.
What was your design process like? What were some of your reference materials?
The design process started with Bill’s initial vision, which had first begun with a team at Intel research. The document they created described the feel of the experience, the themes, and potential levels, but it didn’t have a design for the core activities, what we call ‘game mechanics’. What our team at the lab brought to the process was the understanding that game mechanics can express ideas and emotions. We needed to develop mechanics that made the player feel like they were going on a spiritual journey. This process took place over many discussions, long conversations about the nature of that journey, exploring ways that we might communicate its essence through the gameplay. The reference materials came from Bill’s prior work and from his journals and sketches, also from references he had collected to spiritual texts. We used this material during our early prototyping as we discovered both the essence of the gameplay and the feel of the world. We were especially captivated by Bill’s work with a low resolution security camera and we felt like that look could lend a beautifully mysterious aspect to the game world.
The perception we have of reality is highly based on our visual senses; light is therefore what makes the things around us exist. What is it like to create the light of a video game? What does that imply?
Light in The Night Journey is a mixture of dynamic and rendered elements; we are playing with light in many different ways that leaves the player uncertain about where and when they are. Many of the elements in the game, like the trees, have had shadows pre-rendered on the terrain they sit on, so that there is an illusion of a light source, like the moon we see in the sky. The reflections appear as translucent memories, blended with the terrain, and they have their own light, which is emphasized and which grows the longer we reflect on them. And then, of course, there is the darkness, which is a deep fog that overtakes the world, encroaching slowly at first and then with more impact as it absorbs all of the light around us and finally converges on us to bring on the dreams. We use a mixture of techniques for these lighting elements, some of which are rendered into the 3D elements, and some of which are procedural. The overall effect intentionally leaves the player in a kind of visual limbo that is unlike the experience of most video games.
A strangely lighted atmosphere, between twilight and dawn, reigns over The Night Journey. What kind of feelings were you trying to convey thanks to the lighting?
As I mentioned, the game takes place in a liminal space and time – is it dusk followed by night, or is it night followed by sleep? Both of these are true and yet do not fully describe the uncertainty of this place. The look and feel of the game is striving to create a sense of the ‘bardo’, the state of existence between two lives on earth when one’s consciousness is not connected with a physical body, and we experience a sequence of phenomena ranging from the last breath to the luminosity of true nature. The cycles of the game, wandering and dreaming, loosely represent the six bardos of Tibetan Buddhism, with each of the dream sequences bringing in imagery from our ‘life’ in the game to the universal themes of each stage. Players do not need to know or understand these references to feel the journey through these stages of the bardo, but they do understand that though they begin the game at a human scale – walking, almost creeping in the vastness of the world, and reflecting on its nature – they transform over the cycles of the game until they have broken their physical ties to the game world and have chosen to walk into the final light. The emotions of this journey – curiosity, wonder, confusion, fear, awe, freedom, and acceptance – depend on the player and their commitment to the experience. As the opening quote from St. John of the Cross says:5 ‘Human science is not capable of understanding it, nor experience of describing it. Only the one who has passed through it will know what it means, though there will be no words for it.’
The game plays with the symbol of darkness and enlightenment. What lies in the darkness? What emotions does the night evoke?
The game takes place in a liminal space between day and night, light and dark, that is neither and both at the same time. The world feels like it exists in a perpetual twilight, with darkness encroaching at the horizon. It is meant to give a sense of unease and transition, even as we gain the power of flight and perspective, loosening our bounds with the earth, and giving way to a sense of freedom and lightness where we glide effortlessly. The cycles of the game always begin with a human perspective, locked to the ground, and gradually transition to a soul’s perspective, as we progress and discover. Each time the darkness falls on us, we return to that human perspective until we learn to appreciate both freedom and constraint. The cycles are punctuated by dreams, which begin as somewhat mundane recapitulations of the things we’ve encountered in the world, but over the cycles grow to include a wider perspective of life and death, love and violence, spirit and flesh, and finally the caravan of light, other pilgrims, making their way into the final doorway, transitioning out of this life.
Can you tell us more about the dream sequences?
The dream sequences are the only place in the game where we have original footage for the game. The caravan of light bearers, which leads into the final sequence where the player enters the light and the last doorway, transitioning from the cycles of wandering, is a sequence that Bill gave us specifically for this game. All of the other videos are echoes of prior work – they may not be the exact shots used in other pieces, but they come from the vast archive of work, the pool of Bill’s imagination that he gave us as inspiration for the piece. Bill also designed a procedural system for these sequences that, as I’ve already described, evokes the sense that we are passing through the six bardos, transitioning between two lives.
How has light become a narrative tool?
I don’t know that I would call light a narrative tool in The Night Journey, but it does play a significant part in the experience. From the way in which the light blooms through the reflection sequences, to the final walk into the candlelight, the player is guided by light and in some cases the absence of light throughout the piece. Light and darkness are used metaphorically, and each player’s journey through these elements forms a unique experience.
