Spatial Brain: MOST-EEG Analysis Shows “How” We Use Our Brains To Play 3-D Videogames

Philip Michael Zeman, BEng, PhD (2009)

Dr. Zeman recently completed an interdisciplinary Ph.D. involving Electrical Engineering (advanced signal processing), Neurobiology, and Cognitive Psychology.  The focus of his Ph.D. at the University of Victoria was the development of a new brain activity analysis tool called MOST-EEG.  Since then, he has created a company called Applied Brain and Vision Sciences Inc. , that employs new interdisciplinary technologies toward the development of pharmaceuticals and 'serious' videogames.  While his primary perspective on problems is that of an Engineer, his approach to problem solving incorporates knowledge and convention from multiple disciplines. In a television interview  (new window), Dr. Zeman describes applications of MOST-EEG and reporter Maggie Cox describes the procedure for constructing functional brain models.

Pharmaceuticals and videogames both have the potential to improve brain function

Pharmaceuticals do this directly, through molecular processes, while videogames act indirectly through the senses, by providing new experiences. Pharmaceuticals change the way neurons talk to each other by changing their chemical messages.  Behavioural experiences change the way neurons talk to each other by changing the strength of the connections between them.  This ability of the brain to adapt to new experiences is called neural plasticity. Behavioural experiences cause information to flow through the brain along neural pathways. The “use it or lose it” principle applies to these pathways. Pathways used to think or react (even in a video game) get strengthened while pathways that aren’t used are weakened.  In other words, pharmaceuticals and behavioural experiences both change the brain, but by different means.

In order to test the effects of behavioural experience and pharmaceuticals on brain function we employ electroencephalographic (EEG) data collection methods and analyze the EEG data using a new and unique algorithm called Multiple Origin Spatio-Temporal Modeling (MOST-EEG).  MOST-EEG is a versatile algorithm which can be applied to the analysis of EEG which has been collected in a wide variety of situations or circumstances. In contrast, standard EEG analysis methods require the circumstances in which the data are collected to be well-understood and therefore limited to previously studied experimental tasks.  It is a formidable challenge to use standard EEG analysis methods to fully understand novel situations or to reveal changes to pathways in the brain.  This is one of the primary advantages of MOST-EEG.

As indicated by the figure on the right side of this page and the video below, our primary goal is to discover "how" people use their brains to solve problems in complex behavioural tasks that mirror real world situations.  For our preliminary research, we used MOST-EEG to investigate brain activity while people were playing a videogame.   Once we know how people use their brains to play video games, we can use MOST-EEG to see how they use their brains for other activities like writing essays, singing, playing the piano, taking a quiz, or solving a Rubik's Cube puzzle.  We should also be able to see how the activities of our brains change when people consume a pharmaceutical such as a treatment for depression or Parkinson's disease.  Using MOST-EEG, we might find expected changes (i.e., that the activities of specific areas of the brain diminish or increase) or we might discover that unexpected areas of the brain change or completely different brain systems become significantly more or less active.  Changes in entire brain systems would indicate that people use their brains differently to do a task if they are receiving a pharmaceutical treatment .

Applied Brain and Vision Sciences Inc., is a brain technology development and consulting organization. It has been created to use MOST-EEG technology to investigate how changes in system-level brain function result from pharmaceutical treatments and behavioural therapies. Through Applied Brain and Vision Sciences, Zeman continues to provide data analysis and algorithm development services that go beyond frequency-band filters, and extend to single-sensor and multi-sensor statistical filtering and classification methods.  Data analysis is provided for biometric data (EMG, EEG, MEG, ESR) data relating to behavior or individual characteristics (latency to goal completion, hours of sleep per day), and standard input devices (cameras, eye-trackers, electronic thermometers).

Brain Activity While Playing Videogames and Right Hemisphere Involvement 

Illustrated on this page are preliminary results from a University of Victoria study led by Dr. Ron Skelton, with his Ph.D. student Sharon Lee, and his former student Dr. Philip Zeman in an  investigation of brain activity during videogame play.

In this study, participants were asked to find a particular target in a virtual room in one of two conditions.  In the first condition (guidance), participants had to go to a target that was visible on the floor in front of them.  In the second condition (cue), participants had to go to a target that was not visible, but was marked by a cue nearby.  Because there were 8 different cues in the room, participants had to learn which one marked the target location. In other words, in this condition, participants had to imagine where the target was and then go there.

The MOST-EEG 3D brain maps below show activation of multiple low-level (sensory) and high-level areas of the brain and the ways in which they are coordinated with each other.  Activity was much stronger and more co-ordinated on the right side of the brain while the participants were navigating in the cue condition compared to the guidance condition.  This suggests that brain activity is very different when the participant has to imagine the target location compared to when they can see the target from the starting position.

Figure 1: A brain map of neural activation according to MOST-EEG analysis, showing the right side of the brain. An animated version of this figure at the bottom of this pages provides a better view the brain activations

This brain map shows the predominantly right-hemisphere activation during cue-based navigation in the 3D videogame space.  Presented is the difference in activity between the cue condition and the guidance condition. The red volumes indicate the locations of brain activity that are stronger in the cue condition than in the guidance condition and the red lines show activities that are more co-ordinated.  Because this map illustrates the difference between conditions, it does not show activation and coordination that is common to both conditions.

These results strongly suggest that the act of finding our way in our world requires the right hemisphere of our brain. What is most important is that, while many studies have indicated that the right side of the brain is important for navigation our results provide a much more detailed picture of the multiple brain areas that are engaged.

Activation patterns like this could not have been seen without the MOST-EEG analysis method.  Such patterns are all the more valuable because they objectively reflect real brain activity and are not biased by pre-conceived ideas about what is supposed to be active. Furthermore, the process is a “turn-key” operation. Once EEG data are obtained, they can be fed into the MOST-EEG algorithm and 3D brain maps like these can be produced the same day.

Leaps in Knowledge Using MOST-EEG

"Upon reflecting, I've concluded that the fundamental contribution of MOST-EEG is that it allows us to make leaps in knowledge about brain function and how various stimuli, environmental conditions, and disease treatments affect us.  This has been evident in our own lab, where the results delivered by MOST-EEG have required us to expand our view of our spatial navigation paradigm by challenging our assumptions, and forcing us to pay much more attention to detail and individual participant variability than before."  -- Dr. Philip Michael Zeman 

Interpreting MOST-EEG Results to Understand Cognitive Function and Experimental Effect

Interpreting the results of MOST-EEG analysis is a two-step process.  First, the anatomical locations of activity are mapped onto a brain map of functional anatomy such as Brodmann’s. Then, drawing on a great deal of research in neuroscience, we interpret each activation in the context of the cognitive demands of the behavioural task. The next step is to examine the MOST-EEG measures of co-ordination between the activations and from these, create a cognitive model of the interaction of the functional areas.

Interpreting MOST-EEG Results To Understand the Effects of Pharmaceutical Therapies on Cognition

There are 2 phases required to apply the results of MOST-EEG to understanding drug effects.  The first phase follows the same two steps as above: first, map the activations onto a map of functional anatomy (e.g., Brodmann’s areas), then use the linkages between areas to build a model of the cognitive processes. These two steps are conducted using data from healthy participants who are not taking any pharmaceuticals in order to determine the “normal” profile.  The second phase is to test members of the target population (e.g., those with Parkinson’s) taking the target pharmaceutical (e.g. L-DOPA) while they are on the drug and while they are off the drug.  The “off-drug” profile indicates the way in which their brain has been affected by the disorder, while the “on-drug” profile indicates the degree to which the target drug returns their brain to normal function.

Figure 2. This block diagram (above) describes the process of using MOST-EEG results to evaluate the effect of a pharmaceutical on brain function.  For a the full-sized diagram and an explanation see , "Creating a Cognitive Model From MOST-EEG Results ".

Figure 3. This block diagram (above) describes how the MOST-EEG analysis processing can be incorporated into a drug development study.  For a full-sized diagram and explanation, see  "MOST-EEG and Pharmaceutical Research and Development”.

Sharon A. Lee, Ph.D. student

University of Victoria, Department of Psychology, specializing in Cognitive Neuroscience and Spatial Navigation