Applied Human-Machine and Media Interaction Modeling:

Brain And Vision: Video Games

[Home ][Expertise ][Brain Activity and Video Games ][Online Knowledge Base ][Contact Us]

Eye-Movement Can Tell Us About Strategy

Have you every wondered why your buddy does so much better than you when you play a video game?  Growing up, this was a question of contemplation for me.  Our family didn't have a Nintendo or PlayStation like so many other families did.  Instead of playing video games, I spent much of my youth running around outside, climbing trees, building forts, and exploring the neighbourhood.  As an adult, and a scientist, I've learned that there is more to playing video games than sitting on the couch with a buddy and a pile of chips.  There is actually some strategy involved.  And the good gamers probably employ good strategies without even knowing it.  It is fun to contemplate what those strategies might be.

Eye-movement While Playing PacMan

Below is an example of eye-movement associated with the video game PacMan.  After watching the video, a few characteristics of the eye-movement become clear.  First, the red dot (representing the rough center of vision) is often not on PacMan himself, nor is it on any of the 'enemy' creatures in the video game.  The dot actually travels around the screen.  It is often in front of PacMan or just below.  It also often jumps from one part of the game-playing area to another.  Second, the red dot does not move smoothly across the screen but jumps and wiggles continually. 

If we neglect any possible error of the eye-tracker, this behavior of the eye can be explained in terms of strategy and what visual features are required for the game-player to employ the strategy.  The behavior that the eye-looking position seems to often be in front of PacMan is interesting.  It might be an indication of where the game-player intends to move PacMan.  Remember that the game-player is controlling where PacMan moves on the screen and thus the behavior of "looking ahead" might actually relate more closely to the movement of the joystick than the current Position of PacMan.  This is more clearly illustrated in cases where there are clear jumps of the eye-looking position of areas of the screen to which PacMan is eventually moved by the game-player.  All of this suggests that the gammer is actually thinking slightly in the future of the current game play-- anticipating and planning strategies for collecting the dots and rewards in the game.

Another interesting characteristic of the video is that eye-looking directly at PacMan that does occur (indicated by the red dot) is very brief.  One might consider that at those moments in time where they red dot covers PacMan, the gamer is acting in the present.  For example, the gamers eye-movement direction and speed must match PacMan exactly in order for them to continually look at PacMan.

An interesting and important characteristic of vision and the brain is revealed by the brief movements of the eye to main points of interest on the screen such as PacMan himself, the 'enemy' creatures or prizes that appear in the game. That is, people do not have to continually be directly looking at this important information in order to incorporate it into their internal 'brain' representation of the environment.  They can in fact use multiple parts of their eyes to process various aspects of the visual seen presented in front of them.  The central region of vision, often referred to as foveal vision, maps to a part of the brain important for identifying objects that are presented on the screen.  The off-central of non-foveal part of vision maps to areas of the brain that are important for keeping track of where objects are moving on the screen.  Playing PacMan hence requires both aspects of vision to identify one object from another and to track where the various objects are moving.  I haven't stated here anything about what aspect of vision (central vs. non-central) is used for planning where to move PacMan and I leave this up for debate to interested persons.

The difference between someone who plays PacMan and achieves the high score and the person who can't last more than a few minutes might be reflected by where their eyes move.  It might be reflected by how often they look away from PacMan versus how often they look directly at PacMan.  Who knows?  I'm sure this is a question that science can answer.

Eye-Movement and 1st-Person Game Playing

In an experiment where we recorded eye-movement data while people played video a 1^st-persom video game, analysis revealed emphasis in specific areas of the screen.  An example ‘heat map’ showing the screen locations of eye-movement is given in the figure to the right.  It shows the screen location of most ‘looking’ is on or below the horizon.  In the particular game played, visual features and objects that were relevant to game play were located on or near the floor.  Hence, the eye-movement data support the inference that people were looking at these objects during game play.   The data show greatest amount of looking was toward the floor in the center of the screen.  Note, this ‘heat map’ was not synchronized with actual game-play where features on the screen moved dynamically over time.  Without a time sequence version of this ‘heat map’, we can not see which objects in the 1^st-person environment were looked at.

Figure 1. A ‘heatmap’ illustrating the eye-gaze location of a video game player on a VGA 800x600 computer screen.