Papers on acquisition, analysis and presentation of motion capture data for the study of
cognitive neuro-kinesiology and embodied semantics.

Cognitive Neuro-Kinesiology and Embodied Semantics

Embodied Semantics of Reaching and Grasping - Masson and Bub

Bub, D N, & Masson, M E J, 2010: Grasping beer mugs: on the dynamics of alignment effects induced by handled objects. J. Experim. Psychol. Hum. Percept. Perform., 36(2):341-358.

We examined automatic spatial alignment effects evoked by handled objects. Using color as the relevant cue carried by an irrelevant handled object aligned or misaligned with the response hand, responses to color were faster when the handle aligned with the response hand. Alignment effects were observed only when the task was to make a reach and grasp response. No alignment effects occurred if the response involved a left-right key press. Alignment effects emerged over time, becoming more apparent either when the color cue was delayed or when relatively long, rather than short, response times were analyzed. These results are consistent with neurophysiological evidence indicating that the cued goal state has a modulatory influence on sensorimotor representations, and that handled objects initially generate competition between neural populations coding for a left- or right-handed action that must be resolved before a particular hand is favored.

Masson, M E J, Bub, D N, & Newton-Taylor, M, 2008: Language-based access to gestural components of conceptual knowledge. Q. J. Experim. Psych., 61(6):869-882.

We report two experiments in which production of articulated hand gestures was used to reveal the nature of gestural knowledge evoked by sentences referring to manipulable objects. Two gesture types were examined: functional gestures (executed when using an object for its intended purpose) and volumetric gestures (used when picking up an object simply to move it). Participants read aloud a sentence that referred to an object but did not mention any form of manual interaction (e.g., Jane forgot the calculator) and were cued after a delay of 300 or 750 ms to produce the functional or volumetric gesture associated with the object, or a gesture that was unrelated to the object. At both cue delays, functional gestures were primed relative to unrelated gestures, but no significant priming was found for volumetric gestures. Our findings elucidate the types of motor representations that are directly linked to the meaning of words referring to manipulable objects in sentences.

Bub, D N, Masson, M E J, & Cree, G S, 2008: Evocation of functional and volumetric gestural knowledge by objects and words. Cognition, 106(1):27-58.

We distinguish between grasping gestures associated with using an object for its intended purpose (functional) and those used to pick up an object (volumetric) and we develop a novel experimental framework to show that both kinds of knowledge are automatically evoked by objects and by words denoting those objects. Cued gestures were carried out in the context of depicted objects or visual words. On incongruent trials, the cued gesture was not compatible with gestures typically associated with the contextual item. On congruent trials, the gesture was compatible with the item's functional or volumetric gesture. For both gesture types, response latency was longer for incongruent trials indicating that objects and words elicited both functional and volumetric manipulation knowledge. Additional evidence, however, clearly supports a distinction between these two kinds of gestural knowledge. Under certain task conditions, functional gestures can be evoked without the associated activation of volumetric gestures. We discuss the implication of these results for theories of action evoked by objects and words, and for interpretation of functional imaging results.

Masson, M E J, Bub, D N, & Warren, C M, 2008: Kicking calculators: Contribution of embodied representations to sentence comprehension. J. Memory & Language, 59(3):256-265.

Evocation of motor representations during sentence comprehension was examined by training subjects to make a hand action in response to a visual cue while listening to a sentence. Sentences referred to manipulable objects that were either related or unrelated to the cued action. Related actions pertained either to the function of the object or to its volumetric properties (e.g., shape). The results demonstrate priming of hand actions even though the sentences referred to non-manual interactions with manipulable objects. When sentences described an attentional interaction (looking at the calculator), only functional actions were primed. Sentences describing a non-manual physical interaction (kicking the calculator) primed volumetric as well as functional actions. We describe how seemingly irrelevant motor representations can play a role in constructing sentence meaning.

Bub, D N, & Masson, M E J, 2006: Gestural knowledge evoked by objects as part of conceptual representations. Aphasiology, 20(9):1112-1124.

Background: Theories of embodied knowledge argue that the representation and recruitment of motor processes may be important for deriving the meaning of many linguistic and perceptual elements.
Aims: We examined the conditions under which gestural knowledge associated with manipulable objects is evoked.
Methods & Procedures: A priming paradigm was used in which an object was presented in advance of a photograph of a hand gesture that participants were to mimic. On related trials, the target gesture was the same as the gesture typically used to interact with the object prime. On unrelated trials, the target gesture was not related to the object. In another set of experiments, a Stroop-like paradigm was used in which participants learned to produce manual responses to colour cues. After training, coloured photographs of manipulable objects were presented. The colour-cued gesture was either one typically used with the object or was unrelated to it.
Outcomes & Results: In the priming experiments, response latencies were shorter in the related condition, but only when participants also made an identification response to the object prime. In the Stroop experiments, interference effects indicated that gestures to colour were affected by gestural knowledge associated with the object.
Conclusions: These results indicate that conceptual representations of manipulable objects include specific forms of gestural knowledge that are automatically evoked when observers attend to an object.

Bub, D N, Masson, M E J, & Bukach, C M, 2003: Gesturing and naming: The use of functional knowledge in object identification. Psych. Sci., 14(5):467-472.

Studies using functional imaging show reliable activation of premotor cortex when observers view manipulable objects. This result has led to the view that knowledge of object function, particularly the actions associated with the typical use of objects, may play a causal role in object identification. To obtain relevant evidence regarding this causal role, we asked subjects to learn gesture-color associations and then attempt to identify objects presented in colors denoting functional gestures that were congruent or incongruent with the objects' use. A strong congruency effect was observed when subjects gestured the use of an object, but not when they named an object. We conclude that our procedure constitutes a sensitive measure of the recruitment and causal role of functional knowledge and that this recruitment is not present during object naming. Preliminary evidence, however, indicates that gestures evoked by the volumetric shape of an object do contribute to object naming.

Citing Papers

Fischer, M H, Prinz, J, & Lot, K, 2008: Grasp cueing shows obligatory attention to action goals. Q. J. Experim. Psych., 61(6):860-868.

To understand the grounding of cognitive mechanisms in perception and action, we used a simple detection task to determine how long it takes to predict an action goal from the perception of grasp postures and whether this prediction is under strategic control. Healthy observers detected visual probes over small or large objects after seeing either a precision grip or a power grip posture. Although the posture was uninformative it induced attention shifts to the grasp-congruent object within 350 ms. When the posture predicted target appearance over the grasp-incongruent object, observers' initial strategic allocation of attention was overruled by the congruency between grasp and object. These results might help to characterize the human mirror neuron system and reveal how joint attention tunes early perceptual processes toward action prediction.

Cited Papers

Pulvermüller, F, 2005: Brain mechanisms linking language and action. Nat. Rev. Neurosci., 6(7):576-582.

For a long time the cortical systems for language and actions were believed to be independent modules. However, as these systems are reciprocally connected with each other, information about language and actions might interact in distributed neuronal assemblies. A critical case is that of action words that are semantically related to different parts of the body (for example, 'lick', 'pick' and 'kick'): does the comprehension of these words specifically, rapidly and automatically activate the motor system in a somatotopic manner, and does their comprehension rely on activity in the action system?

Related Papers

Thaler, L, & Goodale, M A, 2010: Beyond distance and direction: The brain represents target locations non-metrically. J. Vision, 10(3):3.

In their day-to-day activities human beings are constantly generating behavior, such as pointing, grasping or verbal reports, on the basis of visible target locations. The question arises how the brain represents target locations. One possibility is that the brain represents them metrically, i.e. in terms of distance and direction. Another equally plausible possibility is that the brain represents locations non-metrically, using for example ordered geometry or topology. Here we report two experiments that were designed to test if the brain represents locations metrically or non-metrically. We measured accuracy and variability of visually guided reach-to-point movements (Experiment 1) and probe-stimulus adjustments (Experiment 2). The specific procedure of informing subjects about the relevant response on each trial enabled us to dissociate the use of non-metric target location from the use of metric distance and direction in head/eye-centered, hand-centered and externally defined (allocentric) coordinates. The behavioral data show that subjects' responses are least variable when they can direct their response at a visible target location, the only condition that permitted the use of non-metric information about target location in our experiments. Data from Experiments 1 and 2 correspond well quantitatively. Response variability in non-metric conditions cannot be predicted based on response variability in metric conditions. We conclude that the brain uses non-metric geometrical structure to represent locations.

Grasping and the Theory of Affordances

de Granville, C, & Fagg, A H, 2010: Learning grasp affordances through human demonstration. J. Autonomous Robots, submitted.

When presented with an object to be manipulated, a robot must identify the available forms of interaction. How might an agent acquire this mapping from object representation to action? In this paper, we describe an approach that learns a mapping from objects to grasps from human demonstration. For a given object, the teacher demonstrates a set of feasible grasps. We cluster these grasps in terms of the position and orientation of the hand relative to the object. Individual clusters in this pose space are represented using probability density functions, and thus correspond to variations around canonical grasp approaches. Multiple clusters are captured through a mixture distribution-based representation. Experimental results demonstrate the feasibility of extracting a compact set of canonical grasps from the human demonstration. Each of these canonical grasps can then be used to parameterize a reach controller that brings the robot hand into a specific spatial relationship with the object.

de Granville, C, Wang, D, Southerland, J, Platt Jr, R, & Fagg, A H, 2009: Grasping affordances: Learning to connect vision to hand action. ed: Sukhatme, G S, The Path to Autonomous Robots, 59-80, Springer.

When presented with an object to be manipulated, a robot must identify the available forms of interaction with the object. How might an agent automatically acquire this mapping from visual description of the object to manipulation action? In this chapter, we describe two components of an algorithm that enable an agent to learn a grasping-oriented representation by observing an object being manipulated by a human teacher. The first component uses the sequence of image/object pose tuples to acquire a model of the object's appearance as a function of the viewing angle.We identify visual features that are robustly observable over a range of similar viewing angles, but that are also discriminative of the set of viewing angles. Given a novel image, the algorithm can then estimate the angle from which the object is being viewed. The second component of the algorithm clusters the sequence of observed hand postures into the functionally distinct ways that the object may be grasped. Experimental results demonstrate the feasibility of extracting a compact set of canonical grasps from this experience. Each of these canonical grasps can then be used to parameterize a reach controller that brings the robot hand into a specific spatial relationship with the object.

Detry, R, Baseski, E, Popovic, M, Touati, Y, Krüger, N, Kroemer, O, Peters, J, & Piater, J, 2009: Learning object-specific grasp affordance densities. Proc. 8th IEEE Int'l Conf. Development & Learning (ICDL'09).

This paper addresses the issue of learning and representing object grasp affordances, i.e. object-gripper relative configurations that lead to successful grasps. The purpose of grasp affordances is to organize and store the whole knowledge that an agent has about the grasping of an object, in order to facilitate reasoning on grasping solutions and their achievability. The affordance representation consists in a continuous probability density function defined on the 6D gripper pose space - 3D position and orientation - within an object-relative reference frame. Grasp affordances are initially learned from various sources, e.g. from imitation or from visual cues, leading to grasp hypothesis densities. Grasp densities are attached to a learned 3D visual object model, and pose estimation of the visual model allows a robotic agent to execute samples from a grasp hypothesis density under various object poses. Grasp outcomes are used to learn grasp empirical densities, i.e. grasps that have been confirmed through experience. We show the result of learning grasp hypothesis densities from both imitation and visual cues, and present grasp empirical densities learned from physical experience by a robot.

Kraft, D, Detry, R, Pugeault, N, Baseski, E, Piater, J, & Krüger, N, 2009: Learning objects and grasp affordances through autonomous exploration. Proc. 7th Int'l Conf. Computer Vision Systems (ICVS 2009).

We describe a system for autonomous learning of visual object representations and their grasp affordances on a robot-vision system. It segments objects by grasping and moving 3D scene features, and creates probabilistic visual representations for object detection, recognition and pose estimation, which are then augmented by continuous characterizations of grasp affordances generated through biased, random exploration. Thus, based on a careful balance of generic prior knowledge encoded in (1) the embodiment of the system, (2) a vision system extracting structurally rich information from stereo image sequences as well as (3) a number of built-in behavioral modules on the one hand, and autonomous exploration on the other hand, the system is able to generate object and grasping knowledge through interaction with its environment.

Castellini, C, Orabona, F, Metta, G, & Sandini, G: Internal models of reaching and grasping. Advanced Robotics, 21(13):1545-1564.

One of the most distinguishing features of cognitive systems is the ability to predict the future course of actions and the results of ongoing behaviors, and in general to plan actions well in advance. Neuroscience has started examining the neural basis of these skills with behavioral or animal studies and it is now relatively well understood that the brain builds models of the physical world through learning. These models are sometimes called 'internal models', meaning that they are the internal rehearsal (or simulation) of the world enacted by the brain. In this paper we investigate the possibility of building internal models of human behaviors with a learning machine that has access to information in principle similar to that used by the brain when learning similar tasks. In particular, we concentrate on models of reaching and grasping, and we report on an experiment in which biometric data collected from human users during grasping was used to train a support vector machine. We then assess to what degree the models built by the machine are faithful representations of the actual human behaviors. The results indicate that the machine is able to predict reasonably well human reaching and grasping, and that prior knowledge of the object to be grasped improves the performance of the machine, while keeping the same computational cost.

de Granville, C, Southerland, J, & Fagg, A H, 2006: Learning grasp affordances through human demonstration. Proc. 5th IEEE Int'l Conf. Development & Learning (ICDL'06).

When presented with an object to be manipulated, a robot must consider the set of actions available for interaction. How might an agent acquire this mapping from object representation to action? In this paper, we describe an approach that learns a mapping from objects to grasps from human demonstration. For a given object, the teacher demonstrates a set of feasible grasps. We cluster these grasps in terms of the corresponding orientation of the hand. Individual clusters of these three-dimensional orientations are represented using probability density functions that take on Gaussian-like shapes, and thus correspond to variations around canonical approach orientations. Multiple clusters are captured through a mixture distribution-based representation. Experimental results demonstrate the feasibility of extracting a compact set of canonical grasps from the human demonstration. Each of these canonical grasps can then be used to parameterize a reach controller that brings the robot hand into a specific (and functional) spatial relationship with the object.

Effect of Affordance on Grasping Movement

Girardi, G, Lindemann, O, & Bekkering, H, 2010: Context effects on the processing of action-relevant object features. J. Experim. Psych. Hum. Percep. Perform., 36(2):330-340.

In 4 experiments, we investigated the effects of object affordance in reach-to-grasp actions. Participants indicated whether a depicted small or large object was natural or manmade by means of different object -grasping responses (i.e., with a power or a precision grip). We observed that the size of the depicted object affected the grasping kinematics (grip aperture) and the reach-onset times of compatible and incompatible actions. Additional experiments showed that the effect of perceived object size on motor response was modulated by contextual action information and the observation of others' actions with the object. Thus, beyond the observation of object affordance effects in natural grasping actions, this study suggests that the coupling between object perception and action is not static and obligatory. Behavioral effects of action-relevant object features seem rather to depend on contextual action information.

Neurophysiology of Reaching and Grasping

Chapman, C S, Gallivan, J P, Wood, D K, Milne, J L, Culham, J C, & Goodale, M A, 2010: Short-term motor plasticity revealed in a visuomotor decision-making task. Behav. Brain Res., 214(1):130-134.

Selecting and executing an action toward only one object in our complex environments presents the visuomotor system with a significant challenge. To overcome this problem, the motor system is thought to simultaneously encode multiple motor plans, which then compete for selection. The decision between motor plans is influenced both by incoming sensory information and previous experience-which itself is comprised of long-term (e.g. weeks, months) and recent (seconds, minutes, hours) information. In this study, we were interested in how the recent trial-to-trial visuomotor experience would be factored into upcoming movement decisions made between competing potential targets. To this aim, we used a unique rapid reaching task to investigate how reach trajectories would be spatially influenced by previous decisions. Our task required subjects to initiate speeded reaches toward multiple potential targets before one was cued in-flight. A novel statistical analysis of the reach trajectories revealed that in cases of target uncertainty, subjects initiated a spatially averaged trajectory toward the midpoint of potential target locations before correcting to the selected target location. Interestingly, when the same target location was consecutively cued, reaches were biased toward that location on the next trial and this effect accumulated across trials. Beyond providing supporting evidence that potential reach locations are encoded and compete in parallel, our results strongly suggest that this motor competition is biased by recent trial history.

Chapman, C S, Gallivan, J P, Wood, D K, Milne, J L, Culham, J C, & Goodale, M A, 2010: Short-term motor plasticity - supplementary material. <www.sciencedirect.com/.../f.doc>.

Chapman, C S, Gallivan, J P, Wood, D K, Milne, J L, Culham, J C, & Goodale, M A, 2010: Reaching for the unknown: Mutliple-target encoding and real-time decision-making in a rapid reach task. Cognition, 116(2):168-176.

Decision-making is central to human cognition. Fundamental to every decision is the ability to internally represent the available choices and their relative costs and benefits. The most basic and frequent decisions we make occur as our motor system chooses and executes only those actions that achieve our current goals. Although these interactions with the environment may appear effortless, this belies what must be incredibly sophisticated visuomotor decision-making processes. In order to measure how visuomotor decisions unfold in real-time, we used a unique reaching paradigm that forced participants to initiate rapid hand movements toward multiple potential targets, with only one being cued after reach onset. We show across three experiments that, in cases of target uncertainty, trajectories are spatially sensitive to the probabilistic distribution of targets within the display. Specifically, when presented with two or three target displays, subjects initiate their reaches toward an intermediary or 'averaged' location before correcting their trajectory in-flight to the cued target location. A control experiment suggests that our effect depends on the targets acting as potential reach locations and not as distractors. This study is the first to show that the 'averaging' of target-directed reaching movements depends not only on the spatial position of the targets in the display but also the probability of acting at each target location.

Chapman, C S, Gallivan, J P, Wood, D K, Milne, J L, Culham, J C, & Goodale, M A, 2010: Reaching for the unknown - supplementary material. <www.sciencedirect.com/.../mmc1.doc>.

Castiello, U, & Begliomini, C, 2008: The cortical control of visually guided grasping. Neuroscientist, 14(2):157-170.

People have always been fascinated by the exquisite precision and flexibility of the human hand. When hand meets object, we confront the overlapping worlds of sensorimotor and cognitive functions. The complex apparatus of the human hand is used to reach for objects, grasp and lift them, manipulate them, and use them to act on other objects. This review examines what is known about the control of the hand by the cerebral cortex. It compares and summarizes results from behavioral neuroscience, electrophysiology, and neuroimaging to provide a detailed description of the neural circuits that facilitate the formation of grip patterns in human and nonhuman primates.

Rand, M K, Shimansky, Y P, Hossain, A B M I, & Stelmach, G E, 2008: Quantitative model of transport-aperture coordination during reach-to-grasp movements. Exp. Brain Res., 188(2):263-274.

It has been found in our previous studies that the initiation of aperture closure during reach-to-grasp movements occurs when the hand distance to target crosses a threshold that is a function of peak aperture amplitude, hand velocity, and hand acceleration. Thus, a stable relationship between those four movement parameters is observed at the moment of aperture closure initiation. Based on the concept of optimal control of movements (Naslin 1969) and its application for reach-to-grasp movement regulation (Hoff and Arbib 1993), it was hypothesized that the mathematical equation expressing that relationship can be generalized to describe coordination between hand transport and finger aperture during the entire reach-to -grasp movement by adding aperture velocity and acceleration to the above four movement parameters. The present study examines whether this hypothesis is supported by the data obtained in experiments in which young adults performed reach-to-grasp movements in eight combinations of two reach-amplitude conditions and four movement-speed conditions. It was found that linear approximation of the mathematical model described the relationship among the six movement parameters for the entire aperture-closure phase with very high precision for each condition, thus supporting the hypothesis for that phase. Testing whether one mathematical model could approximate the data across all the experimental conditions revealed that it was possible to achieve the same high level of data-fitting precision only by including in the model two additional, condition-encoding parameters and using a nonlinear, artificial neural network-based approximator with two hidden layers comprising three and two neurons, respectively. This result indicates that transport-aperture coordination, as a specific relationship between the parameters of hand transport and finger aperture, significantly depends on the condition-encoding variables. The data from the aperture -opening phase also fit a linear model, whose coefficients were substantially different from those identified for the aperture-closure phase. This result supports the above hypothesis for the aperture-opening phase, and consequently, for the entire reach-to-grasp movement. However, the fitting precision was considerably lower than that for the aperture-closure phase, indicating significant trial-to-trial variability of transport -aperture coordination during the aperture-opening phase. Implications for understanding the neural mechanisms employed by the CNS for controlling reach-to-grasp movements and utilization of the mathematical model of transport-aperture coordination for data analysis are discussed.

Gardner, E P, Babu, K S, Ghosh, S, Sherwood, A, & Chen, J, 2007: Neurophysiology of prehension. III. Representation of object features in posterior parietal cortex of the macaque monkey. J. Neurophysiol., 98(6):3708-3730.

Neurons in posterior parietal cortex (PPC) may serve both proprioceptive and exteroceptive functions during prehension, signaling hand actions and object properties. To assess these roles, we used digital video recordings to analyze responses of 83 hand-manipulation neurons in area 5 as monkeys grasped and lifted objects that differed in shape (round and rectangular), size (large and small spheres), and location (identical rectangular blocks placed lateral and medial to the shoulder). The task contained seven stages -approach, contact, grasp, lift, hold, lower, relax-plus a pretrial interval. The four test objects evoked similar spike trains and mean rate profiles that rose significantly above baseline from approach through lift, with peak activity at contact. Although representation by the spike train of specific hand actions was stronger than distinctions between grasped objects, 34% of these neurons showed statistically significant effects of object properties or hand postures on firing rates. Somatosensory input from the hand played an important role as firing rates diverged most prominently on contact as grasp was secured. The small sphere-grasped with the most flexed hand posture-evoked the highest firing rates in 43% of the population. Twenty-one percent distinguished spheres that differed in size and weight, and 14% discriminated spheres from rectangular blocks. Location in the workspace modulated response amplitude as objects placed across the midline evoked higher firing rates than positions lateral to the shoulder. We conclude that area 5 neurons, like those in area AIP, integrate object features, hand actions, and grasp postures during prehension.

Gardner, E P, Ro, J Y, Babu, K S, & Ghosh S, 2007: Neurophysiology of prehension. II. Response diversity in primary somatosensory (S-I) and motor (M-I) cortices. J. Neurophysiol., 97(2):1656-1670.

Prehension responses of 76 neurons in primary somatosensory (S-I) and motor (M-I) cortices were analyzed in three macaques during performance of a grasp and lift task. Digital video recordings of hand kinematics synchronized to neuronal spike trains were compared with responses in posterior parietal areas 5 and AIP /7b (PPC) of the same monkeys during seven task stages: 1) approach, 2) contact, 3) grasp, 4) lift, 5) hold, 6) lower, and 7) relax. S-I and M-I firing patterns signaled particular hand actions, rather than overall task goals. S-I responses were more diverse than those in PPC, occurred later in time, and focused primarily on grasping. Sixty-three percent of S-I neurons fired at peak rates during contact and/or grasping. Lift, hold, and lowering excited fewer S-I cells. Only 8% of S-I cells fired at peak rates before contact, compared with 27% in PPC. M-I responses were also diverse, forming functional groups for hand preshaping, object acquisition, and grip force application. M-I activity began 500 ms before contact, coinciding with the earliest activity in PPC. Activation of specific muscle groups in the hand was paralleled by matching patterns of somatosensory feedback from S-I needed for efficient performance. These findings support hypotheses that predictive and planning components of prehension are represented in PPC and premotor cortex, whereas performance and feedback circuits dominate activity in M-I and S-I. Somatosensory feedback from the hand to S-I enables real-time adjustments of grasping by connections to M-I and updates future prehension plans through projections to PPC.

Gardner, E P, Babu, K S, Reitzen, S D, Ghosh, S, Brown, A S, Chen, J, Hall, A L, Herzlinger, M D, Kohlenstein, J B, & Ro, J Y, 2007: Neurophysiology of prehension. I. Posterior parietal cortex and object-oriented hand behaviors. J. Neurophysiol., 97(1):387-406.

Hand manipulation neurons in areas 5 and 7b/anterior intraparietal area (AIP) of posterior parietal cortex were analyzed in three macaque monkeys during a trained prehension task. Digital video recordings of hand kinematics synchronized to neuronal spike trains were used to correlate firing rates of 128 neurons with hand actions as the animals grasped and lifted rectangular and round objects. We distinguished seven task stages: approach, contact, grasp, lift, hold, lower, and relax. Posterior parietal cortex (PPC) firing rates were highest during object acquisition; 88% of task-related area 5 neurons and 77% in AIP/7b fired maximally during stages 1, 2, or 3. Firing rates rose 200-500 ms before contact, peaked at contact, and declined after grasp was secured. 83% of area 5 neurons and 72% in AIP/7b showed significant increases in mean rates during approach as the fingers were preshaped for grasp. Somatosensory signals at contact provided feedback concerning the accuracy of reach and helped guide the hand to grasp sites. In error trials, tactile information was used to abort grasp, or to initiate corrective actions to achieve task goals. Firing rates declined as lift began. 41% of area 5 neurons and 38% in AIP/7b were inhibited during holding, and returned to baseline when grasp was relaxed. Anatomical connections suggest that area 5 provides somesthetic information to circuits linking AIP/7b to frontal motor areas involved in grasping. Area 5 may also participate in sensorimotor transformations coordinating reach and grasp behaviors and provide on-line feedback needed for goal-directed hand movements.

Fagg, A H, 1996: A computational model of the cortical mechanisms involved in primate grasping, Ph.D. Thesis, University of Southern California.

The act of reaching out, grasping, and manipulating an object involves the integration of information from a variety of sources-from vision of the object of interest, to task requirements, to tactile and proprioceptive information as the grasp is executed. In this thesis, we investigate the cortical mechanisms involved in 1) the translation of a visual description of an object and a representation of the task into an appropriate hand configuration, and 2) the unfolding of this description in time in order to execute the preshape, enclose, grasp, and ungrasp phases of movement. On the basis of behavioral, cell recording, and anatomical data from human and monkey, a computational model of the grasping process is proposed. This model focuses on the roles of the intra-parietal areas (AIP, PIP, and VIP), inferior premotor cortex (F4 and F5), pre-SMA (F6), frontal cortex (area 46), inferiotemporal cortex (IT), and the secondary somatosensory cortex (SII). In the model, AIP serves a dual role of first computing a set of affordances that are appropriate for the object being attended, and then maintaining an active memory of the single affordance as the corresponding grasp is executed. F5 integrates visual, task and memory information in order to select one of the several possible grasps. This brain region then drives the high-level execution of the grasp and monitors its progression. Based upon the hypothesized computational roles of AIP and F5, the model makes several key predictions about the encoding of grasp and object information at both the single unit and population levels. In addition, we present results of a PET (positron emission tomography) study that 1) compares brain activity during the performance of the precision and power grasps, and 2) examines the brain regions responsible for processing an abstract instruction stimulus. Through a technique referred to as Synthetic PET Imaging, we are able to compare the human PET results to the global behavior of the model. We show that this technique can also be used to further constrain the model structure.

Hoff, B, & Arbib, M A, 1993: Models of trajectory formation and temporal interaction of reach and grasp. J. Mot. Behav., 25(3):175-192.

Our goal was to create a principled account of a body of behavioral kinematic data on reaching and grasping. We show how to transform an optimality principle for overall hand transport into a feedback control law and then incorporate look-ahead modules in the controller to compensate for delays in sensory feedback. This model describes the kinematics of hand transport under a variety of circumstances, including target perturbations. We then develop a model for the temporal coordination of reach and grasp. We provide an optimization principle for hand preshaping that trades off the costs of maintaining the hand in an open position and the cost of accelerating the change in grip size. This yields a control system for preshaping. We then show that a model that uses only expected duration for coordination, rather than kinematic or dynamic variables, can describe the kinematics of interaction of hand transport and preshape under a variety of circumstances, including perturbations of object position and object size.

Soechting, J F, & Flanders, M, 1989: Sensorimotor representations for pointing to targets in three dimensional space. J. Neurophysiol., 62(2):582-594.

1. The accuracy with which subjects pointed to targets in extrapersonal space was assessed under a variety of experimental conditions.
2. When subjects pointed in the dark to remembered target locations, they made substantial errors. Errors in distance, measured from the shoulder to the target, were sometimes as much as 15 cm. Errors in direction, also measured from the shoulder, were smaller.
3. An analysis of the information transmitted by the location of the subject's finger about the location of the target showed that the information about the target's distance was consistently lower than the information about its direction.
4. The errors in distance persisted when subjects had their arm in view and pointed in the light to remembered target locations.
5. The errors were much smaller when subjects used a pointer to point to the target or when they were asked to reproduce the position of their finger after it had been passively moved to the target.
6. From these findings we conclude that subjects have a reasonably accurate visual representation of target location and are able to effectively use kinesthetically derived information about target location. We therefore suggest that errors in pointing result from errors in the sensorimotor transformation from the visual representation of the target location to the kinematic representation of the arm movement.

Soechting, J F, & Flanders, M, 1989: Errors in pointing are due to approximations in sensorimotor transformations. J. Neurophysiol., 62(2):595-608.

1. We define an extrinsic frame of reference to represent the location of a point in extrapersonal space relative to a human subject's shoulder, and we define an intrinsic frame of reference to represent the orientation of the arm and forearm.
2. We examined the relations between coordinates in the extrinsic and intrinsic frames of reference under two experimental conditions: when subjects made inaccurate movements by pointing to virtual targets in the dark and when they made accurate movements by pointing to actual targets in the light.
3. When subjects made inaccurate movements, there was a close-to-linear relationship between the orientation angles of the arm (intrinsic coordinates) at its final position and the extrinsic coordinates of the target. When they made accurate movements, these relationships were more nonlinear.
4. Specifically, arm and forearm elevations depended principally on target distance and elevation, whereas the two yaw angles depended mainly on the target's azimuth.
5. We propose that errors in pointing occur because subjects implement a linear approximation to the transformation from extrinsic to intrinsic coordinates and that this transformation is one step in the process of transforming a visually derived representation of target location into an appropriate pattern of muscle activity.

Soechting, J F, Lacquaniti, F, & Terzuolo, C, 1986: Coordination of arm movements in three-dimensional space: Sensorimotor mapping during drawing movements. Neuroscience, 17(2):295-311.

In this paper data are presented concerning the motion of limb segments during drawing movements executed in different planes in free space. The technique used allows the determination of the wrist and elbow positions in space as well as the measurement of the elbow angle of extension. Other kinematic variables are determined trigonometrically. Elbow and shoulder torque is also calculated.
For circles and ellipses, it was found that the motion at the wrist is sinusoidal in two orthogonal directions in the plane of motion. Angular motion, when described by a set of angles previously identified psychophysically as constituting an appropriate coordinate system, is also sinusoidal. Although the number of degrees of freedom of the arm affords many possible ways of performing the task, there is a fixed phase relation between the angles of elevation of the upper arm and forearm for naturally executed movements in all planes of space. Also, the phase of the yaw angles of the upper arm and forearm relative to the angles of elevation are related to the plane of motion and to the slant of ellipses in a fixed manner. There is a simple mapping between angular motion and intended wrist trajectory. Because this mapping is not valid for all planes of space, the actual trajectory can deviate from the intended one. However, the subject has no cognizance of the distortion.
The calculated torque deviates substantially from sinusoidal and does change significantly when the same movement is executed in different planes. Results of simulations and mathematical analysis indicate that the fixed phase relationship between angles of elevation leads to a minimal distortion from sinusoidal motion at the wrist in an average sense and that the characteristic distortions observed in the sagittal plane result inevitably from this constraint on the phase relations. The results support the assumption that the topology of the sensorimotor map used for the production of the movement and for its perception is the same. The problem of invariant relationships between kinematic parameters is discussed and the suggestion is made that they represent a general constraint, leading through learning and practice to an optimal solution in an average sense.

Lacquaniti, F, Soechting, J F, & Terzuolo, C, 1986: Path constraints on point-to-point arm movements in three-dimensional space. Neuroscience, 17(2):313-324.

In this paper data are presented concerning the kinematic and dynamic characteristics of point-to-point arm movements which are inwardly or outwardly directed in three-dimensional space. Elbow and wrist position as well as elbow angle of extension were measured. From these data, other angles were computed trigonometrically and elbow and shoulder torques were calculated. Some of the angles describing arm and forearm motion were found to be linearly related for any given movement. Changes in shoulder and elbow torque were found to be similar to those described for movements restricted to one degree of freedom. Shoulder and elbow motions were not affected when it was required that the orientation of the hand in space remain constant.
These observations were taken to indicate that shoulder and elbow motions are tightly coupled for movements in three-dimensional space and that wrist motion has no influence on this coupling. Linear relations between angles express such coupling. They are taken to result from functional constraints and may facilitate the mapping between extrinsic and intrinsic coordinate systems. Some of the observations pertaining to the torque lead to the hypothesis of a further constraint limiting the number of possible trajectories in a point-to-point movement.

Soechting, J. F., & Terzuolo, C, 1986: An algorithm for the generation of curvilinear wrist motion in an arbitrary plane in three-dimensional space. Neuroscience, 19(4):1393-1405.

The elements of an algorithm are presented which predicts for some simple forms (circles and ellipses) the kinematic and figural aspects of the trajectories of the human wrist when these are drawn in any arbitrary plane of free, three-dimensional space. The algorithm is based on theoretical considerations and experimental data and specifies in a unique way the angular motion at the shoulder and elbow joints by utilizing a coordinate transformation, which is only approximate, between the chosen extrinsic (trajectory) and intrinsic (joint angles) parameters. A way to extend the use of this algorithm to generate any arbitrary complex movement in all possible planes of space is also suggested.

Soechting, J F, & Terzuolo, C, 1987: Organization of arm movements. Motion is segmented. Neuroscience, 23(1):39-51.

A kinematic analysis of human arm trajectories which underlie the production of learned, continuous movements (such as drawing of 'figure 8s' and stars) in free space is presented. The objective of this investigation was to see if a set of rules, which had been identified previously and which are appropriate for generating circular or elliptical motion of the wrist in an arbitrary plane, also hold true for arbitrary, learned trajectories provided one additional assumption is made: that apparently continuous complex movements are composed of unit segments.
The results presented in this paper are consistent with this hypothesis. Furthermore, as predicted by the hypothesis, the wrist trajectory deviates little from planar motion in each segment while the plane of motion can change abruptly from one segment to the next.

Soechting, J F, & Terzuolo, C, 1987: Organization of arm movements in three-dimensional space. Wrist motion is piecewise planar. Neuroscience, 23(1):53-61.

It is shown that human subjects are incapable of producing with the arm, in free space, planned or extemporaneously drawn trajectories in which the plane of wrist motion changes smoothly or continuously. The three-dimensional nature of these movements results from the fact that the plane of motion changes abruptly from one segment of the trajectory to the next, being confined to one plane during each segment ( i.e. piecewise planar).

Hestenes, D, 1994: Invariant body kinematics: I. Saccadic and compensatory eye movements. Neural Networks, 7(1):65-77.

A new invariant formulation of 3D eye-head kinematics improves on the computational advantages of quaternions. This includes a new formulation of Listing's Law parametrized by gaze direction leading to an additive, rather than a multiplicative, saccadic error correction with a gaze vector difference control variable. A completely general formulation of compensatory kinematics characterizes arbitrary rotational and translational motions, vergence computation, and smooth pursuit. The result is an invariant, quantitative formulation of the computational tasks that must be performed by the oculomotor system for accurate 3D gaze control. Some implications for neural network modeling are discussed.

Hestenes, D, 1994: Invariant body kinematics: II. Reaching and neurogeometry. Neural Networks, 7(1):79-88.

Invariant methods for formulating and analyzing the mechanics of the skeleto-muscular system with geometric algebra are further developed and applied to reaching kinematics. This work is set in the context of a neurogeometry research program to develop a coherent mathematical theory of neural sensory-motor control systems.

Fernandez, J M, & Farell, B, 2009: Is perceptual space inherently non-Euclidean? J. Math. Psych., 53(2):86-91.

It is often assumed that the space we perceive is Euclidean, although this idea has been challenged by many authors. Here we show that if spatial cues are combined as described by Maximum Likelihood Estimation, Bayesian, or equivalent models, as appears to be the case, then Euclidean geometry cannot describe our perceptual experience. Rather, our perceptual spatial structure would be better described as belonging to an arbitrarily curved Riemannian space.

Kinematics of Reaching and Grasping

Gielen, C C A M, Vrijenhoek, E J, Flash, T, & Neggers, S F W, 1997: Arm position constraints during pointing and reaching in 3-D space. J. Neurophysiol., 78(2):660-673.

Arm movements in 3-D space were studied to investigate the reduction in the number of rotational degrees of freedom in the shoulder and elbow during pointing movements with the fully extended arm and during pointing movements to targets in various directions and at various distances relative to the shoulder, requiring flexion/extension in the elbow. The postures of both the upper arm and forearm can be described by rotation vectors, which represent these postures as a rotation from a reference position to the current position. The rotation vectors describing the posture of the upper arm and forearm were found to lie in a 2 -D (curved) surface both for pointing with the fully extended arm and for pointing with elbow flexion. This result generalizes on previous results on the reduction of the number of degrees of freedom from three to two in the shoulder for the fully extended arm to a similar reduction in the number of degrees of freedom for the upper arm and forearm for normal arm movements involving also elbow flexion and extension. The orientation of the 2-D surface fitted to the rotation vectors describing the position of the upper arm and forearm was the same for pointing with the extended arm and for movements with flexion/extension of the elbow. The scatter in torsion of the rotation vectors describing the position of the upper arm and forearm relative to the 2-D surface was typically 3-4°, which is small considering the range of ~180 and 360° for torsional rotations of the upper arm and the forearm, respectively. Donders' law states that arm posture for pointing to a target does not depend on previous positions of the arm. The results of our experiments demonstrate that the upper arm violates Donders' law. However, the variations in torsion of the upper arm are small, typically a few degrees. These deviations from Donders' law have been overlooked in previous studies, presumably because the variations are relatively small. These variations may explain the larger scatter of the rotation vectors for arm movements (3-4°) than reported for the eye (1°). Unlike for saccadic eye movements, joint rotations in the shoulder during aiming movements were not all single-axis rotations. On the contrary, the direction of the angular velocity vector varied during the movement in a consistent and reproducible way, depending on amplitude, direction, and starting position of the movement. These results reveal several differences between arm movements during pointing and saccadic eye movements. The implications for our understanding of the coordination of eye and arm movements and for the planning of 3 -D arm movements are discussed.

Soechting, J F, Buneo, C A, Herrmann, U, Flanders, M, 1995: Moving effortlessly in three dimensions: Does Donders' Law apply to arm movement? J. Neurosci., 15(9):6271-6280.

Donders' law, as applied to the arm, predicts that to every location of the hand in space there corresponds a unique posture of the arm as defined by shoulder and elbow angles. This prediction was tested experimentally by asking human subjects to make pointing movements to a select number of target locations starting from a wide range of initial hand locations. The posture of the arm was measured at the start and end of every movement by means of video cameras. It was found that, in general, the posture of the arm at a given hand location does depend on the starting location of the movement and that, consequently, Donders' law is violated in this experimental condition. Kinematic and kinetic factors that could account for the variations in arm posture were investigated. It proved impossible to predict the final posture of the arm purely from kinematics, based on the initial posture of the arm. One hypothesis was successful in predicting final arm postures, namely that the final posture minimizes the amount of work that must be done to transport the arm from the starting location.

Tweed, D, & Vilis, T, 1987: Implications of rotational kinematics for the oculomotor system in three dimensions. J. Neurophysiol., 58(4):832-849.

1. This paper develops three-dimensional models for the vestibuloocular reflex (VOR) and the internal feedback loop of the saccadic system. The models differ qualitatively from previous, one-dimensional versions, because the commutative algebra used in previous models does not apply to the three -dimensional rotations of the eye. 2. The hypothesis that eye position signals are generated by an eye velocity integrator in the indirect path of the VOR must be rejected because in three dimensions the integral of angular velocity does not specify angular position. Computer simulations using eye velocity integrators show large, cumulative gaze errors and post-VOR drift. We describe a simple velocity to position transformation that works in three dimensions. 3. In the feedback control of saccades, eye position error is not the vector difference between actual and desired eye positions. Subtractive feedback models must continuously adjust the axis of rotation throughout a saccade, and they generate meandering, dysmetric gaze saccades. We describe a multiplicative feedback system that solves these problems and generates fixed-axis saccades that accord with Listing's law. 4. We show that Listing's law requires that most saccades have their axes out of Listing's plane. A corollary is that if three pools of short-lead burst neurons code the eye velocity command during saccades, the three pools are not yoked, but function independently during visually triggered saccades. 5. In our three-dimensional models, we represent eye position using four-component rotational operators called quaternions. This is not the only algebraic system for describing rotations, but it is the one that best fits the needs of the oculomotor system, and it yields much simpler models than do rotation matrix or other representations. 6. Quaternion models predict that eye position is represented on four channels in the oculomotor system: three for the vector components of eye position and one inversely related to gaze eccentricity and torsion. 7. Many testable predictions made by quaternion models also turn up in models based on other mathematics. These predictions are therefore more fundamental than the specific models that generate them. Among these predictions are 1) to compute eye position in the indirect path of the VOR, eye or head velocity signals are multiplied by eye position feedback and then integrated; consequently 2) eye position signals and eye or head velocity signals converge on vestibular neurons, and their interaction is multiplicative; 3) tonic neurons carrying different components of the eye position signal are interdependent, so malfunction of one component will affect the others; and 4) in the feedback control of saccades, the error signal is the desired position of the eye divided by the actual position.

Ambike, S, & Schmiedeler, J P, 2008: Time-invariant strategies in coordination of human reaching. ed: Lenarcic, J, & Wenger, P, Advances in Robot Kinematics: Analysis and Design, 205-214, Springer.

This paper addresses validation of a curvature-theory-based time-invariant inverse kinematic model and a related tracking algorithm for human motor control of reaching motions. Human subjects made unconstrained reaching motions in the horizontal plane to fixed targets at three self-selected speeds. Consistent shoulder/elbow joint speed ratios for motions to the same target across speeds were observed, indicating a time-invariant planning strategy. The inverse kinematic model's technique of relating joint motions with a Taylor series expansion is in concert with the leading joint hypothesis. With this approach the tracking algorithm successfully replicated the experimental wrist trajectories, and also predicted the previously observed elbow-led motions for reaching in the ipsilateral hemifield. The elbow leads the arm motion in this hemifield because the shoulder approaches a dwell. A computationally frugal strategy of intermittent path correction based on two error parameters is proposed.

Marotta, J J, Medendorp, W P, & Crawford, J D, 2003: Kinematic rules for upper and lower arm contributions to grasp orientation. J. Neurophysiol., 90(6):3816-3827.

The purpose of the current study was to investigate the contribution of upper and lower arm torsion to grasp orientation during a reaching and grasping movement. In particular, we examined how the visuomotor system deals with the conflicting demands of coordinating upper and lower arm torsion and maintaining Donders' Law of the upper arm (a behavioral restriction of the axes of arm rotation to a two-dimensional "surface"). In experiment 1, subjects reached out and grasped a target block that was presented in one of 19 orientations (5° clockwise increments from horizontal to vertical) at one position in a vertical presentation board. In experiment 2, target blocks were presented in one of three orientations (horizontal, three-quarter, and vertical) at nine different positions in the presentation board. If reach and grasp commands control the proximal and distal arms separately, then one would only expect the lower arm to contribute to grasp orientations and that Donders' Law would hold for the upper arm-independent of grasp orientations. Instead, as the required grasp orientation increased from horizontal to vertical, there was a significant clockwise torsional rotation in the upper arm, which accounted for 9% of the final vertical grasp orientation, and the lower arm, which accounted for 42%. A linear relationship existed between the torsional rotations of the upper and lower arm, which indicates that the components of the arm rotate in coordination with one another. The location-dependent aspects of upper and lower arm torsion remained invariant, however, yielding consistently shaped Donders' "surfaces" (with different torsional offsets) for different grasp orientations. These observations suggest that the entire arm-hand system contributes to grasp orientation, and therefore, the reach/grasp distinction is not directly reflected in proximal-distal kinematics but is better reflected in the distinction between these coordinated orienting rules and the location-dependent kinematic rules for the upper arm that result in Donders' Law for one given grasp orientation.

Postural and Kinematic Synergies - Eigenpostures

Grinyagin, I V, Biryukova, E V, & Maier, M A, 2005: Kinematic and dynamic synergies of human precision-grip movements. J. Neurophysiol, 94(4):2284-2294.

We analyzed the adaptability of human thumb and index finger movement kinematics and dynamics to variations of precision grip aperture and movement velocity. Six subjects performed precision grip opening and closing movements under different conditions of movement velocity and movement aperture (thumb and index finger tip-to-tip distance). Angular motion of the thumb and index finger joints was recorded with a CyberGlove and a three-dimensional biomechanical model was used for solving the inverse dynamics problem during precision grip movements, i.e., for calculating joint torques from experimentally obtained angular variations. The time-varying joint angles and joint torques were analyzed by principal-component analysis to quantify the contributions of individual joints in kinematic and dynamic synergies. At the level of movement kinematics, we found subject-specific angular contributions. However, the adaptation to large aperture, achieved by an increase of the relative contribution of the proximal joints, was subject-invariant. At the level of movement dynamics, the adaptation of thumb-index finger movements to task constraints was similar among all subjects and required the linear scaling of joint torques, the synchronization of joint torques under high velocity conditions, and a flexible redistribution of joint torques between the proximal joint of the thumb and that of the index finger. This work represents one of the first attempts at calculating the joint torques during human precision-grip movements and indicates that the dynamic synergies seem to be remarkably simple compared with the synergies found for movement kinematics.

Weiss, E J, & Flanders, M, 2004: Muscular and postural synergies of the human hand. J. Neurophysiol., 92(1):523-535.

Because humans have limited ability to independently control the many joints of the hand, a wide variety of hand shapes can be characterized as a weighted combination of just two or three main patterns of covariation in joint rotations, or "postural synergies." The present study sought to align muscle synergies with these main postural synergies and to describe the form of membership of motor units in these postural/muscle synergies. Seventeen joint angles and the electromyographic (EMG) activities of several hand muscles (both intrinsic and extrinsic muscles) were recorded while human subjects held the hand statically in 52 specific shapes (i.e., shaping the hand around 26 commonly grasped objects or forming the 26 letter shapes of a manual alphabet). Principal-components analysis revealed several patterns of muscle synergy, some of which represented either coactivation of all hand muscles, or reciprocal patterns of activity (above and below average levels) in the intrinsic index finger and thumb muscles or (to a lesser extent) in the extrinsic four-tendoned extensor and flexor muscles. Single- and multiunit activity was generally a multimodal function of whole hand shape. This implies that motor-unit activation does not align with a single synergy; instead, motor units participate in multiple muscle synergies. Thus it appears that the organization of the global pattern of hand muscle activation is highly distributed. This organization mirrors the highly fractured somatotopy of cortical hand representations and may provide an ideal substrate for motor learning and recovery from injury.

Kamper, D G, Cruz, E G, & Siegel, M P, 2003: Stereotypical fingertip trajectories during grasp. J. Neurophysiol., 90(6):3702-3710.

The kinematics of movement of all five digits was analyzed during reach-and-grasp tasks for a variety of objects. Ten healthy subjects performed 20 trials involving the grasp of five objects of distinct size and shape. Joint angles were recorded, and digit trajectories were computed using forward kinematics. For a given subject, fingertip trajectories were consistent across trials. The different-sized objects largely produced movement along different portions of a stereotypical trajectory described by a logarithmic spiral. The spirals fit the actual finger positions with a mean error across all trials of 0.23 ± 0.25 cm and accounted for over 98% of the variance in finger position. These patterns were consistent independent of initial finger posture. Subjects did not produce straight-line movements, either in Cartesian space or joint space. The direction of the thumb trajectories exhibited a greater dependence on object type than the finger trajectories, but still utilized a small percentage (<5%) of the available workspace. These results suggest that restoration of a small but specific part of the workspace could have significant impact on function following hand impairment.

Santello, M, Flanders, M, & Soechting, J F, 2002: Patterns of hand motion during grasping and the influence of sensory guidance. J. Neurosci., 22(4):1426-1435.

This study was aimed at describing temporal synergies of hand movement and determining the influence of sensory cues on the control of these synergies. Subjects were asked to reach to and grasp various objects under three experimental conditions: (1) memory-guided movements, in which the object was not in view during the movement; (2) virtual object, in which a virtual image of the object was in view but the object was not physically present; and (3) real object, in which the object was in view and physically present. Motion of the arm and of 15 degrees of freedom of the hand was recorded. A principal components analysis was developed to provide a concise description of the spatiotemporal patterns underlying the motion. Vision of the object during the reaching movement had no influence on the kinematics, and the effect of the physical presence of the object became manifest primarily after the fingers had contacted the object. Two principal components accounted for >75% of the variance. For both components, there was a strong positive correlation in the rotations of metacarpophalangeal and proximal interphalangeal joints of the fingers. The first principal component exhibited a pattern of finger extension reversing to flexion, whereas the second principal component became important only in the second half of the reaching movement.

Mason, C R, Gomez, J E, & Ebner, T J, 2001: Hand synergies during reach-to-grasp. J. Neurophysiol., 86(6):2896-2910.

An emerging viewpoint is that the CNS uses synergies to simplify the control of the hand. Previous work has shown that static hand postures for mimed grasps can be described by a few principal components in which the higher order components explained only a small fraction of the variance yet provided meaningful information. Extending that earlier work, this study addressed whether the entire act of grasp can be described by a small number of postural synergies and whether these synergies are similar for different grasps. Five right-handed adults performed five types of reach-to-grasps including power grasp, power grasp with a lift, precision grasp, and mimed power grasp and mimed precision grasp of 16 different objects. The object shapes were cones, cylinders, and spindles, systematically varied in size to produce a large range of finger joint angle combinations. Three-dimensional reconstructions of 21 positions on the hand and wrist throughout the reach-to-grasp were obtained using a four-camera video system. Singular value decomposition on the temporal sequence of the marker positions was used to identify the common patterns ("eigenpostures") across the 16 objects for each task and their weightings as a function of time. The first eigenposture explained an average of 97.3 ± 0.89% (mean ± SD) of the variance of the hand shape, and the second another 1.9 ± 0.85%. The first eigenposture was characterized by an open hand configuration that opens and closes during reach. The second eigenposture contributed to the control of the thumb and long fingers, particularly in the opening of the hand during the reach and the closing in preparation for object grasp. The eigenpostures and their temporal evolutions were similar across subjects and grasps. The higher order eigenpostures, although explaining only small amounts of the variance, contributed to the movements of the fingers and thumb. These findings suggest that much of reach-to-grasp is effected using a base posture with refinements in finger and thumb positions added in time to yield unique hand shapes.

Santello, M, Flanders, M, & Soechting, J F, 1998: Postural hand synergies for tool use. J. Neurosci., 18(23):10105-10115.

Subjects were asked to shape the right hand as if to grasp and use a large number of familiar objects. The chosen objects typically are held with a variety of grips, including "precision" and "power" grips. Static hand posture was measured by recording the angular position of 15 joint angles of the fingers and of the thumb. Although subjects adopted distinct hand shapes for the various objects, the joint angles of the digits did not vary independently. Principal components analysis showed that the first two components could account for >80% of the variance, implying a substantial reduction from the 15 degrees of freedom that were recorded. However, even though they were small, higher-order (more than three) principal components did not represent random variability but instead provided additional information about the object. These results suggest that the control of hand posture involves a few postural synergies, regulating the general shape of the hand, coupled with a finer control mechanism providing for small, subtle adjustments. Because the postural synergies did not coincide with grip taxonomies, the results suggest that hand posture may be regulated independently from the control of the contact forces that are used to grasp an object.

Lacquaniti, F, & Soechting, J F, 1982: Coordination of arm and wrist motion during a reaching task. J. Neurosci., 2(4):399-408.

An analysis of arm movements involving forward projection of the hand in order to reach for and grasp a target at different orientations is presented. The reaching movements required shoulder flexion, elbow extension, and wrist pronation or supination. The relation between elbow and shoulder instantaneous angular position proved to be consistent from trial to trial of each task, independent of movement speed. Further, this relation was not influenced by the presence or absence of a concomitant wrist rotation. During the deceleratory phase of the movement, the slope of elbow angular velocity to shoulder angular velocity was constant and independent of target orientation. Wrist motion was instead highly variable in timing, course, and duration. Supinatory movements tended to be fractionated. On average, the duration of wrist movements was shorter than that of shoulder and elbow motions. The pattern of biceps EMG activity during supinatory and pronatory movements was different. Since motion at the shoulder and elbow was virtually identical in the two cases, net flexor torque at the elbow was also little different. It is concluded that other elbow flexors and extensors also exhibit a task-dependent patterning of activity so as to produce the same net torque. The results are discussed in the context of the internal constraints present during the movements that we examined. These constraints are the inertial coupling between shoulder and elbow motion and those which derive from the bifunctional nature of many of the muscles participating in the movement.

Planning, Goals, and Stimuli Before Movement

Lukos, J R, Ansuini, C, & Santello, M, 2008: Anticipatory control of grasping: Independence of sensorimotor memories for kinematics and kinetics. J. Neurosci., 28(48):12765-12774.

We have recently provided evidence for anticipatory grasp control mechanisms in the kinematic domain by showing that subjects modulate digit placement on an object based on its center of mass (CM) when it can be anticipated (Lukos et al., 2007). This behavior relied on sensorimotor memories about digit contact points and forces required for optimal manipulation. We found that accurate sensorimotor memories depended on the acquisition of implicit knowledge about object properties associated with repeated manipulations of the same object. Whereas implicit knowledge of object properties is essential for anticipatory grasp control, the extent to which subjects can use explicit knowledge to accurately scale digit forces in an anticipatory manner is controversial. Additionally, it is not known whether subjects are able to use explicit knowledge of object properties for anticipatory control of contact points. We addressed this question by asking subjects to grasp and lift an object while providing explicit knowledge of object CM location as visual or verbal cues. Contact point modulation and object roll, a measure of anticipatory force control, were assessed using blocked and random CM presentations. We found that explicit knowledge of object CM enabled subjects to modulate contact points. In contrast, subjects could not minimize object roll in the random condition to the same extent as in the blocked when provided with a verbal or visual cue. These findings point to a dissociation in the effect of explicit knowledge of object properties on grasp kinematics versus kinetics, thus suggesting independent anticipatory processes for grasping.

Ansuini, C, Santello, M, Massaccesi, S, & Castiello, U, 2006: Effects of end-goal on hand shaping. J. Neurophysiol., 95(4):2456-2465.

The aim of the present study was to determine whether hand shaping was affected by planning of an action subsequent to object contact. Ten subjects (5 females and 5 males, ages 19-33) were requested to reach toward and grasp a convex object between the thumb and the four fingers of the right hand and to perform one of the following actions: 1) lift up the object; 2) insert the object into a niche of a similar shape and size as the object, or 3) insert the object into a rectangular niche much larger than the object. Flexion /extension at the metacarpal-phalangeal and proximal interphalangeal joints of all digits were measured using resistive sensors embedded in a glove. Although all experimental conditions required grasping the same object, we found different covariation patterns among finger joint angles across conditions. Gradual preshaping of the hand occurred only when planning object lift or when the end-goal required object placement into the tight niche. In contrast, for the larger niche, gradual preshaping was not evident for the ring and the little finger. Further, reaching movements were faster for movements ending with the larger niche than for the other movement conditions. The present results suggest that hand shaping takes into account end-goal in addition to object geometry. We discuss these findings in the context of forward internal models that allow the prediction of the sensorimotor consequences of motor commands in advance to their execution.

Santello, M, & Soechting, J F, 1998: Gradual molding of the hand to object contours. J. Neurophysiol., 79(3):1307-1320.

Subjects were asked to reach to and to grasp 15 similarly sized objects with the four fingers opposed to the thumb. The objects' contours differed: some presented a concave surface to the fingers, others a flat one, and yet others a convex surface. Flexion/extension at the metacarpal-phalangeal and proximal interphalangeal joints of the fingers was recorded during the reaching movement. We used discriminant analysis, cluster analysis, and information theory to determine the extent to which the shape of the hand was affected by the objects' shapes along a convexity/concavity gradient. Maximum aperture of the hand was reached about midway in the reaching movement. At that time, the hand's posture was influenced by the shape of the object to be grasped but imperfectly. The information transmitted by hand posture about object deshape increased gradually and monotonically as the hand approached the object, reaching a maximum at the time the object was in the grasp of the hand. We also asked subjects to shape the hand so as to grasp the object without moving the arm. Their performance was poorer on this task in the sense that hand shape discriminated among fewer objects and that trial-to-trial variability was greater than when the distal and proximal components of the obmotion were linked. The results indicate that the hand is molded only gradually to the contours of an object to be grasped. Because other parameters of the motion, such as movement direction, for example, already are specified fully early on in a movement, the results also suggest that the specification of diverse aspects of a movement does not evolve at a uniform rate.

Santello, M, & Soechting, J F, 1997: Matching object size by controlling finger span and hand shape. Somatosens. Motor Res., 14(3):203-212.

The ability of human subjects to accurately control finger span (distance between thumb and one finger) was studied. The experiments were performed without visual feedback of the hand and were designed to study the dependence of accuracy on object size, shape, distance, orientation and finger configuration. The effects of finger combination and sensory modality used to perceive object size (vision and haptics) were also studied. Subjects were quite proficient at this task; the small errors tended to be predominantly negative, i.e., finger span < object size. The thumb-little finger combination was less accurate than the other finger combinations, irrespective of the sensory modality used. Subjects made larger under-estimating errors when matching the size of cylinders than when matching cubes and parallelepipeds. No effect of viewing distance, object orientation and finger configuration was found. Accuracy in matching object size was not dependent on the sensory modality used. The question of how the individual degrees of freedom of the fingers and thumb contributed to the control of finger span was also addressed. Principal components analysis showed that two components could characterize the hand postures used, irrespective of object size. The amplitude of the first principal component was constant, and the amplitude of the second scaled linearly with object size. This finding suggests that all of the degrees of freedom of the hand are controlled as a unit. This result is discussed in relation to the 'virtual finger' hypothesis for grasping.

Execution, Perturbations, and Stimuli During Movement

Scherbaum, S, Dshemuchadse, M, Fischer, R, & Goschke, T, 2010: How decisions evolve: The temporal dynamics of action selection. Cognition, 115:407-416.

To study the process of decision-making under conflict, researchers typically analyze response latency and accuracy. However, these tools provide little evidence regarding how the resolution of conflict unfolds over time. Here, we analyzed the trajectories of mouse movements while participants performed a continuous version of a spatial conflict task (the Simon task). We applied a novel combination of multiple regression analysis and distribution analysis to determine how conflict on the present trial and carry-over from the previous trial affect responding. Response on the previous trial and the degree of conflict on the previous and the current trial all influenced performance, but they did so differently: The previous response influenced the early part of the mouse trajectory, but the degree of previous and current conflict influenced later parts. This suggests that in this task experiencing conflict may not proactively ready the system to handle conflict on the next trial; rather, when conflict is experienced on the subsequent trial the previous compensatory processing may be re-activated more efficiently.

Ansuini, C, Tognin, V, Turella, L, & Castiello, U, 2007: Distractor objects affect fingers' angular distances but not fingers' shaping during grasping. Experim. Brain. Res., 178(2):194-205.

The aim of the present study was to determine whether and how hand shaping was affected by the presence of a distractor object adjacent to the tobe- grasped object. Twenty subjects were requested to reach towards and grasp a 'convex' or a 'concave' object in the presence or absence of a distractor object either of the same or different shape than the target object. Flexion/extension at the metacarpal -phalangeal (MCP) and proximal interphalangeal joints of all digits, and abduction angle between digits were measured by resistive sensors embedded in a glove. The results indicate robust interference effects at the level of reach duration and the extent of fingers' abduction angles together with changes at the level of a single joint for the thumb. No distractor effects on individual fingers' joints except for the MCP of the middle and little fingers were found. These findings suggest that the presence of distractor object affects hand shaping in terms of fingers' abduction angles, but not at the level of 'shape dependent' fingers' angular excursions. Furthermore, they support the importance of the thumb for the guidance of selective reach-to-grasp movements. We discuss these results in the context of current theories proposed to explain the object selection processes underlying the control of hand action.

Ansuini, C, Santello, M, Tubaldi, F, Massaccesi, S, & Castiello, U, 2007: Control of hand shaping in response to object shape perturbation. Exp. Brain Res., 180(1):85-96.

This study assessed how hand shaping responds to a perturbation of object shape. In blocked trials (80% of total), subjects were instructed to reach, to grasp and lift a concave or a convex object. In perturbed trials (20% of total), a rotating device allowed for the rapid change from the concave to the convex object or vice versa. In this situation subjects grasped the last presented object. Flexion/extension at the metacarpal -phalangeal and proximal interphalangeal joints of all digits was measured by resistive sensors embedded in a glove. In the blocked condition we found that most joints of the fingers were modulated by the type of the to-be-grasped object during the reach. When object shape was perturbed, reach duration was longer and angular excursion of all fingers differed with respect to blocked trials. For the 'convex => concave' perturbation, a greater degree of finger extension was found than during the blocked 'concave' trials. In contrast, for the 'concave => convex' perturbation, fingers were more flexed than for the blocked 'convex' trials. The thumb reacted to the perturbation showing a similar pattern (i.e., over-flexion with respect to the blocked trials) regardless the 'direction' of the perturbation. The present results suggest that applying an object shape perturbation during a reach-to-grasp action determines a reorganization of all digits. This pattern is suggestive of a control strategy, which assigns to opposing digits different roles.

Ma-Wyatt, A, & McKee, S P, 2007: Visual information throughout a reach determines endpoint precision. Exp. Brain Res., 179(1):55-64.

People make rapid, goal-directed movements to interact with their environment. Because these movements have consequences, it is important to be able to control them with a high level of precision and accuracy. Our hypothesis is that vision guides rapid hand movements, thereby enhancing their accuracy and precision. To test this idea, we asked observers to point to a briefly presented target (110 ms). We measured the impact of visual information on endpoint precision by using a shutter to close off view of the hand 50, 110 and 250 ms into the reach. We found that precision was degraded if the view of the hand was restricted at any time during the reach, despite the fact that the target disappeared long before the reach was completed. We therefore conclude that vision keeps the hand on the planned trajectory. We then investigated the effects of a perturbation of target position during the reach. For these experiments, the target remained visible until the reach was completed. The target position was shifted at 110, 180 or 250 ms into the reach. Early shifts in target position were easily compensated for, but late shifts led to a shift in the mean position of the endpoints; observers pointed to the center of the two locations, as a kind of best bet on the position of the target. Visual information is used to guide the hand throughout a reach and has a significant impact on endpoint precision.

Saccadic Eye Movements

Sailer, U, Eggert, T, Ditterich, J, & Straube, A, 2002: Global effect of a nearby distractor on targeting eye and hand movements. J. Experim. Psych. Hum. Percept. Perform., 28(6):1432-1446.

Eye-hand coordination was investigated with the global effect paradigm. In this paradigm, saccades typically land in between the target and a nearby presented distractor, the configuration's center of gravity. This so-called global effect, or spatial averaging, is attributed to incomplete target selection. Four experiments demonstrated a similar effect for hand movements; thus, eye and hand are coupled during target selection. However, under some conditions the global effect was different for eye and hand, suggesting that their coupling is not achieved through a shared target representation. Instead, eye and hand seem to use 2 separate target representations that exchange information. The convergent amplitudes of eye and hand with simultaneous execution support this interpretation. Latencies showed a similar converging pattern.

Van der Stigchel, S, Meeter, M, & Theeuwes, J, 2006: Eye movement trajectories and what they tell us. Neurosci. Biobehav. Rev., 30(5):666-679.

In the last two decades, research has shown that eye movement trajectories can be modified by situational determinants. These modifications can inform us about the mechanisms that control eye movements and they can yield information about the oculomotor, memory and attention system that is not easily obtained via other sources. Eye movement trajectories can deviate either towards or away from elements in the visual field. We review the conditions in which these deviations are found and the mechanisms underlying trajectory deviations. It is argued that deviations towards an element are caused by the unresolved competition in the oculomotor system between elements in a visual scene. Deviations away from an element are mainly observed in situations in which top-down preparation can influence the target selection process, but the exact cause of such deviations remains unclear.

Tweed, D, 2006: Sensorimotor optimization in higher dimensions. Prog. Brain Res., 165:181-191.

Most studies of neural control have looked at constrained tasks, with only a few degrees of freedom, but real sensorimotor systems are high dimensional - e.g. gaze-control systems that coordinate the head and two eyes have to work with 12 degrees of freedom in all. These extra degrees of freedom matter, because they bring with them new issues and questions, which make it hard to translate low-dimensional findings into theories of real neural control. Here I show that it is possible to predict high-dimensional behavior if we apply the optimization principles introduced by 19th-century neuroscientists like Helmholtz, Listing, and Wundt. Using three examples - the vestibulo-ocular reflex, saccadic eye movements, and depth vision - I show how simple optimization theories can predict complex, unexpected behaviors and reveal fundamental features of sensorimotor control, e.g. that neural circuits perform noncommutative algebra; that in rapid gaze shifts the eye controllers deliver commands with three degrees of freedom, not two; and that the eyes roll about their lines of sight in a way that may simplify stereopsis.

Munoz, D P, 2002: Commentary: saccadic eye movements: overview of neural circuitry. Prog. Brain Res., 140:89-96.

Recent neuroanatomical, neurophysiological, clinical, and brain imaging studies have generated a wealth of data describing the neural control of saccadic eye movements and visual fixation. These studies have identified many of the cortical and subcortical structures involved in controlling the behavior. Critical nodes in the network include regions of the parietal and frontal cortices, basal ganglia, thalamus, superior colliculus, cerebellum, and brainstem reticular formation. Specific functions are likely not localized to only one brain area, but rather, they may be distributed across multiple areas. This commentary is used to review briefly the neural circuitry controlling saccadic eye movements and visual fixation.

Eye-Hand Coordination

Pisella, L, Grea, H, Tilikete, C, Vighetto, A, Desmurget, M, Rode, G, Boisson, D, & Rossetti, Y, 2000: An 'automatic pilot' for the hand in human posterior parietal cortex: toward reinterpreting optic ataxia. Nat. Neurosci., 3(7):729-736.

We designed a protocol distinguishing between automatic and intentional motor reactions to changes in target location triggered at movement onset. In response to target jumps, but not to a similar change cued by a color switch, normal subjects often could not avoid automatically correcting fast aiming movements. This suggests that an 'automatic pilot' relying on spatial vision drives fast corrective arm movements that can escape intentional control. In a patient with a bilateral posterior parietal cortex (PPC) lesion, motor corrections could only be slow and deliberate. We propose that 'on-line' control is the most specific function of the PPC and that optic ataxia could result from a disruption of automatic hand guidance.

Sensorimotor Processing, Interaction and Control

Wolpert, D M, Ghahramani, Z, 2000: Computational principles of movement neuroscience. Nat. Neurosci., 3(Suppl.):1212-1217.

Unifying principles of movement have emerged from the computational study of motor control. We review several of these principles and show how they apply to processes such as motor planning, control, estimation, prediction and learning. Our goal is to demonstrate how specific models emerging from the computational approach provide a theoretical framework for movement neuroscience.

Fecteau, J H, & Munoz, D P, 2006: Salience, relevance, and firing: a priority map for target selection. Trends Cog. Sci., 10(8):382-390.

The salience map is a crucial concept for many theories of visual attention. On this map, each object in the scene competes for selection - the more conspicuous the object, the greater its representation, and the more likely it will be chosen. In recent years, the firing patterns of single neurons have been interpreted using this framework. Here, we review evidence showing that the expression of salience is remarkably similar across structures, remarkably different across tasks, and modified in important ways when the salient object is consistent with the goals of the participant. These observations have important ramifications for theories of attention. We conclude that priority - the combined representation of salience and relevance - best describes the firing properties of neurons.

Fecteau, J H, & Munoz, D P, 2007: Warning signals influence motor processing. J. Neurophysiol., 97(2):1600-1609.

When observers initiate responses to visual targets, they do so sooner when a preceding stimulus indicates that the target will appear shortly. This consequence of a warning signal may change neural activity in one of four ways. On the sensory side, the warning signal may speed up the rate at which the target is registered by the brain or enhance the magnitude of its signal. On the motor end, the warning signal may lower the threshold required to initiate a response or speed up the rate at which activity accumulates to reach threshold. Here, we describe which explanation is better supported. To accomplish this end, monkeys performed different versions of a cue-target task while we monitored the activity of visuomotor and motor neurons in the superior colliculus. Although the cue target task was designed to measure the properties of reflexive spatial attention, there are two events in this task that produce nonspecific warning effects: a central reorienting event (brightening of central fixation marker) that is used to direct attention away from the cue, and the presentation of the cue itself. Monopolizing on these tendencies, we show that warning effects are associated with several changes in neural activity: the target-related response is enhanced, the threshold for initiating a saccade is lowered, and the rate at which activity accumulates toward threshold rises faster. Ultimately, the accumulation of activity toward threshold predicted behavior most closely. In the discussion, we describe the implications and limitations of these data for theories of warning effects and potential avenues for future research.

Connolly, J D, Goodale, M A, Cant, J S, & Munoz, D P, 2007: Effector-specific fields for motor preparation in the human frontal cortex. NeuroImage, 34(3):1209-1219.

We investigated the neural correlates of advance motor preparation in two experiments that required a movement in response to a peripheral visual stimulus. In one experiment (the memory delay paradigm), subjects knew the target location during a preparatory 'memory delay' interval; in the other experiment they did not know the target location during a 'gap period' (the gap paradigm). In both experiments we further varied the effector that was instructed, either the eye or the forelimb. An area that codes motor preparation should exhibit increases during the memory delay and gap period and such increases should predict some attribute of performance (planning to use the eye or the forelimb). We first identified the frontoparietal visuomotor areas using standard fMRI block designs. Subjects were then scanned using event-related fMRI. With the exception of primary motor cortex (M1), all areas (putative lateral intraparietal area (putLIP), dorsal premotor cortex (PMd), frontal eye field (FEF), ventral frontal eye field (FEFv), supplementary motor area (SMA)) showed gap and memory delay activation for both saccades and pointing. Gap activity in the frontal areas was higher than in the parietal area(s) investigated. The observation that 'memory delay' activity was equivalent or less than gap activity in all areas suggests that what is commonly considered to be memory-related responses largely represents advance motor preparation. Certain areas showed increased activation during the gap or memory delay intervals for pointing (PMd, FEF, FEFv) or saccades (SMA, putLIP). These observations suggest an important role of the frontal cortex in advance motor preparation.

Watanabe, M, & Munoz, D P, 2010: Presetting basal ganglia for volitional actions. J. Neurosci., 30(30):10144-10157.

The basal ganglia (BG) have been considered as a key structure for volitional action preparation. Neurons in the striatum, the main BG input stage, increase activity gradually before volitional action initiation. However, because of the diversity of striatal motor commands, such as automatic (sensory driven) and volitional (internally driven) actions, it is still unclear whether an appropriate set of neurons encoding volitional actions are activated selectively for volitional action preparation. Here, using the antisaccade paradigm (look away from a visual stimulus), we dissociated neurons in the caudate nucleus, the oculomotor striatum, encoding predominantly automatic saccades toward the stimulus and volitional saccades in the opposite direction of the stimulus in monkeys. We found that before actual saccade directions were defined by visual stimulus appearance, neurons encoding volitional saccades increased activity with elapsed time from fixation initiation and by a temporal gap between fixation point disappearance and stimulus appearance. Their activity was further enhanced by an antisaccade instruction and correlated with antisaccade behavior. Neurons encoding automatic saccades also increased activity with elapsed time from fixation initiation and by a fixation gap. However, the activity of this type of neuron was not enhanced by an antisaccade instruction nor correlated with antisaccade behavior. We conclude that caudate neurons integrate nonspatial signals, such as elapsed time from fixation initiation, fixation gap, and task instructions, to preset BG circuits in favor of volitional actions to compete against automatic actions even before automatic and volitional commands are programmed with spatial information.

Motor Habituation, Adaptation and Learning

Arce, F, Novick, I, & Vaadia, E, 2010: Discordant tasks and motor adjustments affect interactions between adaptations to altered kinematics and dynamics. Front. Human Neurosci., 3:65.

Motor control and adaptation are multi-determinate processes with complex interactions. This is reflected for example in the ambiguous nature of interactions during sequential adaptation of reaching under kinematics and dynamics perturbations. It has been suggested that perturbations based on the same kinematic parameter interfere. Others posited that opposing motor adjustments underlie interference. Here, we examined the influence of discordances in task and in motor adjustments on sequential adaptations to visuomotor rotation and viscous force field perturbations. These two factors - perturbation direction and task discordance - have been examined separately by previous studies, thus the inherent difficulty to identify the roots of interference. Forty-eight human subjects adapted sequentially to one or two types of perturbations, of matched or conflicting directions. We found a gradient of interaction effects based on perturbation direction and task discordance. Perturbations of matched directions showed facilitation while perturbations of opposite directions, which required opposing motor adjustments, interfered with each other. Further, interaction effects increased with greater task discordance. We also found that force field and visuomotor rotation had mutual anterograde and retrograde effects. However, we found independence between anterograde and retrograde interferences between similar tasks. The results suggest that the newly acquired internal models of kinematic and dynamic perturbations are not independent but they share common neuronal resources and interact between them. Such overlap does not necessarily imply competition of resources. Rather, our results point to an additional principle of sensorimotor adaptation allowing the system to tap or harness common features across diverse sensory inputs and task contexts whenever available.

Diedrichsen, J, White, O, Newman, D, & Lally, N, 2010: Use-dependent and error-based learning of motor behaviors. J. Neurosci., 30(15):5159-5166.

Human motor behavior is constantly adapted through the process of error-based learning. When the motor system encounters an error, its estimate about the body and environment will change, and the next movement will be immediately modified to counteract the underlying perturbation. Here, we show that a second mechanism, use-dependent learning, simultaneously changes movements to become more similar to the last movement. In three experiments, participants made reaching movements toward a horizontally elongated target, such that errors in the initial movement direction did not have to be corrected. Along this task-redundant dimension, we were able to induce use-dependent learning by passively guiding movements in a direction angled by 8° from the previous direction. In a second study, we show that error-based and use-dependent learning can change motor behavior simultaneously in opposing directions by physically constraining the direction of active movements. After removal of the constraint, participants briefly exhibit an error-based aftereffect against the direction of the constraint, followed by a longer-lasting use -dependent aftereffect in the direction of the constraint. In the third experiment, we show that these two learning mechanisms together determine the solution the motor system adopts when learning a motor task.

Mattar, A A, Ostry, D J, 2007: Neural averaging in motor learning. J. Neurophysiol., 97(1):220-228.

The capacity for skill development over multiple training episodes is fundamental to human motor function. We have studied the process by which skills evolve with training by progressively modifying a series of motor learning tasks that subjects performed over a 1-mo period. In a series of empirical and modeling studies, we show that performance undergoes repeated modification with new learning. Each in a series of prior training episodes contributes such that present performance reflects a weighted average of previous learning. Moreover, we have observed that the relative weighting of skills learned wholly in the past changes with time. This suggests that the neural substrate of skill undergoes modification after consolidation.

Fecteau, J H, & Munoz, D P, 2003: Exploring the consequences of the previous trial. Nature Rev. Neurosci., 4(6):435-443.

In tasks that are designed to explore cognitive functioning, the response on each trial is a function of the combination of experimental conditions that occurred on that and the previous trial. Because the previous trial influences performance, the event presented during or the action required by the previous trial must leave an imprint on the brain's activity that carries through to the next trial. These imprints are manifest in the activity of single neurons that participate in producing the response. Previous trial effects address disparate cognitive phenomena, such as response priming, task switching and inhibition of return, and the neural bases of previous trial effects can be envisioned as changes in salience of the target or the goal of the action on a spatial map.

Thoroughman, K A, Fine, M S, & Taylor, J A, 2007: Trial-by-trial motor adaptation: a window into elemental neural computation. Prog. Brain. Res., 165:373-382.

How does the brain compute? To address this question, mathematical modelers, neurophysiologists, and psychophysicists have sought behaviors that provide evidence of specific neural computations. Human motor behavior consists of several such computations [Shadmehr, R., Wise, S.P. (2005). MIT Press: Cambridge, MA], such as the transformation of a sensory input to a motor output. The motor system is also capable of learning new transformations to produce novel outputs; humans have the remarkable ability to alter their motor output to adapt to changes in their own bodies and the environment [Wolpert, D.M., Ghahramani, Z. (2000). Nat. Neurosci., 3: 1212-1217]. These changes can be long term, through growth and changing body proportions, or short term, through changes in the external environment. Here we focus on trial-by-trial adaptation, the transformation of individually sensed movements into incremental updates of adaptive control. These investigations have the promise of revealing important basic principles of motor control and ultimately guiding a new understanding of the neuronal correlates of motor behaviors.

van Beers, R J, 2009: Motor learning is optimally tuned to the properties of motor noise. Neuron, 63(3):406-417.

In motor learning, our brain uses movement errors to adjust planning of future movements. This process has traditionally been studied by examining how motor planning is adjusted in response to visuomotor or dynamic perturbations. Here, I show that the learning strategy can be better identified from the statistics of movements made in the absence of perturbations. The strategy identified this way differs from the learning mechanism assumed in mainstream models for motor learning. Crucial for this strategy is that motor noise arises partly centrally, in movement planning, and partly peripherally, in movement execution. Corrections are made by modification of central planning signals from the previous movement, which include the effects of planning but not execution noise. The size of the corrections is such that the movement variability is minimized. This physiologically plausible strategy is optimally tuned to the properties of motor noise, and likely underlies learning in many motor tasks.

Whitwell, R L, & Goodale, M A, 2009: Updating the programming of a precision grip is a function of recent history of available feedback. Experim. Brain. Res., 194(4):619-629.

In a recent study (Whitwell et al. in Exp Brain Res 185:111-119, 2008), we showed that the visuomotor system is "cognitively impenetrable" to the extent that explicit predictive knowledge of the availability of visual feedback on an upcoming trial fails to optimize grasping. The results suggested that the effects of trial history, rather than the anticipatory knowledge of the nature of an upcoming trial, plays the most significant role in how the availability of visual feedback is exploited by the visuomotor system when programming grip aperture (e.g., opening the hand wider when visual feedback is unavailable). Here, we provide direct evidence that trial history indeed plays a critical role in the programming of grip aperture. Twelve individuals grasped objects of three different sizes placed at one of two distances either with or without visual feedback of the hand and object (closed- or open-loop trials, respectively). Runs of four consecutive closed- or open-loop trials were interleaved with sequences of closed and open-loop trials that alternated back and forth from trial to trial. Peak grip aperture (PGA) decreased linearly with successive closed-loop trials and increased linearly with successive open-loop trials. We also compared PGA for trials that were preceded by a run of four consecutive closed- (or open-loop) trials with trials that were preceded by only one closed- (or open-loop) trial. This analysis indicated that consistency in the runs of closed- or open-loop trials significantly reduced the effect of the availability of feedback on grasping in the trial following the run. We conclude that while the margin of error observed in precision grasping is largely a function of the availability of visual feedback on the current trial, it is evidently also a function of the recent history of the availability of visual feedback on previous trials.

Dorris, M C, Paré, M, & Munoz, D P, 2000: Immediate neural plasticity shapes motor performance. J. Neurosci., 20(1):RC52.

The consolidation of motor skills necessitates long-lasting changes in the nervous system. For the most part, plasticity has been documented in motor systems after training and longterm adaptation. However, there has been no demonstration of immediate neural changes associated with the rapid adaptation of motor behavior required to interact with a dynamic environment. To address this issue, we explored the changes in performance (reaction time) of rhesus monkeys that executed saccadic eye movements to one of two visual stimuli while monitoring the preparatory activity of neurons in the superior colliculus, a structure close to the motor output. Similar to the well established sequential effects observed in human manual responses, each monkey displayed reaction times to target locations that were organized in a sequential pattern, becoming progressively shorter with each preceding repeated movement and longer with each preceding nonrepeated movement. This sequential pattern of performance modification was associated with concordant changes in the preparatory activity of superior colliculus neurons in advance of the saccadic target presentation. These data indicate that neural properties are continuously shaped by use-related experience in a manner consistent with the progressive adaptation of motor behavior.

Tremblay, L, Welsh, T N, & Elliott D, 2005: Between-trial inhibition and facilitation in goal-directed aiming: manual and spatial asymmetries. Experim. Brain. Res., 160(1):79-88.

Three experiments were conducted with right-handed participants to examine between-trial inhibition and facilitation effects in goal-directed aiming. Participants were required to execute rapid left-hand or right -hand aiming movements upon illumination of a target light in left or right space. Thus, from trial to trial, participants executed movements to either the same target location or a different target location with the either same hand or the other hand. Our reaction time results indicated that participants were particularly slow in initiating their movements when they were required to return to the same target location with the other hand. This was especially the case when the right hand was required to move to a target just occupied by the left hand. For both reaction time and movement time the right hand but not the left hand exhibited an advantage when it was required to perform the same movement two times in a row. Taken together these results suggest that inhibition of return, in a target-target paradigm, is more associated with the particular spatial location of the target than the organization of a specific movement to that location. Moreover, the between-trial facilitation observed for the right hand may reflect the ability of the left cerebral hemisphere to maintain an already parameterized motor program over a short intertrial interval.

Körding, K P, & Wolpert, D M, 2004: The loss function of sensorimotor learning. Proc. Nat'l Acad. Sci., 101(26):9839-9842.

Motor learning can be defined as changing performance so as to optimize some function of the task, such as accuracy. The measure of accuracy that is optimized is called a loss function and specifies how the CNS rates the relative success or cost of a particular movement outcome. Models of pointing in sensorimotor control and learning usually assume a quadratic loss function in which the mean squared error is minimized. Here we develop a technique for measuring the loss associated with errors. Subjects were required to perform a task while we experimentally controlled the skewness of the distribution of errors they experienced. Based on the change in the subjects' average performance, we infer the loss function. We show that people use a loss function in which the cost increases approximately quadratically with error for small errors and significantly less than quadratically for large errors. The system is thus robust to outliers. This suggests that models of sensorimotor control and learning that have assumed minimizing squared error are a good approximation but tend to penalize large errors excessively.

Ganesh, G, Haruno, M, Kawato, M, & Burdet, E, 2010: Motor memory and local minimization of error and effort, not global optimization, determine motor behavior. J. Neurophysiol., 104(1):382-390.

Many real life tasks that require impedance control to minimize motion error are characterized by multiple solutions where the task can be performed either by co-contracting muscle groups, which requires a large effort, or, conversely, by relaxing muscles. However, human motor optimization studies have focused on tasks that are always satisfied by increasing impedance and that are characterized by a single error-effort optimum. To investigate motor optimization in the presence of multiple solutions and hence optima, we introduce a novel paradigm that enables us to let subjects repetitively (but inconspicuously) use different solutions and observe how exploration of multiple solutions affect their motor behavior. The results show that the behavior is largely influenced by motor memory with subjects tending to involuntarily repeat a recent suboptimal task-satisfying solution even after sufficient experience of the optimal solution. This suggests that the CNS does not optimize co-activation tasks globally but determines the motor behavior in a tradeoff of motor memory, error, and effort minimization.

Izawa, J, Rane, T, Donchin, O, & Shadmehr, R, 2008: Motor adaptation as a process of reoptimization. J. Neurosci., 28(11):2883-2891.

Adaptation is sometimes viewed as a process in which the nervous system learns to predict and cancel effects of a novel environment, returning movements to near baseline (unperturbed) conditions. An alternate view is that cancellation is not the goal of adaptation. Rather, the goal is to maximize performance in that environment. If performance criteria are well defined, theory allows one to predict the reoptimized trajectory. For example, if velocity-dependent forces perturb the hand perpendicular to the direction of a reaching movement, the best reach plan is not a straight line but a curved path that appears to overcompensate for the forces. If this environment is stochastic (changing from trial to trial), the reoptimized plan should take into account this uncertainty, removing the overcompensation. If the stochastic environment is zero-mean, peak velocities should increase to allow for more time to approach the target. Finally, if one is reaching through a via-point, the optimum plan in a zero-mean deterministic environment is a smooth movement but in a zero-mean stochastic environment is a segmented movement. We observed all of these tendencies in how people adapt to novel environments. Therefore, motor control in a novel environment is not a process of perturbation cancellation. Rather, the process resembles reoptimization: through practice in the novel environment, we learn internal models that predict sensory consequences of motor commands. Through reward-based optimization, we use the internal model to search for a better movement plan to minimize implicit motor costs and maximize rewards.

Braun, D A, Aertsen, A, Wolpert, D M, & Mehring, C, 2009: Learning optimal adaptation strategies in unpredictable motor tasks. J. Neurosci., 29(20):6472-6478.

Picking up an empty milk carton that we believe to be full is a familiar example of adaptive control, because the adaptation process of estimating the carton's weight must proceed simultaneously with the control process of moving the carton to a desired location. Here we show that the motor system initially generates highly variable behavior in such unpredictable tasks but eventually converges to stereotyped patterns of adaptive responses predicted by a simple optimality principle. These results suggest that adaptation can become specifically tuned to identify task-specific parameters in an optimal manner.

Smith, M A, Ghazizadeh, A, & Shadmehr, R, 2006: Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol., 4:e179.

Multiple processes may contribute to motor skill acquisition, but it is thought that many of these processes require sleep or the passage of long periods of time ranging from several hours to many days or weeks. Here we demonstrate that within a timescale of minutes, two distinct fast-acting processes drive motor adaptation. One process responds weakly to error but retains information well, whereas the other responds strongly but has poor retention. This twostate learning system makes the surprising prediction of spontaneous recovery (or adaptation rebound) if error feedback is clamped at zero following an adaptation -extinction training episode. We used a novel paradigm to experimentally confirm this prediction in human motor learning of reaching, and we show that the interaction between the learning processes in this simple two-state system provides a unifying explanation for several different, apparently unrelated, phenomena in motor adaptation including savings, anterograde interference, spontaneous recovery, and rapid unlearning. Our results suggest that motor adaptation depends on at least two distinct neural systems that have different sensitivity to error and retain information at different rates.

Computational Neuroanatomy of Motor Action

Shadmehr, R, & Krakauer, J W, 2009: Computational neuroanatomy of voluntary motor control. ed: Gazzaniga, M S, The Cognitive Neurosciences, 587-597, MIT Press.

We review some of the impairments in motor control, motor learning and higher-order motor control in patients with lesions of the cerebellum, parietal cortex, and basal ganglia. We attempt to explain some of these impairments in terms of computational ideas such as state estimation, optimization, prediction, cost, and reward. We suggest that a function of the cerebellum is system identification: to built internal models that predict sensory outcome of motor commands and correct motor commands through internal feedback. A function of the parietal cortex is state estimation: to integrate the predicted proprioceptive and visual outcomes with sensory feedback to form a belief about how the commands affected the states of the body and the environment. A function of basal ganglia is related to optimal control: learning costs and rewards associated with sensory states and estimating the "cost-to-go" during execution of a motor task.

Darvas, F, Miller, K J, Rao, R P N, & Ojemann, J G, 2009: Nonlinear phase-phase cross-frequency coupling mediates communication between distant sites in human neocortex. J. Neurosci., 29(2):426-435.

Human cognition is thought to be mediated by large-scale interactions between distant sites in the neocortex. Synchronization between different cortical areas has been suggested as one possible mechanism for corticocortical interaction. Here, we report robust, directional cross-frequency synchronization between distant sensorimotor sites in human neocortex during a movement task. In four subjects, electrocorticographic recordings from the cortical surface revealed a low-frequency rhythm (10-13 Hz) that combined with a higher frequency (77-82 Hz) in a ventral region of the premotor cortex to produce a third rhythm at the sum of these two frequencies in a distant motor site. Such cross-frequency coupling implies a nonlinear interaction between these cortical sites. These findings demonstrate that task-specific, phase -phase coupling can support communication between distant areas of the human neocortex.

Shadmehr, R, & Krakauer, J W, 2008: A computational neuroanatomy for motor control. Experim. Brain Res., 185(3):359-381.

The study of patients to infer normal brain function has a long tradition in neurology and psychology. More recently, the motor system has been subject to quantitative and computational characterization. The purpose of this review is to argue that the lesion approach and theoretical motor control can mutually inform each other. Specifically, one may identify distinct motor control processes from computational models and map them onto specific deficits in patients. Here we review some of the impairments in motor control, motor learning and higher-order motor control in patients with lesions of the corticospinal tract, the cerebellum, parietal cortex, the basal ganglia, and the medial temporal lobe. We attempt to explain some of these impairments in terms of computational ideas such as state estimation, optimization, prediction, cost, and reward. We suggest that a function of the cerebellum is system identification: to build internal models that predict sensory outcome of motor commands and correct motor commands through internal feedback. A function of the parietal cortex is state estimation: to integrate the predicted proprioceptive and visual outcomes with sensory feedback to form a belief about how the commands affected the states of the body and the environment. A function of basal ganglia is related to optimal control: learning costs and rewards associated with sensory states and estimating the "cost-to-go" during execution of a motor task. Finally, functions of the primary and the premotor cortices are related to implementing the optimal control policy by transforming beliefs about proprioceptive and visual states, respectively, into motor commands.

Shadmehr, R, & Krakauer, J W, 2007: Towards a computational neuropsychology of action. Prog. Brain Res., 165:383-394.

From a computational perspective, the act of using a tool and making a movement involves solving three kinds of problems: we need to learn the costs that are associated with our actions as well as the rewards that we may experience at various sensory states. We need to learn how our motor commands produce changes in things that we can sense. Finally, we must learn how to actually produce the motor commands that are needed so that we minimize the costs and maximize the rewards. The various computational problems appear to require different kinds of error signals that guide their learning, and might rely on different kinds of contextual cues that allow their recall. Indeed, there may be different neural structures that compute these functions. Here we use this computational framework to review the motor control capabilities of two important patients who have been studied extensively from the neuropsychological perspective: HM, who suffered from severe amnesia; and BG, who suffered from apraxia. When viewed from a computational perspective, the capabilities and deficits of these patients provide insights into the neural basis of our ability to willfully move our limbs and interact with the objects around us.

Shadmehr, R, & Mussa-Ivaldi, F A, 1994: Adaptive representation of dynamics during learning of a motor task. J. Neurosci., 14(5):3208-3224.

We investigated how the CNS learns to control movements in different dynamical conditions, and how this learned behavior is represented. In particular, we considered the task of making reaching movements in the presence of externally imposed forces from a mechanical environment. This environment was a force field produced by a robot manipulandum, and the subjects made reaching movements while holding the end -effector of this manipulandum. Since the force field significantly changed the dynamics of the task, subjects' initial movements in the force field were grossly distorted compared to their movements in free space. However, with practice, hand trajectories in the force field converged to a path very similar to that observed in free space. This indicated that for reaching movements, there was a kinematic plan independent of dynamical conditions. The recovery of performance within the changed mechanical environment is motor adaptation. In order to investigate the mechanism underlying this adaptation, we considered the response to the sudden removal of the field after a training phase. The resulting trajectories, named aftereffects, were approximately mirror images of those that were observed when the subjects were initially exposed to the field. This suggested that the motor controller was gradually composing a model of the force field, a model that the nervous system used to predict and compensate for the forces imposed by the environment. In order to explore the structure of the model, we investigated whether adaptation to a force field, as presented in a small region, led to aftereffects in other regions of the workspace. We found that indeed there were aftereffects in workspace regions where no exposure to the field had taken place; that is, there was transfer beyond the boundary of the training data. This observation rules out the hypothesis that the subject's model of the force field was constructed as a narrow association between visited states and experienced forces; that is, adaptation was not via composition of a look-up table. In contrast, subjects modeled the force field by a combination of computational elements whose output was broadly tuned across the motor state space. These elements formed a model that extrapolated to outside the training region in a coordinate system similar to that of the joints and muscles rather than end-point forces. This geometric property suggests that the elements of the adaptive process represent dynamics of a motor task in terms of the intrinsic coordinate system of the sensors and actuators.

Motor Commands and Noise

Jones, K E, Hamilton, A F, & Wolpert, D M, 2002: Sources of signal-dependent noise during isometric force production. J. Neurophysiol., 88(3):1533-1544.

It has been proposed that the invariant kinematics observed during goal-directed movements result from reducing the consequences of signal-dependent noise (SDN) on motor output. The purpose of this study was to investigate the presence of SDN during isometric force production and determine how central and peripheral components contribute to this feature of motor control. Peripheral and central components were distinguished experimentally by comparing voluntary contractions to those elicited by electrical stimulation of the extensor pollicis longus muscle. To determine other factors of motor-unit physiology that may contribute to SDN, a model was constructed and its output compared with the empirical data. SDN was evident in voluntary isometric contractions as a linear scaling of force variability (SD) with respect to the mean force level. However, during electrically stimulated contractions to the same force levels, the variability remained constant over the same range of mean forces. When the subjects were asked to combine voluntary with stimulation-induced contractions, the linear scaling relationship between the SD and mean force returned. The modeling results highlight that much of the basic physiological organization of the motor-unit pool, such as range of twitch amplitudes and range of recruitment thresholds, biases force output to exhibit linearly scaled SDN. This is in contrast to the square root scaling of variability with mean force present in any individual motor-unit of the pool. Orderly recruitment by twitch amplitude was a necessary condition for producing linearly scaled SDN. Surprisingly, the scaling of SDN was independent of the variability of motoneuron firing and therefore by inference, independent of presynaptic noise in the motor command. We conclude that the linear scaling of SDN during voluntary isometric contractions is a natural by-product of the organization of the motor-unit pool that does not depend on signal-dependent noise in the motor command. Synaptic noise in the motor command and common drive, which give rise to the variability and synchronization of motoneuron spiking, determine the magnitude of the force variability at a given level of mean force output.

Harris, C M, & Wolpert, D M, 1998: Signal-dependent noise determines motor planning. Nature, 394(6695):780-784.

When we make saccadic eye movements or goal-directed arm movements, there is an infinite number of possible trajectories that the eye or arm could take to reach the target. However, humans show highly stereotyped trajectories in which velocity profiles of both the eye and hand are smooth and symmetric for brief movements. Here we present a unifying theory of eye and arm movements based on the single physiological assumption that the neural control signals are corrupted by noise whose variance increases with the size of the control signal. We propose that in the presence of such signal-dependent noise, the shape of a trajectory is selected to minimize the variance of the final eye or arm position. This minimum -variance theory accurately predicts the trajectories of both saccades and arm movements and the speed -accuracy trade-off described by Fitt's law. These profiles are robust to changes in the dynamics of the eye or arm, as found empirically. Moreover, the relation between path curvature and hand velocity during drawing movements reproduces the empirical 'two-thirds power law.' This theory provides a simple and powerful unifying perspective for both eye and arm movement control.

Lacquaniti, F, Terzuolo, C A, & Viviani, P, 1983: The law relating kinematic and figural aspects of drawing movements. Acta Psychologica, 54(1-3):115-130.

Considerable evidence exists that the velocity of execution of handwriting and drawing movements depends on some global metric properties of the movement (size, linear extent etc.). Recent experiments have demonstrated that the instantaneous velocity also depends on the local curvature of the trajectory, that is, on the differential geometrical properties of the movement. In this paper we investigate further the role of the differential factors. Experiments are described in which drawing movements of simple geometrical forms and scribbles are performed either freely and extemporaneously, or in the presence of external constraints. It is shown that, at any time during the movement, the velocity component related to differential factors only depends on the value of the curvature of the trajectory at the same time (no dynamics). The relation can be described quantitatively as a specific Power Law and applies to all movements considered here, including those which are performed by following the edge of a template. The fact that the velocity of execution increases with the radius of curvature implies a built-in tendency of the motor control system to keep angular velocity relatively constant and qualifies the Isogony Principle proposed previously. The specific exponent of the Power Law suggests a possible interpretation of this empirical relation.

Viviani, P, & Schneider, R, 1991: A developmental study of the relationship between geometry and kinematics in drawing movements. J. Experim. Psychol. Hum. Percep. Perform., 17(2):198-218.

Trajectory and kinematics of drawing movements are mutually constrained by functional relationships that reduce the degrees of freedom of the hand-arm system. Previous investigations of these relationships are extended here by considering their development in children 5-12 yrs of age. Performances in a simple motor task, the continuous tracing of elliptic trajectories, demonstrate that both the phenomenon of isochrony (increase of the average movement velocity with the linear extent of the trajectory) and the so-called two -thirds power law (relation between tangential velocity and curvature) are qualitatively present already at the age of 5 yrs. The quantitative aspects of these regularities evolve with age, however, and steady-state adult performance is not attained even by the oldest children. The power-law formalism developed in previous reports is generalized to encompass these developmental aspects of the control of movement.

Miller, K J, Schalk, G, Fetz, E E, Nijs, M, Ojemann, J G, & Rao, R P N, 2010: Cortical activity during motor execution, motor imagery, and imagery-based online feedback. Proc. Nat'l Acad. Sci., 107(9):4430-4435.

Imagery of motor movement plays an important role in learning of complex motor skills, from learning to serve in tennis to perfecting a pirouette in ballet. What and where are the neural substrates that underlie motor imagery-based learning? We measured electrocorticographic cortical surface potentials in eight human subjects during overt action and kinesthetic imagery of the same movement, focusing on power in "high frequency" (76-100 Hz) and "low frequency" (8-32 Hz) ranges. We quantitatively establish that the spatial distribution of local neuronal population activity during motor imagery mimics the spatial distribution of activity during actual motor movement. By comparing responses to electrocortical stimulation with imagery -induced cortical surface activity, we demonstrate the role of primary motor areas in movement imagery. The magnitude of imagery-induced cortical activity change was ~25% of that associated with actual movement. However, when subjects learned to use this imagery to control a computer cursor in a simple feedback task, the imagery-induced activity change was significantly augmented, even exceeding that of overt movement.

Miller, K J, Schalk, G, Fetz, E E, Nijs, M, Ojemann, J G, & Rao, R P N, 2010: Cortical activity during motor execution, motor imagery, and imagery-based online feedback - Correction. Proc. Nat'l Acad. Sci., 107(15):7113.

Buonomano, D V, & Laje, R, 2010: Population clocks: motor timing with neural dynamics. Trends Cog. Sci., 14(12):520-527.

An understanding of sensory and motor processing will require elucidation of the mechanisms by which the brain tells time. Open questions relate to whether timing relies on dedicated or intrinsic mechanisms and whether distinct mechanisms underlie timing across scales and modalities. Although experimental and theoretical studies support the notion that neural circuits are intrinsically capable of sensory timing on short scales, few general models of motor timing have been proposed. For one class of models, population clocks, it is proposed that time is encoded in the time-varying patterns of activity of a population of neurons. We argue that population clocks emerge from the internal dynamics of recurrently connected networks, are biologically realistic and account for many aspects of motor timing.

Population Coded Motor Basis Functions

Shadmehr, R, Donchin, O, Hwang, E-J, Hemminger, S E, & Rao, A, 2005: Learning dynamics of reaching. ed: Riehle, A, & Vaadia, E, Motor Cortex in Voluntary Movements: A distributed system for distributed function, 297-328, CRC Press.

When one moves their hand from one point to another, the brain guides the arm by relying on neural structures that estimate physical dynamics of the task. For example, if one is about to lift a bottle of milk that appears full rather than empty, the brain takes into account the subtle changes in the dynamics of the task and this is reflected in the altered motor commands. The neural structures that compute the task's dynamics are "internal models" that transform the desired motion into motor commands. Internal models are learned with practice and are a fundamental part of voluntary motor control. What do internal models compute, and which neural structures perform that computation? We approach these problems by considering a task where physical dynamics of reaching movements are altered by force fields that act on the hand. Experiments by a number of laboratories on this paradigm suggest that internal models are sensorimotor transformations that map a desired sensory state of the arm into an estimate of forces, i.e., a model of the inverse dynamics of the task. If this computation is represented as a population code via a flexible combination of basis functions, then one can infer activity fields of the bases from the patterns of generalization. We provide a mathematical technique that facilitates this inference by analyzing trial-to-trial changes in performance. Results suggest that internal models are computed with bases that are directionally tuned to limb motion in intrinsic coordinates of joints and muscles, and this tuning is modulated multiplicatively as a function of static position of the limb. That is, limb position acts as a gain field on directional tuning. Some of these properties are consistent with activity fields of neurons in the motor cortex and the cerebellum. We suggest that activity fields of these cells are reflected in human behavior in the way that we learn and generalize patterns of dynamics in reaching movements.

Thoroughman, K A, & Taylor, J A, 2005: Rapid reshaping of human motor generalization. J. Neurosci., 25(39):8948-8953.

People routinely learn how to manipulate new tools or make new movements. This learning requires the transformation of sensed movement error into updates of predictive neural control. Here, we demonstrate that the richness of motor training determines not only what we learn but how we learn. Human subjects made reaching movements while holding a robotic arm whose perturbing forces changed directions at the same rate, twice as fast, or four times as fast as the direction of movement, therefore exposing subjects to environments of increasing complexity across movement space. Subjects learned all three environments and learned the low- and medium-complexity environments equally well. We found that subjects lessened their movement-by-movement adaptation and narrowed the spatial extent of generalization to match the environmental complexity. This result demonstrated that people can rapidly reshape the transformation of sense into motor prediction to best learn a new movement task. We then modeled this adaptation using a neural network and found that, to mimic human behavior, the modeled neuronal tuning of movement space needed to narrow and reduce gain with increased environmental complexity. Prominent theories of neural computation have hypothesized that neuronal tuning of space, which determines generalization, should remained fixed during learning so that a combination of neuronal outputs can underlie adaptation simply and flexibly. Here, we challenge those theories with evidence that the neuronal tuning of movement space changed within minutes of training.

Thoroughman, K A, & Shadmehr, R, 1999: Electromyographic correlates of learning internal models of reaching movements. J. Neurosci., 19(19):8573-8588.

Theoretical and psychophysical studies have suggested that humans learn to make reaching movements in novel dynamic environments by building specific internal models (IMs). Here we have found electromyographic correlates of internal model formation. We recorded EMG from four muscles as subjects learned to move a manipulandum that created systematic forces (a "force field"). We also simulated a biomechanical controller, which generated movements based on an adaptive IM of the inverse dynamics of the human arm and the manipulandum. The simulation defined two metrics of muscle activation. The first metric measured the component of the EMG of each muscle that counteracted the force field. We found that early in training, the field-appropriate EMG was driven by an error feedback signal. As subjects practiced, the peak of the field-appropriate EMG shifted temporally to earlier in the movement, becoming a feedforward command. The gradual temporal shift suggests that the CNS may use the delayed error -feedback response, which was likely to have been generated through spinal reflex circuits, as a template to learn a predictive feedforward response. The second metric quantified formation of the IM through changes in the directional bias of each muscle's spatial EMG function, i.e., EMG as a function of movement direction. As subjects practiced, co-activation decreased, and the directional bias of each muscle's EMG function gradually rotated by an amount that was specific to the field being learned. This demonstrates that formation of an IM can be represented through rotations in the spatial tuning of muscle EMG functions. Combined with other recent work linking spatial tunings of EMG and motor cortical cells, these results suggest that rotations in motor cortical tuning functions could underlie representation of internal models in the CNS.

Hwang, E J, Donchin, O, Smith, M A, & Shadmehr, R, 2003: A gain-field encoding of limb position and velocity in the internal model of arm dynamics. PLoS Biol., 1:209-220.

Adaptability of reaching movements depends on a computation in the brain that transforms sensory cues, such as those that indicate the position and velocity of the arm, into motor commands. Theoretical consideration shows that the encoding properties of neural elements implementing this transformation dictate how errors should generalize from one limb position and velocity to another. To estimate how sensory cues are encoded by these neural elements, we designed experiments that quantified spatial generalization in environments where forces depended on both position and velocity of the limb. The patterns of error generalization suggest that the neural elements that compute the transformation encode limb position and velocity in intrinsic coordinates via a gain-field; i.e., the elements have directionally dependent tuning that is modulated monotonically with limb position. The gain-field encoding makes the counterintuitive prediction of hypergeneralization: there should be growing extrapolation beyond the trained workspace. Furthermore, nonmonotonic force patterns should be more difficult to learn than monotonic ones. We confirmed these predictions experimentally.

Criscimagna-Hemminger, S E, Donchin, O, Gazzaniga, M S, & Shadmehr, R, 2003: Learned dynamics of reaching movements generalize from dominant to nondominant arm. J. Neurophysiol., 89(1):168-176.

Accurate performance of reaching movements depends on adaptable neural circuitry that learns to predict forces and compensate for limb dynamics. In earlier experiments, we quantified generalization from training at one arm position to another position. The generalization patterns suggested that neural elements learning to predict forces coded a limb's state in an intrinsic, muscle-like coordinate system. Here, we test the sensitivity of these elements to the other arm by quantifying inter-arm generalization. We considered two possible coordinate systems: an intrinsic (joint) representation should generalize with mirror symmetry reflecting the joint's symmetry and an extrinsic representation should preserve the task's structure in extrinsic coordinates. Both coordinate systems of generalization were compared with a naïve control group. We tested transfer in right-handed subjects both from dominant to nondominant arm (D->ND) and vice versa (ND->D). This led to a 2 × 3 experimental design matrix: transfer direction (D->ND/ND->D) by coordinate system (extrinsic, intrinsic, control). Generalization occurred only from dominant to nondominant arm and only in extrinsic coordinates. To assess the dependence of generalization on callosal inter -hemispheric communication, we tested commissurotomy patient JW. JWshowed generalization from dominant to nondominant arm in extrinsic coordinates. The results suggest that when the dominant right arm is used in learning dynamics, the information could be represented in the left hemisphere with neural elements tuned to both the right arm and the left arm. In contrast, learning with the nondominant arm seems to rely on the elements in the nondominant hemisphere tuned only to movements of that arm.

Thoroughman, K A, & Shadmehr, R, 2000: Learning of action through adaptive combination of motor primitives. Nature, 407(6805):742-747.

Understanding how the brain constructs movements remains a fundamental challenge in neuroscience. The brain may control complex movements through flexible combination of motor primitives1, where each primitive is an element of computation in the sensorimotor map that transforms desired limb trajectories into motor commands. Theoretical studies have shown that a system's ability to learn action depends on the shape of its primitives2. Using a time-series analysis of error patterns, here we show that humans learn the dynamics of reaching movements through a flexible combination of primitives that have gaussianlike tuning functions encoding hand velocity. The wide tuning of the inferred primitives predicts limitations on the brain's ability to represent viscous dynamics. We find close agreement between the predicted limitations and the subjects' adaptation to new force fields. The mathematical properties of the derived primitives resemble the tuning curves of Purkinje cells in the cerebellum. The activity of these cells may encode primitives that underlie the learning of dynamics.

Donchin, O, & Shadmehr, R, 2002: Linking motor learning to function approximation: Learning in an unlearnable force field. ed: Dietterich, T G, Becker, S, & Ghahramani, Z, Adv. Neur. Infor. Proc. Sys., 14:197-203.

Reaching movements require the brain to generate motor commands that rely on an internal model of the task's dynamics. Here we consider the errors that subjects make early in their reaching trajectories to various targets as they learn an internal model. Using a framework from function approximation, we argue that the sequence of errors should reflect the process of gradient descent. If so, then the sequence of errors should obey hidden state transitions of a simple dynamical system. Fitting the system to human data, we find a surprisingly good fit accounting for 98% of the variance. This allows us to draw tentative conclusions about the basis elements used by the brain in transforming sensory space to motor commands. To test the robustness of the results, we estimate the shape of the basis elements under two conditions: in a traditional learning paradigm with a consistent force field, and in a random sequence of force fields where learning is not possible. Remarkably, we find that the basis remains invariant.

Donchin, O, Francis, J T, & Shadmehr, R, 2003: Quantifying generalization from trial-by-trial behavior of adaptive systems that learn with basis functions: theory and experiments in human motor control. J. Neurosci., 23(27):9032-9045.

During reaching movements, the brain's internal models map desired limb motion into predicted forces. When the forces in the task change, these models adapt. Adaptation is guided by generalization: errors in one movement influence prediction in other types of movement. If the mapping is accomplished with population coding, combining basis elements that encode different regions of movement space, then generalization can reveal the encoding of the basis elements. We present a theory that relates encoding to generalization using trial-by-trial changes in behavior during adaptation. We consider adaptation during reaching movements in various velocity-dependent force fields and quantify how errors generalize across direction. We find that the measurement of error is critical to the theory. A typical assumption in motor control is that error is the difference between a current trajectory and a desired trajectory (DJ) that does not change during adaptation. Under this assumption, in all force fields that we examined, including one in which force randomly changes from trial to trial, we found a bimodal generalization pattern, perhaps reflecting basis elements that encode direction bimodally. If the DJ was allowed to vary, bimodality was reduced or eliminated, but the generalization function accounted for nearly twice as much variance. We suggest, therefore, that basis elements representing the internal model of dynamics are sensitive to limb velocity with bimodal tuning; however, it is also possible that during adaptation the error metric itself adapts, which affects the implied shape of the basis elements.

Shadmehr, R, 2004: Generalization as a behavioral window to the neural mechanisms of learning internal models. Human Movement Science, 23(5):543-568.

In generating motor commands, the brain seems to rely on internal models that predict physical dynamics of the limb and the external world. How does the brain compute an internal model? Which neural structures are involved? We consider a task where a force field is applied to the hand, altering the physical dynamics of reaching. Behavioral measures suggest that as the brain adapts to the field, it maps desired sensory states of the arm into estimates of force. If this neural computation is performed via a population code, i. e., via a set of bases, then activity fields of the bases dictate a generalization function that uses errors experienced in a given state to influence performance in any other state. The patterns of generalization suggest that the bases have activity fields that are directionally tuned, but directional tuning may be bimodal. Limb positions as well as contextual cues multiplicatively modulate the gain of tuning. These properties are consistent with the activity fields of cells in the motor cortex and the cerebellum. We suggest that activity fields of cells in these motor regions dictate the way we represent internal models of limb dynamics.

Bhushan, N, & Shadmehr, R, 1999: Computational architecture of human adaptive control during learning of reaching movements in force fields. Biol. Cybern., 81(1):39-60.

Learning to make reaching movements in force fields was used as a paradigm to explore the system architecture of the biological adaptive controller. We compared the performance of a number of candidate control systems that acted on a model of the neuromuscular system of the human arm and asked how well the dynamics of the candidate system compared with the movement characteristics of 16 subjects. We found that control via a supra-spinal system that utilized an adaptive inverse model resulted in dynamics that were similar to that observed in our subjects, but lacked essential characteristics. These characteristics pointed to a different architecture where descending commands were influenced by an adaptive forward model. However, we found that control via a forward model alone also resulted in dynamics that did not match the behavior of the human arm. We considered a third control architecture where a forward model was used in conjunction with an inverse model and found that the resulting dynamics were remarkably similar to that observed in the experimental data. The essential property of this control architecture was that it predicted a complex pattern of near-discontinuities in hand trajectory in the novel force field. A nearly identical pattern was observed in our subjects, suggesting that generation of descending motor commands was likely through a control system architecture that included both adaptive forward and inverse models. We found that as subjects learned to make reaching movements, adaptation rates for the forward and inverse models could be independently estimated and the resulting changes in performance of subjects from movement to movement could be accurately accounted for. Results suggested that the adaptation of the forward model played a dominant role in the motor learning of subjects. After a period of consolidation, the rates of adaptation in the internal models were significantly larger than those observed before the memory had consolidated. This suggested that consolidation of motor memory coincided with freeing of certain computational resources for subsequent learning.

Goodbody, S J, & Wolpert, D M, 1998: Temporal and amplitude generalization in motor learning. J. Neurophysiol., 79(4):1825-1838.

A fundamental feature of human motor control is the ability to vary effortlessly over a substantial range, both the duration and amplitude of our movements. We used a three-dimensional robotic interface, which generated novel velocity dependent forces on the hand, to investigate how adaptation to these altered dynamics experienced only for movements at one temporal rate and amplitude generalizes to movements made at a different rate or amplitude. After subjects had learned to make a single point-to-point movement in a novel velocity-dependent force field, we examined the generalization of this learning to movements of both half the duration or twice the amplitude. Such movements explore a state-space not experienced during learning-any changes in behavior are due to generalization of the learning, the form of which was used to probe the intrinsic constraints on the motor control process. The generalization was assessed by determining the force field in which subjects produced kinematically normal movements. We found substantial generalization of the motor learning to the new movements supporting a nonlocal representation of the control process. Of the fields tested, the form of the generalization was best characterized by linear extrapolation in a state-space representation of the controller. Such an intrinsic constraint on the motor control process can facilitate the scaling of natural movements.

Neural Population Codes and Bayesian Inference

Pouget, A, Dayan, P, & Zemel, R, 2000: Information processing with population codes. Nat. Rev. Neurosci., 1(2):125-131.

Information is encoded in the brain by populations or clusters of cells, rather than by single cells. This encoding strategy is known as population coding. Here we review the standard use of population codes for encoding and decoding information, and consider how population codes can be used to support neural computations such as noise removal and nonlinear mapping. More radical ideas about how population codes may directly represent information about stimulus uncertainty are also discussed.

Ma, W J, Beck, J M, Latham, P E, & Pouget, A, 2006: Bayesian inference with probabilistic population codes. Nat. Neurosci., 9(11):1432-1438.

Recent psychophysical experiments indicate that humans perform near-optimal Bayesian inference in a wide variety of tasks, ranging from cue integration to decision making to motor control. This implies that neurons both represent probability distributions and combine those distributions according to a close approximation to Bayes' rule. At first sight, it would seem that the high variability in the responses of cortical neurons would make it difficult to implement such optimal statistical inference in cortical circuits. We argue that, in fact, this variability implies that populations of neurons automatically represent probability distributions over the stimulus, a type of code we call probabilistic population codes. Moreover, we demonstrate that the Poisson-like variability observed in cortex reduces a broad class of Bayesian inference to simple linear combinations of populations of neural activity. These results hold for arbitrary probability distributions over the stimulus, for tuning curves of arbitrary shape and for realistic neuronal variability.

Seung, H S, & Sompolinsky, H, 1993: Simple models for reading neuronal population codes. Proc. Nat'l Acad. Sci., 90(22):10749-10753.

In many neural systems, sensory information is distributed throughout a population of neurons. We study simple neural network models for extracting this information. The inputs to the networks are the stochastic responses of a population of sensory neurons tuned to directional stimuli. The performance of each network model in psychophysical tasks is compared with that of the optimal maximum likelihood procedure. As a model of direction estimation in two dimensions, we consider a linear network that computes a population vector. Its performance depends on the width of the population tuning curves and is maximal for width, which increases with the level of background activity. Although for narrowly tuned neurons the performance of the population vector is significantly inferior to that of maximum likelihood estimation, the difference between the two is small when the tuning is broad. For direction discrimination, we consider two models: a perceptron with fully adaptive weights and a network made by adding an adaptive second layer to the population vector network. We calculate the error rates of these networks after exhaustive training to a particular direction. By testing on the full range of possible directions, the extent of transfer of training to novel stimuli can be calculated. It is found that for threshold linear networks the transfer of perceptual learning is nonmonotonic. Although performance deteriorates away from the training stimulus, it peaks again at an intermediate angle. This nonmonotonicity provides an important psychophysical test of these models.

Berniker, M, & Körding, K P, 2011: Bayesian approaches to sensory integration for motor control. Wiley Interdisciplinary Reviews: Cognitive Science, in press Feb 2011.

The processing of sensory information is fundamental to the basic operation of the nervous system. Our nervous system uses this sensory information to gain knowledge of our bodies and the world around us. This knowledge is of great importance as it provides the coherent and accurate information necessary for successful motor control. Yet, all this knowledge is of an uncertain nature because we obtain information only through our noisy sensors. We are thus faced with the problem of integrating many uncertain pieces of information into estimates of the properties of our bodies and the surrounding world. Bayesian approaches to estimation formalize the problem of how this uncertain information should be integrated. Utilizing this approach, many studies make predictions that faithfully predict human sensorimotor behavior.

Körding, K P, & Wolpert, D M, 2004: Bayesian integration in sensorimotor learning. Nature, 427(6971):244-247.

When we learn a new motor skill, such as playing an approaching tennis ball, both our sensors and the task possess variability. Our sensors provide imperfect information about the ball's velocity, so we can only estimate it. Combining information from multiple modalities can reduce the error in this estimate. On a longer time scale, not all velocities are a priori equally probable, and over the course of a match there will be a probability distribution of velocities. According to Bayesian theory, an optimal estimate results from combining information about the distribution of velocities-the prior-with evidence from sensory feedback. As uncertainty increases, when playing in fog or at dusk, the system should increasingly rely on prior knowledge. To use a Bayesian strategy, the brain would need to represent the prior distribution and the level of uncertainty in the sensory feedback. Here we control the statistical variations of a new sensorimotor task and manipulate the uncertainty of the sensory feedback. We show that subjects internally represent both the statistical distribution of the task and their sensory uncertainty, combining them in a manner consistent with a performance-optimizing Bayesian process. The central nervous system therefore employs probabilistic models during sensorimotor learning.

Körding, K P, Ku, S P, & Wolpert, D M, 2004: Bayesian integration in force estimation. J Neurophysiol., 92(5):3161-3165.

When we interact with objects in the world, the forces we exert are finely tuned to the dynamics of the situation. As our sensors do not provide perfect knowledge about the environment, a key problem is how to estimate the appropriate forces. Two sources of information can be used to generate such an estimate: sensory inputs about the object and knowledge about previously experienced objects, termed prior information. Bayesian integration defines the way in which these two sources of information should be combined to produce an optimal estimate. To investigate whether subjects use such a strategy in force estimation, we designed a novel sensorimotor estimation task. We controlled the distribution of forces experienced over the course of an experiment thereby defining the prior. We show that subjects integrate sensory information with their prior experience to generate an estimate. Moreover, subjects could learn different prior distributions. These results suggest that the CNS uses Bayesian models when estimating force requirements.

Körding, K P, & Wolpert, D M, 2006: Bayesian decision theory in sensorimotor control. Trends Cognit. Sci., 10(7):319-326.

Action selection is a fundamental decision process for us, and depends on the state of both our body and the environment. Because signals in our sensory and motor systems are corrupted by variability or noise, the nervous system needs to estimate these states. To select an optimal action these state estimates need to be combined with knowledge of the potential costs or rewards of different action outcomes. We review recent studies that have investigated the mechanisms used by the nervous system to solve such estimation and decision problems, which show that human behaviour is close to that predicted by Bayesian Decision Theory. This theory defines optimal behaviour in a world characterized by uncertainty, and provides a coherent way of describing sensorimotor processes.

Poggio, T, 1990: A theory of how the brain might work. Cold Spring Harbor Symp. Quant. Biol., 55:899-910.

I wish to propose a quite speculative new version of the grandmother cell theory to explain how the brain, or parts of it, may work. In particular, I discuss how the visual system may learn to recognize 3-dimensional objects. The model would apply directly to the cortical cells involved in visual face recognition. I also outline the relationship of our theory to existing models of the cerebellum and of motor control. Specific biophysical mechanisms can be readily suggested as part of a basic type of neural circuitry that can learn to approximate multidimensional input/output mappings from sets of examples and that is expected to be replicated in differenr regions of the brain and across modalities. The theory predicts a specific type of population coding that represents an extension of schemes such as look-up tables. I conclude with some speculations about the trade-off between memory and computation and the evolution of intelligence.

Pouget, A, & Sejnowski, T J, 1997: Spatial transformations in the parietal cortex using basis functions. J. Cog. Neurosci., 9(2):222-237.

Sensorimotor transformations are nonlinear mappings of sensory inputs to motor responses. We explore here the possibility that the responses of single neurons in the parietal cortex serve as basis functions for these transformations. Basis function decomposition is a general method for approximating nonlinear functions that is computationally efficient and well suited for adaptive modification. In particular, the responses of single parietal neurons can be approximated by the product of a Gaussian function of retinal location and a sigmoid function of eye position, called a gain field. A large set of such functions forms a basis set that can be used to perform an arbitrary motor response through a direct projection. We compare this hypothesis with other approaches that are commonly used to model population codes, such as computational maps and vectorial representations. Neither of these alternatives can fully account for the responses of parietal neurons, and they are computationally less efficient for nonlinear transformations. Basis functions also have the advantage of not depending on any coordinate system or reference frame. As a consequence, the position of an object can be represented in multiple reference frames simultaneously, a property consistent with the behavior of hemineglect patients with lesions in the parietal cortex.

Ajemian, R, Bullock, D, & Grossberg, S, 2001: A model of movement coordinates in the motor cortex: posture-dependent changes in the gain and direction of single cell tuning curves. Cereb. Cortex, 11(12):1124-1135.

This article outlines a methodology for investigating the coordinate systems by which movement variables are encoded in the firing rates of individual motor cortical neurons. Recent neurophysiological experiments have probed the issue of underlying coordinates by examining how cellular preferred directions (as determined by the center-out task) change with posture. Several key experimental findings have resulted that constrain hypotheses about how motor cortical cells encode movement information. But while the significance of shifts in preferred direction is well known and widely accepted, posture-dependent changes in the depth of modulation of a cell's tuning curve - that is, gain changes - have not been similarly identified as a means of coordinate inference. This article develops a vector field framework in which the preferred direction and the gain of a cell's tuning curve are viewed as dual components of a unitary response vector. The formalism can be used to compute how each aspect of cell response covaries with posture as a function of the coordinate system in which a given cell is hypothesized to encode its movement information. Such an integrated approach leads to a model of motor cortical cell activity that codifies the following four observations: (i) cell activity correlates with hand movement direction; (ii) cell activity correlates with hand movement speed; (iii) preferred directions vary with posture; and (iv) the modulation depth of tuning curves varies with posture. Finally, the model suggests general methods for testing coordinate hypotheses at the single-cell level and simulates an example protocol for three possible coordinate systems: Cartesian spatial, shoulder-centered, and joint angle.

Tkacik, G, Prentice, J S, Balasubramanian, V, & Schneidman, E, 2010: Optimal population coding by noisy spiking neurons. Proc. Nat'l Acad. Sci., 107(32):14419-14424.

In retina and in cortical slice the collective response of spiking neural populations is well described by "maximum-entropy" models in which only pairs of neurons interact. We asked, how should such interactions be organized to maximize the amount of information represented in population responses? To this end, we extended the linear-nonlinear-Poisson model of single neural response to include pairwise interactions, yielding a stimulus-dependent, pairwise maximum-entropy model. We found that as we varied the noise level in single neurons and the distribution of network inputs, the optimal pairwise interactions smoothly interpolated to achieve network functions that are usually regarded as discrete-stimulus decorrelation, error correction, and independent encoding. These functions reflected a trade-off between efficient consumption of finite neural bandwidth and the use of redundancy to mitigate noise. Spontaneous activity in the optimal network reflected stimulus-induced activity patterns, and single-neuron response variability overestimated network noise. Our analysis suggests that rather than having a single coding principle hardwired in their architecture, networks in the brain should adapt their function to changing noise and stimulus correlations.

Zemel, R S, Huys, Q J M, Natarajan, R, & Dayan, P, 2004: Probabilistic computation in spiking populations. Advances in Neural Information Processing Systems (NIPS'04),, v.16.

As animals interact with their environments, they must constantly update estimates about their states. Bayesian models combine prior probabilities, a dynamical model and sensory evidence to update estimates optimally. These models are consistent with the results of many diverse psychophysical studies. However, little is known about the neural representation and manipulation of such Bayesian information, particularly in populations of spiking neurons. We consider this issue, suggesting a model based on standard neural architecture and activations. We illustrate the approach on a simple random walk example, and apply it to a sensorimotor integration task that provides a particularly compelling example of dynamic probabilistic computation.

Natarajan, R, Huys, Q J M, Dayan, P, & Zemel, R S, 2008: Encoding and decoding spikes for dynamic stimuli. Neural Computation,, 20(9):2325-2360.

Naturally occurring sensory stimuli are dynamic. In this letter, we consider how spiking neural populations might transmit information about continuous dynamic stimulus variables. The combination of simple encoders and temporal stimulus correlations leads to a code in which information is not readily available to downstream neurons. Here, we explore a complex encoder that is paired with a simple decoder that allows representation and manipulation of the dynamic information in neural systems. The encoder we present takes the form of a biologically plausible recurrent spiking neural network where the output population recodes its inputs to produce spikes that are independently decodeable. We show that this network can be learned in a supervised manner by a simple local learning rule.

Huys, Q J M, Zemel, R, Natarajan, R, & Dayan, P, 2007: Fast population coding. Neural Computation, 19(2):404-441.

Uncertainty coming from the noise in its neurons and the ill-posed nature of many tasks plagues neural computations. Maybe surprisingly, many studies show that the brain manipulates these forms of uncertainty in a probabilistically consistent and normative manner, and there is now a rich theoretical literature on the capabilities of populations of neurons to implement computations in the face of uncertainty. However, one major facet of uncertainty has received comparatively little attention: time. In a dynamic, rapidly changing world, data are only temporarily relevant. Here, we analyze the computational consequences of encoding stimulus trajectories in populations of neurons. For the most obvious, simple, instantaneous encoder, the correlations induced by natural, smooth stimuli engender a decoder that requires access to information that is nonlocal both in time and across neurons. This formally amounts to a ruinous representation. We show that there is an alternative encoder that is computationally and representationally powerful in which each spike contributes independent information; it is independently decodable, in other words. We suggest this as an appropriate foundation for understanding time-varying population codes. Furthermore, we show how adaptation to temporal stimulus statistics emerges directly from the demands of simple decoding.

Shinomoto, S, Sakai, Y, & Funahashi, S, 1999: The Ornstein-Uhlenbeck process does not reproduce spiking statistics of neurons in prefrontal cortex. Neural Computation, 11(4):935-951.

Cortical neurons of behaving animals generate irregular spike sequences. Recently, there has been a heated discussion about the origin of this irregularity. Softky and Koch (1993) pointed out the inability of standard single-neuron models to reproduce the irregularity of the observed spike sequences when the model parameters are chosen within a certain range that they consider to be plausible. Shadlen and Newsome (1994), on the other hand, demonstrated that a standard leaky integrate-and-fire model can reproduce the irregularity if the inhibition is balanced with the excitation. Motivated by this discussion, we attempted to determine whether the Ornstein-Uhlenbeck process, which is naturally derived from the leaky integration assumption, can in fact reproduce higher-order statistics of biological data. For this purpose, we consider actual neuronal spike sequences recorded from the monkey prefrontal cortex to calculate the higher-order statistics of the interspike intervals. Consistency of the data with the model is examined on the basis of the coefficient of variation and the skewness coefficient, which are, respectively, a measure of the spiking irregularity and a measure of the asymmetry of the interval distribution. It is found that the biological data are not consistent with the model if the model time constant assumes a value within a certain range believed to cover all reasonable values. This fact suggests that the leaky integrate-and-fire model with the assumption of uncorrelated inputs is not adequate to account for the spiking in at least some cortical neurons.

Shadlen, M N, & Newsome, W T, 1994: Noise, neural codes and cortical organization. Current Opinion in Neurobiology, 4(4):569-579.

Cortical circuitry must facilitate information transfer in accordance with a neural code. In this article we examine two candidate neural codes: information is represented in the spike rate of neurons, or information is represented in the precise timing of individual spikes. These codes can be distinguished by examining the physiological basis of the highly irregular interspike intervals typically observed in cerebral cortex. Recent advances in our understanding of cortical microcircuitry suggest that the timing of neuronal spikes conveys little, if any, information. The cortex is likely to propagate a noisy rate code through redundant, patchy interconnections.

Bayesian Inference in Spiking Neural Networks

Perfors, A, Tenenbaum, J B, Griffiths, T L, & Xu, F, 2011: A tutorial introduction to Bayesian models of cognitive development. Cognition, in press March 2011.

We present an introduction to Bayesian inference as it is used in probabilistic models of cognitive development. Our goal is to provide an intuitive and accessible guide to the what, the how, and the why of the Bayesian approach: what sorts of problems and data the framework is most relevant for, and how and why it may be useful for developmentalists. We emphasize a qualitative understanding of Bayesian inference, but also include information about additional resources for those interested in the cognitive science applications, mathematical foundations, or machine learning details in more depth. In addition, we discuss some important interpretation issues that often arise when evaluating Bayesian models in cognitive science.

Rao, R P N, 2007: Neural models of Bayesian belief propagation. ed: Doya, K, Ishii, S, Pouget, A, & Rao, R P N, The Bayesian Brain: Probabilistic Approaches to Neural Coding, 235-264, MIT Press.

In this chapter, we describe how neural circuits could implement a general algorithm for Bayesian inference known as belief propagation. The belief propagation algorithm involves passing "messages" (probabilities) between the nodes of a graphical model that captures the causal structure of the environment. We review the basic notion of graphical models and illustrate the belief propagation algorithm with an example. We investigate potential neural implementations of the algorithm based on networks of leaky integrator neurons and describe how such networks can perform sequential and hierarchical Bayesian inference. Simulation results are presented for comparison with neurobiological data. We conclude the chapter by discussing other recent models of inference in neural circuits and suggest directions for future research. Some of the ideas reviewed in this chapter have appeared in prior publications [30, 31, 32, 42]; these may be consulted for additional details and results not included in this chapter.

Rao, R P N, 2005: Hierarchical Bayesian inference in networks of spiking neurons. Advances in Neural Information Processing Systems (NIPS'05), v.17.

There is growing evidence from psychophysical and neurophysiological studies that the brain utilizes Bayesian principles for inference and decision making. An important open question is how Bayesian inference for arbitrary graphical models can be implemented in networks of spiking neurons. In this paper, we show that recurrent networks of noisy integrate-and-fire neurons can perform approximate Bayesian inference for dynamic and hierarchical graphical models. The membrane potential dynamics of neurons is used to implement belief propagation in the log domain. The spiking probability of a neuron is shown to approximate the posterior probability of the preferred state encoded by the neuron, given past inputs. We illustrate the model using two examples: (1) a motion detection network in which the spiking probability of a direction-selective neuron becomes proportional to the posterior probability of motion in a preferred direction, and (2) a two-level hierarchical network that produces attentional effects similar to those observed in visual cortical areas V2 and V4. The hierarchical model offers a new Bayesian interpretation of attentional modulation in V2 and V4.

Rao, R P N, 2005: Bayesian inference and attention in the visual cortex. Neuroreport, 16(16):1843-1848.

The responses of neurons in cortical areas V2 and V4 can be significantly modulated by attention to particular locations within an input image. We show that such e¡ects emerge naturally when perception is viewed as a probabilistic inference process governed by Bayesian principles and implemented in hierarchical cortical networks. The proposed model can explain a rich variety of attention-related responses in cortical area V4 including multiplicative modulation of tuning curves, restoration of neural responses in the presence of distracting stimuli, and influence of attention on neighboring unattended locations. Our results suggest a new interpretation of attention as a cortical mechanism for reducing perceptual uncertainty by combining top-down task-relevant information with bottom-up sensory inputs in a probabilistic manner.

Rao, R P N, 2004: Bayesian computation in recurrent neural circuits. Neural Computation, 16(1):1-38.

A large number of human psychophysical results have been successfully explained in recent years using Bayesian models. However, the neural implementation of such models remains largely unclear. In this paper, we show that a network architecture commonly used to model the cerebral cortex can implement Bayesian inference for an arbitrary hidden Markov model. We illustrate the approach using an orientation discrimination task and a visual motion detection task. In the case of orientation discrimination, we show that the model network can infer the posterior distribution over orientations and correctly estimate stimulus orientation in the presence of significant noise. In the case of motion detection, we show that the resulting model network exhibits direction selectivity and correctly computes the posterior probabilities over motion direction and position. When used to solve the well-known random dots motion discrimination task, the model generates responses that mimic the activities of evidence-accumulating neurons in cortical areas LIP and FEF. The framework introduced in the paper posits a new interpretation of cortical activities in terms of log posterior probabilities of stimuli occurring in the natural world.

Bayesian Action Selection and Reinforcement Learning

Rao, R P N, 2010: Decision making under uncertainty: a neural model based on partially observable Markov decision processes. Front. Comput. Neurosci., 4:146.

A fundamental problem faced by animals is learning to select actions based on noisy sensory information and incomplete knowledge of the world. It has been suggested that the brain engages in Bayesian inference during perception but how such probabilistic representations are used to select actions has remained unclear. Here we propose a neural model of action selection and decision making based on the theory of partially observable Markov decision processes (POMDPs). Actions are selected based not on a single "optimal" estimate of state but on the posterior distribution over states (the "belief" state). We show how such a model provides a unified framework for explaining experimental results in decision making that involve both information gathering and overt actions. The model utilizes temporal difference (TD) learning for maximizing expected reward. The resulting neural architecture posits an active role for the neocortex in belief computation while ascribing a role to the basal ganglia in belief representation, value computation, and action selection. When applied to the random dots motion discrimination task, model neurons representing belief exhibit responses similar to those of LIP neurons in primate neocortex. The appropriate threshold for switching from information gathering to overt actions emerges naturally during reward maximization. Additionally, the time course of reward prediction error in the model shares similarities with dopaminergic responses in the basal ganglia during the random dots task. For tasks with a deadline, the model learns a decision making strategy that changes with elapsed time, predicting a collapsing decision threshold consistent with some experimental studies. The model provides a new framework for understanding neural decision making and suggests an important role for interactions between the neocortex and the basal ganglia in learning the mapping between probabilistic sensory representations and actions that maximize rewards.

Rao, R P N, 2001: Spike timing dependent Hebbian plasticity as temporal difference learning. Neural Computation, 13(10):2221-2237.

A spike-timing-dependent Hebbian mechanism governs the plasticity of recurrent excitatory synapses in the neocortex: synapses that are activated a few milliseconds before a postsynaptic spike are potentiated, while those that are activated a few milliseconds after are depressed. We show that such amechanism can implement a form of temporal difference learning for prediction of input sequences. Using a biophysical model of a cortical neuron, we show that a temporal difference rule used in conjunction with dendritic backpropagating action potentials reproduces the temporally asymmetric window of Hebbian plasticity observed physiologically. Furthermore, the size and shape of the window vary with the distance of the synapse from the soma. Using a simple example, we show how a spike-timing-based temporal difference learning rule can allow a network of neocortical neurons to predict an input a few milliseconds before the input's expected arrival.

Rao, R P N, & Sejnowski, T J, 2003: Self-organizing neural systems based on predictive learning. Phil. Trans. Roy. Soc. A, 361(1807):1149-1175.

The ability to predict future events based on the past is an important attribute of organisms that engage in adaptive behaviour. One prominent computational method for learning to predict is called temporal -difference (TD) learning. It is so named because it uses the difference between successive predictions to learn to predict correctly. TD learning is well suited to modelling the biological phenomenon of conditioning, wherein an organism learns to predict a reward even though the reward may occur later in time. We review a model for conditioning in bees based on TD learning. The model illustrates how the TD-learning algorithm allows an organism to learn an appropriate sequence of actions leading up to a reward, based solely on reinforcement signals. The second part of the paper describes how TD learning can be used at the cellular level to model the recently discovered phenomenon of spike-timing-dependent plasticity. Using a biophysical model of a neocortical neuron, we demonstrate that the shape of the spike-timing-dependent learning windows found in biology can be interpreted as a form of TD learning occurring at the cellular level. We conclude by showing that such spike-based TD-learning mechanisms can produce direction selectivity in visual-motion-sensitive cells and can endow recurrent neocortical circuits with the powerful ability to predict their inputs at the millisecond time-scale.

Attentional Modulation of Synaptic Efficacy

Rao, R P N, & Ballard, D H, 2004: Probabilistic models of attention based on iconic representations and predictive coding. ed: Itti, L, Rees, G, & Tsotsos, J, Neurobiology of Attention, 553-561, Academic Press.

We describe two models of attention that utilize probabilistic principles to compute task-relevant variables. In the first model, objects and visual scenes are represented iconically using spatial filters at multiple scales. A maximum likelihood-based approach is used to compute the location of a target in a given scene. The eye movements generated by such a strategy are shown to be similar to human eye movement patterns elicited during visual search in naturalistic scenes. The second model is based on the statistical concept of predictive coding. It assumes that top-down feedback from higher cortical areas conveys predictions of expected activity at lower levels while the errors in prediction are conveyed through feedforward connections. The model explains how multiple objects in a scene can be recognized sequentially without an explicit spotlight of attention. An extension of the model provides an interpretation of object-based versus spatial attention in terms of interactions between "what" and "where" networks in the visual pathway.

Ahveninen, J, Hämäläinen, M, Jääskeläinen, I P, Ahlfors, S P, Huang, S, Lin, F-H, Raij, T, Sams, M, Vasios, C E, & Belliveau, J W: Attention-driven auditory cortex short-term plasticity helps segregate relevant sounds from noise. Proc. Nat'l Acad. Sci., 108(10):4182-4187.

How can we concentrate on relevant sounds in noisy environments? A "gain model" suggests that auditory attention simply amplifies relevant and suppresses irrelevant afferent inputs. However, it is unclear whether this suffices when attended and ignored features overlap to stimulate the same neuronal receptive fields. A "tuning model" suggests that, in addition to gain, attention modulates feature selectivity of auditory neurons. We recorded magnetoencephalography, EEG, and functional MRI (fMRI) while subjects attended to tones delivered to one ear and ignored opposite-ear inputs. The attended ear was switched every 30 s to quantify how quickly the effects evolve. To produce overlapping inputs, the tones were presented alone vs. during white-noise masking notch-filtered ±1/6 octaves around the tone center frequencies. Amplitude modulation (39 vs. 41 Hz in opposite ears) was applied for "frequency tagging" of attention effects on maskers. Noise masking reduced early (50-150 ms; N1) auditory responses to unattended tones. In support of the tuning model, selective attention canceled out this attenuating effect but did not modulate the gain of 50-150 ms activity to nonmasked tones or steady-state responses to the maskers themselves. These tuning effects originated at nonprimary auditory cortices, purportedly occupied by neurons that, without attention, have wider frequency tuning than ±1/6 octaves. The attentional tuning evolved rapidly, during the first few seconds after attention switching, and correlated with behavioral discrimination performance. In conclusion, a simple gain model alone cannot explain auditory selective attention. In nonprimary auditory cortices, attention-driven short-term plasticity retunes neurons to segregate relevant sounds from noise.

Task Switching

Pruszynski, J A, King, G L, Boisse, L, Scott, S H, Flanagan, J R, & Munoz, D P, 2010: Stimulus-locked responses on human arm muscles reveal a rapid neural pathway linking visual input to arm motor output. Eur. J. Neurosci., 32(6):1049-1057.

Previous studies have demonstrated that humans are sometimes capable of initiating arm movements towards visual stimuli at extremely short latencies, implying the presence of a short-latency neural pathway linking visual input to limb motor output. However, little is known about the neural mechanisms that underlie such hastened arm responses. One clue may come from recent demonstrations that the appearance of a visual target can elicit a rapid response in neck muscles that is time-locked to target appearance and functionally relevant for orienting gaze (head and eye) towards the target. Because oculomotor structures thought to contribute to 'visual responses' on neck muscles also target some arm muscles via a tecto -reticulo-spinal pathway, we hypothesized that a similar visual response would be present in arm muscles. Our results were consistent with this hypothesis as we observed the presence of rapid arm muscle activity (< 100 ms latency) that was time-locked to target appearance and not movement onset. We further found that the visual response in arm muscles: (i) was present only when an immediate reach towards the target was required; (ii) had a magnitude that was predictive of reaction time; (iii) was tuned to target location in a manner appropriate for moving the arm towards the target; and (iv) was more prevalent in shoulder muscles than elbow muscles. These results provide evidence for a rapid neural pathway linking visual input to arm motor output and suggest the presence of a common neural mechanism for hastening eye, head and arm movements.

Cameron, I G M, Watanabe, M, Pari, G, & Munoz, D P, 2010: Executive impairment in Parkinson's disease: response automaticity and task switching. Neuropsychologia, 48(7):1948-1957.

Patients with Parkinson's disease (PD) show slowed movement initiation and can have deficits in executive function, leading to impairments in controlling involuntary behavior. This results in difficulties performing an antisaccade, which requires one to suppress an automatic eye movement (a prosaccade) to a visual stimulus, and execute a voluntary eye movement in the opposite direction. Antisaccade deficits are similar to those seen in task switching, whereby one is required to change a response after performing a different behavior. Both antisaccade (Hood et al., 2007) and task switching (Cools, Barker, Sahakian, & Robbins, 2001) deficits in PD have been attributed to fronto-basal ganglia (BG) dysfunction. Previously, we demonstrated with functional magnetic resonance imaging that BG circuitry is important to both task switching and voluntary saccade generation, as greater caudate activation was seen when healthy young adults first prepared a prosaccade, but then switched to an antisaccade (Cameron, Coe, et al., 2009). Therefore, we hypothesized that PD patients would have difficulty switching from one saccade response to the other, with particular impairment in switching from a pro to an antisaccade. Here, we not only confirmed this prediction, but also showed that PD patients performed better than controls in switching from an anti to a prosaccade. This suggests that task switching deficits in PD are particularly pronounced when more automatic behavior needs to be overridden with alternative behavior. We suggest that this occurs primarily at the level of establishing the appropriate task set, which is an internalized rule that governs how to respond.

Watanabe, M, & Munoz, D P, 2009: Neural correlates of conflict resolution between automatic and volitional actions by basal ganglia. Eur. J. Neurosci., 30(11):2165-2176.

A dominant basal ganglia (BG) model consists of two functionally opposite pathways: one facilitates motor output and the other suppresses it. Although this idea was originally proposed to account for motor deficits, it has been extended recently also to explain cognitive deficits. Here, we employed the antisaccade paradigm (look away from a stimulus) to address the role of the caudate nucleus, the main BG input stage where the two pathways diverge, in conflict resolution. Using single neuron recordings in awake monkeys, we identified the following three groups of neurons. The first group of neurons showed activity consistent with sensory-driven (automatic) saccades toward a contralateral visual stimulus. The second group of neurons showed activity consistent with internally driven (volitional) saccades toward the contralateral side regardless of stimulus locations. The third group of neurons showed similar firing characteristics with the second group of neurons, except that their preferred saccade direction was ipsilateral. The activity of the three groups of neurons was correlated with behavioral outcome. Based on these findings, we suggest the following hypothesis: the first and second groups of neurons encoding automatic and volitional saccades, respectively, might give rise to the facilitation (direct) pathway and promote saccades toward the opposite directions, which creates a response conflict. This conflict could be resolved by the third group of caudate neurons, which might give rise to the suppression (indirect) pathway and attenuate inappropriate saccade commands toward the stimulus.

Cameron, I G M, Coe, B C, Watanabe, M, Stroman, P W, & Munoz, D P: Role of the basal ganglia in switching a planned response. Eur. J. Neurosci., 29(12):2413-2425.

The ability to perform an appropriate response in the presence of competing alternatives is a critical facet of human behavioral control. This is especially important if a response is prepared for execution but then has to be changed suddenly. A popular hypothesis of basal ganglia (BG) function suggests that its direct and indirect pathways could provide a neural mechanism to rapidly switch from one planned response to an alternative. However, if one response is more dominant or 'automatic' than the other, the BG might have a different role depending on switch direction. We built upon the pro- and antisaccade tasks, two models of automatic and voluntary behavior, respectively, and investigated whether the BG are important for switching any planned response in general, or if they are more important for switching from a more automatic response to a response that is more difficult to perform. Subjects prepared either a pro- or antisaccade but then had to switch it unexpectedly on a subset of trials. The results revealed increased striatal activation for switching from a pro- to an antisaccade but this did not occur for switching from an anti- to a prosaccade. This activation pattern depended on the relative difficulty in switching, and it was distinct from frontal eye fields, an area shown to be more active for antisaccade trials than for prosaccade trials. This suggests that the BG are important for compensating for differences in response difficulty, facilitating the rapid switching of one response for another.

Cameron, I G M, Watanabe, M, & Munoz, D P, 2007: Contrasting instruction change with response change in task switching. Experim. Brain Res., 182(2):233-248.

Switching between two tasks results in switch costs, which are increased error rates and response times in comparison to repeating a task. Switch costs are attributed to a change in task set, which is the internalized rule of how to respond to a stimulus. However, it is not clear if this is because the instruction about which task to perform has changed, or because a programmed response has changed. We examined this question by changing the instruction about whether to perform a pro or an antisaccade to a stimulus, before or after the stimulus was presented. As a saccade response is specified by instruction plus stimulus position, changing the instruction after the stimulus was present resulted in a change in the specified response, whereas changing the instruction beforehand did not. Three experiments investigated; (i) if changing instruction alone or changing the specified response produced switch costs; (ii) if predictability of switching instruction influenced switch costs; and (iii) if predictability of stimulus position influenced switch costs. Regardless of instruction or stimulus predictability, switch costs for both pro and antisaccades consistently resulted if the specified response switched. This suggests that a pro or antisaccade motor program was automatically programmed based on a presented instruction and stimulus position. Therefore, the given physical information drove switch costs, even if subjects could predict a change in task. This study demonstrates that switch costs result if changing an instruction changes a programmed response.

Cognitive vs. Biomechanical Effects on Motor Performance

Loehr, J D, & Palmer, C, 2007: Cognitive and biomechanical influences in pianists' finger tapping. Experim. Brain Res., 178(4):518-528.

Movement sequences such as typing or tapping display important interactions among finger movements arising from anticipatory motion (preparing for upcoming events) and coupling (non-independence among fingers). We examined pianists' finger tapping for the influence of cognitive chunking processes and biomechanical coupling constraints. In a synchronization-continuation task, pianists repeatedly tapped four -finger sequences that differed in terms of the chunks that formed subsequences and in the transitions among physically adjacent or non-adjacent fingers. Chunking influenced intertap intervals, regardless of the particular fingers tapped; the final tap of each chunk was lengthened and less variable relative to other taps. The particular fingers tapped influenced peak finger heights, consistency of motion, and velocity -acceleration patterns, regardless of chunking. Thus, cognitive constraints influenced timing, whereas biomechanical factors influenced motion trajectories. These findings provide an important caveat for study of anticipatory motion by documenting the influence of biomechanical coupling on motion trajectories.

Loehr, J D, & Palmer, C, 2009: Sequential and biomechanical factors constrain timing and motion in tapping. J. Motor Behav., 41(2):128-136.

The authors examined how timing accuracy in tapping sequences is influenced by sequential effects of preceding finger movements and biomechanical interdependencies among fingers. Skilled pianists tapped sequences at 3 rates; in each sequence, a finger whose motion was more or less independent of other fingers' motion was preceded by a finger to which it was more or less coupled. Less independent fingers and those preceded by a more coupled finger showed large timing errors and change in motion because of the preceding finger's motion. Motion change correlated with shorter intertap intervals and increased with rate. Thus, timing of sequence elements is not independent of the motion trajectories that individuals use to produce them. Neither motion nor its relation to timing is invariant across rates.

Self Organization and Cognition - Order out of Chaos

McClelland, J L, 2010: Emergence in cognitive science. Topics Cog. Sci., 2:7510-770.

The study of human intelligence was once dominated by symbolic approaches, but over the last 30 years an alternative approach has arisen. Symbols and processes that operate on them are often seen today as approximate characterizations of the emergent consequences of sub- or nonsymbolic processes, and a wide range of constructs in cognitive science can be understood as emergents. These include representational constructs (units, structures, rules), architectural constructs (central executive, declarative memory), and developmental processes and outcomes (stages, sensitive periods, neurocognitive modules, developmental disorders). The greatest achievements of human cognition may be largely emergent phenomena. It remains a challenge for the future to learn more about how these greatest achievements arise and to emulate them in artificial systems.

McClelland, J L, Botvinick, M M, Noelle, D C, Plaut, D C, Rogers, T T, Seidenberg, M S, & Smith, L B, 2010: Letting structure emerge: connectionist and dynamical systems approaches to understanding cognition. Trends Cog. Sci., 14(8):348-356.

Connectionist and dynamical systems approaches explain human thought, language and behavior in terms of the emergent consequences of a large number of simple noncognitive processes. We view the entities that serve as the basis for structured probabilistic approaches as abstractions that are occasionally useful but often misleading: they have no real basis in the actual processes that give rise to linguistic and cognitive abilities or to the development of these abilities. Although structured probabilistic approaches can be useful in determining what would be optimal under certain assumptions, we propose that connectionist, dynamical systems, and related approaches, which focus on explaining the mechanisms that give rise to cognition, will be essential in achieving a full understanding of cognition and development.

Griffiths, T L, Chater, N, Kemp, C, Perfors, A, & Tenenbaum, J B, 2010: Probabilistic models of cognition: exploring representations and inductive biases. Trends Cog. Sci., 14(8):357-364.

Cognitive science aims to reverse-engineer the mind, and many of the engineering challenges the mind faces involve induction. The probabilistic approach to modeling cognition begins by identifying ideal solutions to these inductive problems. Mental processes are then modeled using algorithms for approximating these solutions, and neural processes are viewed as mechanisms for implementing these algorithms, with the result being a top-down analysis of cognition starting with the function of cognitive processes. Typical connectionist models, by contrast, follow a bottom-up approach, beginning with a characterization of neural mechanisms and exploring what macro-level functional phenomena might emerge. We argue that the top-down approach yields greater flexibility for exploring the representations and inductive biases that underlie human cognition.

Rogers, T T, & McClelland, J L, 2011: Semantics without categorization. ed: Pothos, E M, & Wills, A J, Formal approaches to categorization, Cambridge University Press.

One long-standing hypothesis places categorization at the heart of human semantic abilities. As Rosch (1978) put it, "...what one wishes to gain from one's categories is a great deal of information about the environment while conserving finite resources as much as possible." Indeed, the idea that semantic abilities are supported by categorization processes is so pervasive that it is seldom treated as a hypothesis. In the preceding quotation, for instance, Rosch inquires only what one wants from one's categories-as though the question of whether our semantic memory system actually employs category representations is itself beyond question.
We briefly summarize some of the challenges that have motivated our work. Multiple category representation: objects in the world usually belong simultaneously to many different categories. Category coherence: how does the semantic system "know" for which groupings of items it should form a stored category representation? Primacy of different category structures in development, maturity, and dissolution: why are more general category distinctions acquired earlier by preverbal infants, why are intermediate-level names learned earliest in development, why are they so robust in healthy adults,and why are the more general distinctions also more robust to semantic impairment? Domain-specific patterns of inductive projection: a new item cannot be categorized until one knows how to weight its properties, but because these weights depend upon the category, they cannot be determined until the item has been categorized.
One response to these challenges, of course, is the further development of categorization-based approaches to semantic knowledge. Our own work, however, is motivated by the observation that categorization is not the only efficient mechanism for storing and generalizing knowledge about the world. A tradition of work extending back to the distributed memory model of McClelland and Rumelhart (1986) and the semantic memory models of Rumelhart (Rumelhart, 1990; Rumelhart & Todd, 1993) and Hinton (1981; 1986) provides an alternative mechanism that, like categorization, is efficient in that it does not require addition of a new representational element for every new class or learning event, and that also promotes semantic generalization-all without requiring any internal process of categorization. The internal representations employed in our approach do not correspond to explicit, discrete representations of different classes of objects, and the processes that govern generalization of knowledge to new items do not involve the assignment of the item to one of a finite number of discrete equivalence classes.
Consequently, questions about which categories are stored in memory, or how the system determines which category to use when generalizing, are moot in this framework. Furthermore, our framework suggests potential mechanisms that account for the variety of phenomena summarized above, and also provides some clues as to how the semantic system may be organized in the brain. Thus we believe that our approach-semantics without categorization-can resolve many of the puzzles faced by categorization-based approaches to semantic knowledge.

Hinton, G E, McClelland, J L, & Rumelhart, D E, 1986: Distributed representations. ed: Rumelhart, D E, & McClelland, J L, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations, MIT Press.

Given a network of simple computing elements and some entities to be represented, the most straightforward scheme is to use one computing element for each entity. This is called a local representation. It is easy to understand and easy to implement because the structure of the physical network mirrors the structure of the knowledge it contains. The naturalness and simplicity of this relationship between the knowledge and the hardware that implements it have led many people to simply assume that local representations are the best way to use parallel hardware. There are, of course, a wide variety of more complicated implementations in which there is no one-to-one correspondence between concepts and hardware units, but these implementations are only worth considering if they lead to increased efficiency or interesting emergent properties that cannot be conveniently achieved using local representations.
This chapter describes one type of representation that is less familiar and harder to think about than local representations. Each entity is represented by a pattern of activity distributed over many computing elements, and each computing element is involved in representing many different entities. The strength of this more complicated kind of representation does not lie in its notational convenience or its ease of implementation in a conventional computer, but rather in the efficiency with which it makes use of the processing abilities of networks of simple, neuron-like computing elements.

Skarda, C A, & Freeman, W J, 1987: How brains make chaos in order to make sense of the world. Behavioral and Brain Sciences, 10(2):161-173.

Recent "connectionist" models provide a new explanatory alternative to the digital computer as a model for brain function. Evidence from our EEG research on the olfactory bulb suggests that the brain may indeed use computational mechanisms like those found in connectionist models. In the present paper we discuss our data and develop a model to describe the neural dynamics responsible for odor recognition and discrimination. The results indicate the existence of sensory- and motor-specific information in the spatial dimension of EEG activity and call for new physiological metaphors and techniques of analysis. Special emphasis is placed in our model on chaotic neural activity. We hypothesize that chaotic behavior serves as the essential ground state for the neural perceptual apparatus, and we propose a mechanism for acquiring new forms of patterned activity corresponding to new learned odors. Finally, some of the implications of our neural model for behavioral theories are briefly discussed. Our research, in concert with the connectionist work, encourages a reevaluation of explanatory models that are based only on the digital computer metaphor.

Kohonen, T, 2003: Self-organized maps of sensory events. Phil. Trans. Roy. Soc. A, 361(1807):1177-1186.

Over the years, many divergent meanings have been associated with the term 'self-organization', e.g. automatic creation of structured systems and optimization of parameters in adaptive learning. In this paper, we shall discuss a special type of data-driven self-organization, namely, automatic formation of ordered, compressed representations of sensory events. Such ordered and organized representations of an organism's experiences and environment exist in the nervous systems, where specific feature-sensitive information-processing functions are usually associated with these representations. As a matter of fact, three types of neuronal organization called 'brain maps' can be distinguished: sets of feature-sensitive cells, ordered projections between neuronal layers, and ordered maps of abstract features, respectively. The latter are most intriguing as they may also reflect quite abstract properties of the input data in an orderly fashion. It is proposed that such 'maps' are learned in a process that involves competition between sets of neural cells on common input data, and sensitization or tuning of the most strongly responding cells and their local neighbours to this input. While serving as a model for brain maps, the 'self-organizing map' principle has been used as an analytical tool in exploratory data analysis. In the latter, it has had practical applications ranging from industrial process control to marketing analyses, and from linguistics to bioinformatics.

J. Physiol. Special Issue on Neuro-Computation: From Sensorimotor Integration to Computational Frameworks

Grandjean, B, Hepp-Reymond, M, & Maier, M A, 2007: The functional role of different neural activation profiles during precision grip: An artificial neural network approach. J. Physiol. - Paris, 101(1-3):9-21.

A dynamic and recurrent artificial neural network was used to investigate the functional properties of firing patterns observed in the primary motor (M1) and the primary somatosensory (S1) cortex of the behaving monkey during control of precision grip force. In the behaving monkey it was found that neurons in M1 and in S1 increase their firing activity with increasing grip force, as do the intrinsic and extrinsic hand muscles implicated in the task. However, some neurons also decreased their activity as a function of increasing force. The functional implication of these latter neurons is not clear and has not been elucidated so far. In order to explore their functional implication, we therefore simulated patterns of neural activity in artificial neural networks that represent cortical, spinal and afferent neural populations and tested whether particular activity profiles would emerge as a function of the input and of the connectivity of these networks. The functional implication of units with emergent or imposed decreasing activity was then explored.
Decreasing patterns of activity in M1 units did not emerge from the networks. However, the same networks generated decreasing activity if imposed as target patterns. As indicated by the emerging weight space, M1 projection units with decreasing patterns are functionally less involved in driving alpha motoneurons than units with increasing profiles. Furthermore, these units did not provide significant fusimotor drive, whereas those with increasing profiles did. Fusimotor drive was a function of the (imposed) form of muscle spindle afferent activity: with gamma (fusimotor) drive, muscle spindle afferents provided signals other than muscle length (as observed experimentally). The network solutions thus predict a functional dichotomy between increasing and decreasing M1 neurons: the former primarily drive alpha and gamma motoneurons, the latter only weakly alpha motoneurons.

Blanc, J-L, & Coq, J-O, 2007: Coding processes involved in the cortical representation of complex tactile stimuli. J. Physiol. - Paris, 101(1-3):22-31.

To understand how information is coded in the primary somatosensory cortex (S1) we need to decipher the relationship between neural activity and tactile stimuli. Such a relationship can be formally measured by mutual information. The present study was designed to determine how S1 neuronal populations code for the multidimensional kinetic features (i.e. random, time-varying patterns of force) of complex tactile stimuli, applied at different locations of the rat forepaw. More precisely, the stimulus localization and feature extraction were analyzed as two independent processes, using both rate coding and temporal coding strategies. To model the process of stimulus kinetic feature extraction, multidimensional stimuli were projected onto lower dimensional subspace and then clustered according to their similarity. Different combinations of stimuli clustering were applied to differentiate each stimulus identification process. Information analyses show that both processes are synergistic, this synergy is enhanced within the temporal coding framework. The stimulus localization process is faster than the stimulus feature extraction process. The latter provides more information quantity with rate coding strategy, whereas the localization process maximizes the mutual information within the temporal coding framework. Therefore, combining mutual information analysis with robust clustering of complex stimuli provides a framework to study neural coding mechanisms related to complex stimuli discrimination.

Fix, J, Rougier, N, & Alexandre, F, 2007: From physiological principles to computational models of the cortex. J. Physiol. - Paris, 101(1-3):32-39.

Understanding the brain goes through the assimilation of an increasing amount of biological data going from single cell recording to brain imaging studies and behavioral analysis. The description of cognition at these three levels provides us with a grid of analysis that can be exploited for the design of computational models. Beyond data related to specific tasks to be emulated by models, each of these levels also lays emphasis on principles of computation that must be obeyed to really implement biologically inspired computations. Similarly, the advantages of such a joint approach are twofold: computational models are a powerful tool to experiment brain theories and assess them on the implementation of realistic tasks, such as visual search tasks. They are also a way to explore and exploit an original formalism of asynchronous, distributed and adaptive computations with such precious properties as self-organization, emergence, robustness and more generally abilities to cope with an intelligent interaction with the world. In this article, we first discuss three levels at which a cortical circuit might be observed to provide a modeler with sufficient information to design a computational model and illustrate this principle with an application to the control of visual attention.

Gervais, R, Buonviso, N, Martin, C, & Ravel, N, 2007: What do electrophysiological studies tell us about processing at the olfactory bulb level? J. Physiol. - Paris, 101(1-3):40-45.

Electrophysiological recordings performed in the mammalian olfactory bulb (OB) aimed at deciphering neural rules supporting neural representation of odors. In spite of a fairly large number of available data, no clear picture emerges yet in the mammalian OB. This paper summarizes some important findings and underlines the fact that difference in experimental conditions still represents a major limitation to the emergence of a synthetic view. More specifically, we examine to what extent the absence or the presence of anaesthetic influence OB neuronal responsiveness. In addition, we will see that recordings of either single cell activity or populational activity provide quite different pictures. As a result some experimental approaches provide data underlying sensory properties of OB neurons while others emphasize their capabilities of integrating incoming sensory information with attention, motivation and previous experience.

Perrinet, L U, & Masson, G S, 2007: Modeling spatial integration in the ocular following response using a probabilistic framework. J. Physiol. - Paris, 101(1-3):46-55.

The machinery behind the visual perception of motion and the subsequent sensori-motor transformation, such as in ocular following response (OFR), is confronted to uncertainties which are efficiently resolved in the primate's visual system. We may understand this response as an ideal observer in a probabilistic framework by using Bayesian theory [Weiss, Y., Simoncelli, E.P., Adelson, E.H., 2002. Motion illusions as optimal percepts. Nature Neuroscience, 5(6), 598-604, doi:10.1038/nn858] which we previously proved to be successfully adapted to model the OFR for different levels of noise with full field gratings. More recent experiments of OFR have used disk gratings and bipartite stimuli which are optimized to study the dynamics of center-surround integration. We quantified two main characteristics of the spatial integration of motion: (i) a finite optimal stimulus size for driving OFR, surrounded by an antagonistic modulation and (ii) a direction selective suppressive effect of the surround on the contrast gain control of the central stimuli [Barthélemy, F.V., Vanzetta, I., Masson, G.S., 2006. Behavioral receptive field for ocular following in humans: dynamics of spatial summation and center-surround interactions. Journal of Neurophysiology, (95), 3712-3726, doi:10.1152/jn.00112.2006]. Herein, we extended the ideal observer model to simulate the spatial integration of the different local motion cues within a probabilistic representation. We present analytical results which show that the hypothesis of independence of local measures can describe the spatial integration of the motion signal. Within this framework, we successfully accounted for the contrast gain control mechanisms observed in the behavioral data for center-surround stimuli. However, another inhibitory mechanism had to be added to account for suppressive effects of the surround.

Modolo, J, Mosekilde, E, & Beuter, A, 2007: New insights offered by a computational model of deep brain stimulation. J. Physiol. - Paris, 101(1-3):56-63.

Deep brain stimulation (DBS) is a standard neurosurgical procedure used to treat motor symptoms in about 5% of patients with Parkinson's disease (PD). Despite the indisputable success of this procedure, the biological mechanisms underlying the clinical benefits of DBS have not yet been fully elucidated. The paper starts with a brief review on the use of DBS to treat PD symptoms. The second section introduces a computational model based on the population density approach and the Izhikevich neuron model. We explain why this model is appropriate for investigating macroscopic network effects and exploring the physiological mechanisms which respond to this treatment strategy (i.e., DBS). Finally, we present new insights into the ways this computational model may help to elucidate the dynamic network effects produced in a cerebral structure when DBS is applied.

Montagnini, A, Mamassian, P, Perrinet, L, Castet, E, & Masson, G S, 2007: Bayesian modeling of dynamic motion integration. J. Physiol. - Paris, 101(1-3):64-77.

The quality of the representation of an object's motion is limited by the noise in the sensory input as well as by an intrinsic ambiguity due to the spatial limitation of the visual motion analyzers (aperture problem). Perceptual and oculomotor data demonstrate that motion processing of extended objects is initially dominated by the local 1D motion cues, related to the object's edges and orthogonal to them, whereas 2D information, related to terminators (or edge-endings), takes progressively over and leads to the final correct representation of global motion. A Bayesian framework accounting for the sensory noise and general expectancies for object velocities has proven successful in explaining several experimental findings concerning early motion processing [Weiss, Y., Adelson, E., 1998. Slow and smooth: a Bayesian theory for the combination of local motion signals in human vision. MIT Technical report, A.I. Memo 1624]. In particular, these models provide a qualitative account for the initial bias induced by the 1D motion cue. However, a complete functional model, encompassing the dynamical evolution of object motion perception, including the integration of different motion cues, is still lacking. Here we outline several experimental observations concerning human smooth pursuit of moving objects and more particularly the time course of its initiation phase, which reflects the ongoing motion integration process. In addition, we propose a recursive extension of the Bayesian model, motivated and constrained by our oculomotor data, to describe the dynamical integration of 1D and 2D motion information. We compare the model predictions for object motion tracking with human oculomotor recordings.

Touboul, J, & Faugeras, O, 2007: The spikes trains probability distributions: a stochastic calculus approach. J. Physiol. - Paris, 101(1-3):78-98.

We discuss the statistics of spikes trains for different types of integrate-and-fire neurons and different types of synaptic noise models. In contrast with the usual approaches in neuroscience, mainly based on statistical physics methods such as the Fokker-Planck equation or the mean-field theory, we chose the point of the view of the stochastic calculus theory to characterize neurons in noisy environments. We present four stochastic calculus techniques that can be used to find the probability distributions attached to the spikes trains. We illustrate the power of these techniques for four types of widely used neuron models. Despite the fact that these techniques are mathematically intricate we believe that they can be useful for answering questions in neuroscience that naturally arise from the variability of neuronal activity. For each technique we indicate its range of applicability and its limitations.

El Boustani, S, Pospischil, M, Rudolph-Lilith, M, & Destexhe, A, 2007: Activated cortical states: Experiments, analyses and models. J. Physiol. - Paris, 101(1-3):99-109.

In awake animals, the cerebral cortex displays an "activated" state, with distinct characteristics compared to other states like slow-wave sleep or anesthesia. These characteristics include a sustained depolarized membrane potential (Vm) and irregular firing activity. In the present paper, we evaluate our understanding of cortical activated states from a computational neuroscience point of view. We start by reviewing the electrophysiological characteristics of activated cortical states based on recordings and analysis performed in awake cat association cortex. These analyses show that cortical activity is characterized by an apparent Poisson-distributed stochastic dynamics, both at the single-cell and population levels, and that single cells display a high-conductance state dominated by inhibition. We next overview computational models of the "awake" cortex, and perform the same analyses as in the experiments. Many properties identified experimentally are indeed reproduced by models, such as depolarized Vm, irregular firing with apparent Poisson statistics, and the determinant role of inhibitory fluctuations on spiking. However, other features are not well reproduced, such as firing statistics and the conductance state of the membrane, suggesting that the network state displayed by models is not entirely correct. We also show how networks can approach a correct conductance state, suggesting ways by which future models will generate activity fully consistent with experimental data.

Brovelli, A, Coquelin, P-A, & Boussaoud, D, 2007: Estimating the hidden learning representations. J. Physiol. - Paris, 101(1-3):110-117.

Successful adaptation relies on the ability to learn the consequence of our actions in different environments. However, understanding the neural bases of this ability still represents one of the great challenges of system neuroscience. In fact, the neuronal plasticity changes occurring during learning cannot be fully controlled experimentally and their evolution is hidden. Our approach is to provide hypotheses about the structure and dynamics of the hidden plasticity changes using behavioral learning theory. In fact, behavioral models of animal learning provide testable predictions about the hidden learning representations by formalizing their relation with the observables of the experiment (stimuli, actions and outcomes). Thus, we can understand whether and how the predicted learning processes are represented at the neural level by estimating their evolution and correlating them with neural data. Here, we present a Bayesian model approach to estimate the evolution of the internal learning representations from the observations of the experiment (state estimation), and to identify the set of models' parameters (parameter estimation) and the class of behavioral model (model selection) that are most likely to have generated a given sequence of actions and outcomes. More precisely, we use Sequential Monte Carlo methods for state estimation and the maximum likelihood principle (MLP) for model selection and parameter estimation. We show that the method recovers simulated trajectories of learning sessions on a single-trial basis and provides predictions about the activity of different categories of neurons that should participate in the learning process. By correlating the estimated evolutions of the learning variables, we will be able to test the validity of different models of instrumental learning and possibly identify the neural bases of learning.

Viéville, T, Chemla, S, & Kornprobst, P, 2007: How do high-level specifications of the brain relate to variational approaches? J. Physiol. - Paris, 101(1-3):118-135.

High-level specification of how the brain represents and categorizes the causes of its sensory input allows to link "what is to be done" (perceptual task) with "how to do it" (neural network calculation). In this article, we describe how the variational framework, which encountered a large success in modeling computer vision tasks, has some interesting relationships, at a mesoscopic scale, with computational neuroscience. We focus on cortical map computations such that "what is to be done" can be represented as a variational approach, i.e., an optimization problem defined over a continuous functional space. In particular, generalizing some existing results, we show how a general variational approach can be solved by an analog neural network with a given architecture and conversely. Numerical experiments are provided as an illustration of this general framework, which is a promising framework for modeling macro-behaviors in computational neuroscience.

Siri, B, Quoy, M, Delord, B, Cessac, B, & Berry, H, 2007: Effects of Hebbian learning on the dynamics and structure of random networks with inhibitory and excitatory neurons. J. Physiol. - Paris, 101(1-3):136-148.

The aim of the present paper is to study the effects of Hebbian learning in random recurrent neural networks with biological connectivity, i.e. sparse connections and separate populations of excitatory and inhibitory neurons. We furthermore consider that the neuron dynamics may occur at a (shorter) time scale than synaptic plasticity and consider the possibility of learning rules with passive forgetting. We show that the application of such Hebbian learning leads to drastic changes in the network dynamics and structure. In particular, the learning rule contracts the norm of the weight matrix and yields a rapid decay of the dynamics complexity and entropy. In other words, the network is rewired by Hebbian learning into a new synaptic structure that emerges with learning on the basis of the correlations that progressively build up between neurons. We also observe that, within this emerging structure, the strongest synapses organize as a small-world network. The second effect of the decay of the weight matrix spectral radius consists in a rapid contraction of the spectral radius of the Jacobian matrix. This drives the system through the "edge of chaos" where sensitivity to the input pattern is maximal. Taken together, this scenario is remarkably predicted by theoretical arguments derived from dynamical systems and graph theory.

Anthropometrics and Biomechanics

Ivancevic, V G, 2010: Nonlinear complexity of human biodynamics engine. Nonlinear Dynamics, in press May 2010.

This paper reviews a nonlinear complexity within the Human Biodynamics Engine (HBE), a world-class human neuro-musculoskeletal simulator, developed at the Department of Defense, Australia. The HBE development is based on an anthropomorphic tree of Euclidean motion groups SE(3), with 270 active degrees of freedom, realistic muscular mechanics and hierarchical neural-like control. The HBE is formulated in the fashion of nonlinear dynamics/control of highly complex biophysical and robotic systems and developed for the purpose of neuro-musculoskeletal injury prediction. The following aspects of the HBE development are described: geometrical, dynamical, control, physiological, biomedical, AI, behavioral and complexity. Several simulation examples are provided.

Kinematic Data Acquisition and Analysis

Statistical Analysis

Page, Á, Mata, V, Hoyos, J V, & Porcar, R, 2007: Experimental determination of instantaneous screw axis in human motions. Error analysis. Mech. Mach. Theory, 42(4):429-441.

The location of the instantaneous screw axis (ISA) is essential in order to obtain useful kinematic models of the human body for applications such as prosthesis and orthoses design or even to help in disease diagnosis techniques. In this paper, dual vectors will be used to represent and operate with kinematic screws with the purpose of locating the instantaneous screw axes which characterize this instantaneous motion. A photogrammetry system based on markers will be used to obtain the experimental data from which the kinematic magnitudes will be obtained. A comprehensive analysis of the errors in the measurement of kinematic parameters has been developed, obtaining explicit expressions for them based on the number of markers and their distribution. Finally, the developed methodology has been applied to the experimental determination of the ISA during an alternative motion of flexion and extension of the back.

De Groote, F, De Laet, T, Jonkers, I, De Schutter, J, 2007: Bayesian filtering and smoothing techniques in human motion analysis. J. Biomech., 40(suppl. 2):S407.

During motion analysis the movement of the human body is derived from the position of surface mounted markers. The most important sources of measurement errors are soft tissue artifacts (STA) and noise. A generic model underlies the data processing. The degrees of freedom of the model are represented by the generalized coordinates q. During inverse kinematics, the traditional approach, an estimate of q is obtained by a nonlinear least squares fit between the model and the measurements of the markers. Since this approach estimates q at each time step separately, the a priori knowledge that human movement is smooth can inherently not be included. Further, numerical differentiation of q leads to exploding errors on q' and q" and will therefore influence the joint reaction moments and forces. In contrast Bayesian filtering and smoothing techniques, which are proposed in this abstract, allow us to use the knowledge about the smoothness of the movement and to estimate q' and q" along with q.

Teu, K K, & Kim, W, 2006: Estimation of the axis of a screw motion from noisy data - a new method based on Plücker lines. J. Biomech., 39(15):2857-2862.

The problems of estimating the motion and orientation parameters of a body segment from two n point-set patterns are analyzed using the Plücker coordinates of a line (Plücker lines). The aim is to find algorithms less complex than those in conventional use, and thus facilitating more accurate computation of the unknown parameters. All conventional techniques use point transformation to calculate the screw axis. In this paper, we present a novel technique that directly estimates the axis of a screw motion as a Plücker line. The Plücker line can be transformed via the dual-number coordinate transformation matrix. This method is compared with Schwartz and Rozumalski [2005. A new method for estimating joint parameters from motion data. Journal of Biomechanics 38, 107-116] in simulations of random measurement errors and systematic skin movements. Simulation results indicate that the methods based on Plücker lines (Plücker line method) are superior in terms of extremely good results in the determination of the screw axis direction and position as well as a concise derivation of mathematical statements. This investigation yielded practical results, which can be used to locate the axis of a screw motion in a noisy environment. Developing the dual transformation matrix (DTM) from noisy data and determining the screw axis from a given DTM is done in a manner analogous to that for handling simple rotations. A more robust approach to solve for the dual vector associated with DTM is also addressed by using the eigenvector and the singular value decomposition.

Ringer, M, & Lasenby, J, 2004: A procedure for automatically estimating model parameters in optical motion capture. Image & Vision Computing, 22(10):843-850.

Model-based optical motion capture systems require knowledge of the position of the markers relative to the underlying skeleton, the lengths of the skeleton's limbs, and which limb each marker is attached to. These model parameters are typically assumed and entered into the system manually, although techniques exist for calculating some of them, such as the position of the markers relative to the skeleton's joints. We present a fully automatic procedure for determining these model parameters. It tracks the 2D positions of the markers on the cameras' image planes and determines which markers lie on each limb before calculating the position of the underlying skeleton. The only assumption is that the skeleton consists of rigid limbs connected with ball joints. The proposed system is demonstrated on a number of real data examples and is shown to calculate good estimates of the model parameters in each.

Gamage, S S H U, & Lasenby, J, 2002: New least squares solutions for estimating the average centre of rotation and the axis of rotation. J. Biomech., 35(1):87-93.

A new method is proposed for estimating the parameters of ball joints, also known as spherical or revolute joints and hinge joints with a fixed axis of rotation. The method does not require manual adjustment of any optimisation parameters and produces closed form solutions. It is a least squares solution using the whole 3D motion data set. We do not assume strict rigidity but only that the markers maintain a constant distance from the centre or axis of rotation. This method is compared with other methods that use similar assumptions in the cases of random measurement errors, systematic skin movements and skin movements with random measurement noise. Simulation results indicate that the new method is superior in terms of the algorithm used, the closure of the solution, consistency and minimal manual parameter adjustment. The method can also be adapted to joints with translational movements.

Rivest, L P, 2001: A directional model for the statistical analysis of movement in three dimensions. Biometrika, 88(3):779-791.

Movement of an object in three dimensions involves rotation and translation. The data for analysis are the coordinates of landmarks on the object, recorded at several time points. Statistical models for describing the rotational component of the movement of an object are proposed in this work. Under the fixed-axis model, the object rotates around an axis that does not change in time. The angular position of the object is then characterised by its rotation axis and an angle giving the extent of the rotation of the object with respect to a reference point. More complicated models occur when the rotation axis varies in time. Under the fixed-angle model the quaternions for the time-varying orientations of the object are shown to lie in a two-dimensional great circle on the surface of the unit sphere in four dimensions. Simple estimators are given of the common rotation axis and of the angles characterising the time varying orientation of the object. A score statistic for testing the fit of the fixed-axis model is constructed and methods for handling the autocorrelation of the errors at neighbouring data points are provided. The analysis of data, collected by an OPTOTRACK camera system on a rotating forearm, illustrates the methodology presented in this paper.

Rancourt, D, Rivest, L P, & Asselin, J, 2000: Using orientation statistics to investigate variations in human kinematics. Applied Statistics, 49(1):81-94.

This paper applies orientation statistics to investigate variations in upper limb posture of human subjects drilling at six different locations on a vertical panel. Some of the drilling locations are kinematically equivalent in that the same posture could be used for these locations. Upper limb posture is measured by recording the co-ordinates of four markers attached to the subjects hand, forearm, arm and torso. A 3×3 rotation characterizes the relative orientation of one body segment with respect to another. Replicates are available since each subject drilled at the same location five times. Upper limb postures for the six drilling locations are compared by one-way analysis-of-variance tests for rotations. These tests rely on tangent space approximations at the estimated modal rotation of the sample. A parameterization of rotations in terms of unit quaternions simplifies the computations. The analysis detects significant differences in posture between all pairs of drilling locations. The smallest changes, less than 10° at all joints, are obtained for the kinematically equivalent pairs of locations. A short discussion of the biomechanical interpretation of these findings is presented.

Chèze, L, Fregly, B J, & Dimnet, J, 1998: Determination of joint functional axes from noisy marker data using the finite helical axis. Human Movement Sci., 17(1):1-15.

When video-based motion analysis systems are used to measure segmental kinematics, the finite helical displacement computed between two adjacent body segments in two successive positions i, i + 1 is often used to approximate the instantaneous joint movement. The measured trajectories of the external markers glued on the skin are very perturbed compared to the real displacement of the bony structure, and the inaccuracy in the measurement leads to stochastic errors in the position and direction of the finite helical axis of motion (FHA). As the errors associated with the FHA estimates are inversely proportional to the rotaion magnitude (Woltring, H.J., Huiskes, R., de Lange, A., 1983. Measurement error influence on helical axis accuracy in the description of 3D finite joint movement in biomechanics. In: Woo, S.L., Mates, R.E. (Eds.), Biomechanics symposium AMD 56 (FED 1), New York ASME, pp. 19-22), it is illusive to expect to assess the helical displacement between two neighbouring positions, and so to describe the joint evolution using FHA theory in such a context.
A quantification of the errors on the FHA parameters computed between two successive positions i, i + 1 is proposed in this paper, using a numerical simulation of the knee joint evolution during gait. This case has been chosen because previous studies (Cappozzo, A., Catani, F., Leardini, A., 1993. Skin movement artifacts in human movement photogrammetry. Proceedings of the 14th Congress of the International Society of Biomechanics. Paris, France, pp. 238-239) have experimentally described skin and soft tissues perturbations. The results obtained from this simulation lead to two conclusions: first, they confirm the relative FHA cannot be used to represent accurately the joint kinematics during a given movement; and second, they allow a prediction of the minimum joint displacement required in order to have a reliable determination of the helical axis.
The aim of this paper is not to present a new calculation method for the FHA, but to propose an alternative use of the FHA. It is generally assumed to describe a joint displacement using a sequence of rotations about three successive axes. In this case, the difficulty for clinical applications is to correctly locate these axes, in order that they coincide with the functional axes of the considered joint. If the FHA theory is used to determine the location and orientation of these functional axes from corresponding pure movement recording, then the results can be very accurate provided that the measured displacement between the two finite positions i and j be sufficient with respect to the perturbing noise. One consequence of this remark is that the rotation axis of the considered, joint may remain stable in the range of motion between these positions.
An example of this altenative use of the FHA is displayed in this paper which concerns the determination of the flexion/extension axis of the elbow joint. The elbow joint has been chosen for two reasons: first, it deals with a stable joint rotation axis and second, experimental data were available on a fleshless upper-limb on which the flexion/extension axis of the elbow was marked by an anatomy specialist.

Meskers, C G M, van der Helm, F C T, Rozendaal, L A, & Rozing, P M, 1997: In vivo estimation of the glenohumeral joint rotation center from scapular bony landmarks by linear regression. J. Biomech., 31(1):93-96.

In this paper, a method is described for in vivo prediction of the glenohumeral joint rotation center (GH-r), necessary for the construction of a humerus local coordinate system in shoulder kinematic studies. The three-dimensional positions of five scapula bony landmarks as well as a large number of data points on the surface of the glenoid and humeral head were collected at 36 sets of cadaver scapulae and adjacent humeri. The position of GH-r in each scapula was estimated by mathematically fitting spheres to the glenoid and humeral head. GH-r prediction from scapula geometry parameters by linear regression resulted in a RMSE between measured and predicted GH-r of 2.32 mm for the x-coordinate, 2.69 mm for the y-coordinate and 3 .04 mm for the z-coordinate. Application in vivo revealed a random humerus orientation error due to measurement inaccuracies of 1.35, 0.29 and 1.26° standard deviation per rotation angle. The estimated total humerus orientation error including the offset error due to the regression model inaccuracy was 2.86, 0.84 and 2.69° standard deviation. As these errors were about 15 and 20% of, respectively, the intra- and inter-subject variability of the humerus orientations measured, it is concluded that the method described in this paper allows for an adequate construction of a humerus local coordinate system.

Challis, J H, 1995: A procedure for determining rigid body transformation parameters. J. Biomech., 28(6):733-737.

For many biomechanical applications it is necessary to determine the parameters which describe the transformation of a rigid body from one reference frame to another. These parameters are a scaling factor, an attitude matrix, and a translation vector. The paper presents a new procedure for the determination of these parameters incorporating the work of Arun et al. [IEEE Trans. Pattern Anal. Machine Intell, 9, 698-700 (1987)] but expanding their analysis to allow for the determination of a scale factor, the scalar weighting of the least-squares problem, and the problem of obtaining the incorrect determinant when determining the attitude matrix. The procedure, which requires the coordinates of three or more noncollinear points, is based around the singular value decomposition, and provides a least-squares estimate of the rigid body transformation parameters. Examples are presented of the use of this procedure for determining the attitude of a rigid body, and for osteometric scaling. When used for osteometric scaling mirror transformations are possible, therefore a right-hand specimen can be scaled to the left-hand side of another specimen.

Kalman Filtering

LaViola Jr, J J, 2003: A comparison of unscented and extended Kalman filtering for estimating quaternion motion. Proc. 2003 American Control Conference, 2435-2440.

The unscented Kalman filter is a superior alternative to the extended Kalman filter for a variety of estimation and control problems. However, its effectiveness for improving human motion tracking for virtual reality applications in the presence of noisy data has been unexplored. In this paper, we present an empirical study comparing the performance of unscented and extended Kalman filtering for improving human head and hand tracking. Specifically, we examine human head and hand orientation motion signals, represented with quaternions, which are critical for correct viewing perspectives in virtual reality. Our experimental results and analysis indicate that unscented Kalman filtering performs equivalently with extended Kalman filtering. However, the additional computational overhead of the unscented Kalman filter and quasi-linear nature of the quaternion dynamics lead to the conclusion that the extended Kalman filter is a better choice for estimating quaternion motion in virtual reality applications.

Miners, B, 2001: Kalman filtering and prediction for hand tracking. Advanced DSP Project Report, University of Guelph.

A need exists in today's society for intuitive hand gesture interaction between human and machine. Although a non-invasive technique is desirable, using a magnetic system for hand tracking avoids current issues with noninvasive techniques and allows one to concentrate on other aspects of gesture interaction. Unfortunately, magnetic tracking systems are highly susceptible to electromagnetic interference and signal noise. Filtering this sensory data will improve the quality of the input for interpretation, along with providing a measure of error for subsequent processes in hand gesture interpretation. This project evaluates the suitability of the extended Kalman filter and unscented Kalman filter for use in filtering and prediction of the hand movement as measured through a magnetic tracking system.
A computer simulation was used to evaluate the performance of the extended and unscented Kalman filters for use in hand tracking. This simulation used a simplified model of the hand dynamics since an accurate model for human body dynamics is a large problem on its own, and a non-linear observation model was used in each of the filters. Process and measurement noise was introduced and the performance was analyzed using the average mean squared error. The unscented Kalman filter resulted in a lower average mean squared error with a lower estimated variance on average. The improvements observed through use of the unscented Kalman filter were more significant in highly non-linear or discontinuous scenarios.
The simplicity of implementing a complex process or measurement model is a definite advantage of the UKF over the EKF, however, this simplicity comes at the expense of CPU time. The Matlab implementation of the UKF required almost twice as much CPU time as the equivalent EKF implementation in this project. Future research includes evaluation of the algorithms using real-world test data and investigation of alternative process models that more accurately model hand movement. Alternative techniques that are potentially useful and should be investigated further include the covariance intersection technique and interacting multiple models Kalman filter.

Forward and Inverse Kinematics

Mihelj, M, 2006: Inverse kinematics of human arm based on multisensor data integration. J. Intelligent & Robotic Sys., 47(2):139-153.

The paper considers a technique for computation of the inverse kinematic model of the human arm. The approach is based on measurements of the hand position and orientation as well as acceleration and angular rate of the upper arm segment. A quaternion description of orientation is used to avoid singularities in representations with Euler angles. A Kalman filter is designed to integrate sensory data from three different types of sensors. The algorithm enables estimation of human arm posture, which can be used in trajectory planning for rehabilitation robots, evaluation of motion of patients with movement disorders, and generation of virtual reality environments.

Tolani, D, Goswami, A, & Badler, N I, 2000: Real-time inverse kinematics techniques for anthropomorphic limbs. Graphical Models, 62(4):353-388.

In this paper we develop a set of inverse kinematics algorithms suitable for an anthropomorphic arm or leg. We use a combination of analytical and numerical methods to solve generalized inverse kinematics problems including position, orientation, and aiming constraints. Our combination of analytical and numerical methods results in faster and more reliable algorithms than conventional inverse Jacobian and optimization-based techniques. Additionally, unlike conventional numerical algorithms, our methods allow the user to interactively explore all possible solutions using an intuitive set of parameters that define the redundancy of the system.

Regnier, S, Ouezdou, F B, & Bidaud, P, 1997: Distributed method for inverse kinematics of all serial manipulators. Mech. Mach. Theory, 32(7):855-868.

This paper deals with a new numerical method to solve the inverse kinematics of all serial, special or general, manipulators. This method uses new concept from Distributed Articial Inteligence, multi-agent systems which alows to distribute the resolution of this problem. This concept is used with a new formulation of the problem associated to each local frame. This iterative and distributed algorithm is able to find all solutions of the inverse kinematics for all kinds of manipulators (6R, 5R1P, 4R2P, 3R3P). Moreover, we show that this method can be applied to redundant manipulators.

Hardware Considerations

Dipietro, L, Sabatini, A M, & Dario, P, 2008: A survey of glove-based systems and their applications. IEEE Trans. Sys. Man Cyber. C: App. & Rev., 38(4):461-482.

Hand movement data acquisition is used in many engineering applications ranging from the analysis of gestures to the biomedical sciences. Glove-based systems represent one of the most important efforts aimed at acquiring hand movement data. While they have been around for over three decades, they keep attracting the interest of researchers from increasingly diverse fields. This paper surveys such glove systems and their applications. It also analyzes the characteristics of the devices, provides a road map of the evolution of the technology, and discusses limitations of current technology and trends at the frontiers of research. A foremost goal of this paper is to provide readers who are new to the area with a basis for understanding glove systems technology and how it can be applied, while offering specialists an updated picture of the breadth of applications in several engineering and biomedical sciences areas.

Welch, G, Allen, B D, Ilie, A, & Bishop, G, 2007: Measurement sample time optimization for human motion tracking/capture systems. Proc. 2007 IEEE Virtual Reality Conf. (VR 2007).

Many human motion tracking systems average, integrate, or correlate device samples over some non-zero period of time in order to produce a single system-level measurement. This is done to reduce the effects of device noise, or because one has no choice with some devices. The problem is that the target (user) is likely to be moving throughout the sample time, and this movement will in effect introduce additional "noise" (uncertainty) into the measurements. In this paper we introduce a method to optimize the device sampling time at the measurement level. The central idea is to determine the sampling time that maximizes device noise filtering while minimizing the impact of the expected target motion over the interval. We present the theory behind the approach, and both simulated and real-world examples.

Motion Estimation

Dubbelman, G, van der Mark, W, & Groen, F C A, 2008: Accurate and robust ego-motion estimation using expectation maximization. Proc. 2008 IEEE/RSJ Int'l Conf. Intelligent Robots & Systems (IROS 2008), 3914-3920.

A novel robust visual-odometry technique, called EM-SE(3) is presented and compared against using the random sample consensus (RANSAC) for ego-motion estimation. In this contribution, stereo-vision is used to generate a number of minimal-set motion hypothesis. By using EM-SE(3), which involves expectation maximization on a local linearization of the rigid-body motion group SE(3), a distinction can be made between inlier and outlier motion hypothesis. At the same time a robust mean motion as well as its associated uncertainty can be computed on the selected inlier motion hypothesis. The datasets used for evaluation consist of synthetic and large real-world urban scenes, including several independently moving objects. Using these data-sets, it will be shown that EM-SE(3) is both more accurate and more efficient than RANSAC.

Teulière, C, Marchand, E, & Eck, L, 2010: Using multiple hypothesis in model-based tracking. Proc. IEEE Int'l Conf. Robotics & Automation (ICRA 2010).

Classic registration methods for model-based tracking try to align the projected edges of a 3D model with the edges of the image. However, wrong matches at low level can make these methods fail. This paper presents a new approach allowing to retrieve multiple hypothesis on the camera pose from multiple low -level hypothesis. These hypothesis are integrated into a particle filtering framework to guide the particle set toward the peaks of the distribution. Experiments on simulated and real video sequences show the improvement in robustness of the resulting tracker.

Studies using the Flock of Birds

Wong, Y, Kim, W, & Ying, N, 2005: Passive motion characteristics of the talocrural and the subtalar joint by dual Euler angles. J. Biomech., 38(12):2480-2485.

The objective of this study is to validate previous descriptions of hindfoot kinematics using dual Euler angle methods in a passive cadaveric model. The dual Euler angle method was chosen so as to facilitate description of the translational and rotational movement occurring at both the ankle and subtalar joints. A non-metal experimental set-up was fabricated to generate motion in foot cadaver specimens. Three -dimensional kinematic data of the ankle joint complex was collected from ten knee-below foot cadaver specimens using a Flock of Birds electromagnetic tracking device. The data correlates well with previously published kinematic descriptions of the ankle subtalar joint complex. Both the ankle and subtalar joint show 6 degree of freedom motion and multiaxial characteristics.
The motions of the talocrural joint, the talocalcaneal joint, and the gross motion between the foot and the shank were analyzed. During dorsiflexion-plantarflexion the motion of the calcaneus with respect to the tibia occurs mainly at the ankle joint, with little motion at the subtalar joint. The subtalar joint contributes more than the ankle joint during inversion-eversion.

Ying, N, & Kim, W, Wong, Y, & Kam, B H, 2004: Analysis of passive motion characteristics of the ankle joint complex using dual Euler angle parameters. Clin. Biomech., 19(2):153-160.

Objective. To apply the dual Euler angles method to investigate the passive motion characteristics of the human ankle joint complex.
Design. Three-dimensional kinematic data of the ankle joint complex was collected from 10 knee-below foot cadaver specimens.
Background. Besides the Euler angles and screw axis methods, the dual Euler angles method has been proposed as an alternative approach to quantify general spatial human joint motion. The dual Euler angles method provides a way to combine rotational and translational joint motions and to interpret motions in Cartesian coordinate systems, which can avoid the problems caused by the use of the joint coordinate system due to non-orthogonality.
Methods. A non-metal experimental setup was fabricated to generate motion in foot cadaver specimens. The kinematic data during passive dorsiflexion-plantarflexion was measured using an electromagnetic tracking device.
Results. The kinematic coupling characteristics and the respective contribution of the ankle joint and the subtalar joint to the gross motion of the foot with respect to the shank were analyzed based on dual Euler angle parameters. The results obtained in this study are generally in agreement with the observations reported previously.
Conclusions. The dual Euler angles method is suitable for analyzing the motion characteristics of the ankle joint complex. The motion at the ankle joint complex involves rotations about and translations along three axes.
Relevance. Our finding using the dual Euler angle methods allows precise identification of the respective contributions of the subtalar joint and the ankle joint during the motion of ankle joint complex as a whole.

Ying, N, & Kim, W, 2002: Use of dual Euler angles to quantify the three-dimensional joint motion and its application to the ankle joint complex. J. Biomech., 35(12):1647-1657.

This paper presents a modified Euler angles method, dual Euler angles approach, to describe general spatial human joint motions. In dual Euler angles approach, the three-dimensional joint motion is considered as three successive screw motions with respect to the axes of the moving segment coordinate system; accordingly, the screw motion displacements are represented by dual Euler angles. The algorithm for calculating dual Euler angles from coordinates of markers on the moving segment is also provided in this study. As an example, the proposed method is applied to describe motions of ankle joint complex during dorsiflexion-plantarflexion. A Flock of Birds electromagnetic tracking device (FOB) was used to measure joint motion in vivo. Preliminary accuracy tests on a gimbal structure demonstrate that the mean errors of dual Euler angles evaluated by using source data from FOB are less than 1° for rotations and 1 mm for translations, respectively. Based on the pilot study, FOB is feasible for quantifying human joint motions using dual Euler angles approach.

Accuracy and Calibration of the Flock of Birds

Assink, N, Bergmana, G J D, Knoester, B, Winters, J C, & Dijkstra, P U, 2008: Assessment of the cervical range of motion over time, differences between results of the Flock of Birds and the EDI-320: A comparison between an electromagnetic tracking system and an electronic inclinometer. Manual Therapy, 13(5):450-455.

The objective of this study was to analyse cervical range of motion, assessed over time by means of a digital inclinometer (EDI-320) and a three-dimensional electromagnetic tracking device (Flock of Birds).
The maximum active cervical range of motion was assessed with two measurement devices in three sessions over time, with 6-week intervals. In total, 26 women and 24 men (mean age: 44.4, SD: 9.9) without known pathology of the cervical spine participated. Four movements were measured axial rotation with the cervical spine in a flexed and in an extended position, flexion-extension, and lateral bending.
The results showed that the factor time was significant for rotation in extension and rotation in flexion. The factor device was significant for all movements measured, and the interaction term between time and device was significant for all movements except rotation in extension.
The Flock of Birds measured significantly higher ranges of motion on all motions except for lateral bending. A substantial variation in cervical range of motion was observed over time (ranging from -5.6 to 8 .1) as well as between devices (ranging from -13.1 to 29.9).
Substantial and significant differences in cervical range of motion were found over time as well as differences between the Flock of Birds and the EDI-320.

Yoon, H-J, Zhao, K D, Rebellato, J, An, K-N, & Keller, E E, 2006: Kinematic study of the mandible using an electromagnetic tracking device and custom dental appliance: Introducing a new technique. J. Biomech., 39(12):2325-2330.

The purpose of this study is to introduce a new technique for recording the kinematics of the temporomandibular joint and incisors, using an electromagnetic tracking device and custom dental appliance. Five normal subjects took part in this kinematic study (4 females, 1 male, mean age of 34.8 years). Subjects' mandibular motion during maximal opening tasks were recorded on two different days and linear distance (LD) (i.e., the LD between the start and end position) and curvilinear path (CP) (i.e., the curvilinear distance along the curve between the start and end position) were calculated for the lower incisor landmark and both condyles in the sagittal plane (in mm). In the present study, the range of incisal movements (LD: 34.9 to 54.3 mm, CP: 36.5 to 60.3 mm) and that of condylar movements (LD: 7.5 to 25.3 mm, CP: 10.6 to 27.6 mm) in the sagittal plane during opening are in the normal range compared to the previous literature. The ability of subjects to reproduce the same motion between the two sessions was also calculated. Differences due to trial sessions and different repetitions within a session were negligible, indicating that the method can be used to assess changes between testing conditions in healthy subjects, and patients pre- and post-operatively.

Duck, T R, Ferreira, L M, King, G J W, & Johnson, J A, 2004: Assessment of screw displacement axis accuracy and repeatability for joint kinematic description using an electromagnetic tracking device. J. Biomech., 37(1):163-167.

Screw displacement axes (SDAs) have been employed to describe joint kinematics in biomechanical studies. Previous reports have investigated the accuracy of SDAs combining various motion analysis techniques and smoothing procedures. To our knowledge, no study has assessed SDA accuracy describing the relative movement between adjacent bodies with an electromagnetic tracking system. This is important, since in relative motion, neither body is fixed and consequently sensitivity to potential measurement errors from both bodies may be significant. Therefore, this study assessed the accuracy of SDAs for describing relative motion between two moving bodies. We analyzed numerical simulated data, and physical experimental data recorded using a precision jig and electromagnetic tracking device. The numerical simulations demonstrated SDA position accuracy (p=0.04) was superior for single compared to relative body motion, whereas orientation accuracy (p=0.2) was similar. Experimental data showed data-filtering (Butterworth filter) improved SDA position and orientation accuracies for rotation magnitudes smaller or equal to 5.0°, with no effect at larger rotation magnitudes (p<0.05). This suggests that in absence of a filter, SDAs should only be calculated at rotations of greater than 5.0°. For rotation magnitudes of 0.5° (5.0°) about the SDA, SDA position and orientation error measurements determined from filtered experimental data were 3.75±0.30 mm (3.31±0.21 mm), and 1.10±0.04° (1.04±0.03°), respectively. Experimental accuracy values describing the translation along and rotation about the SDA, were 0.06±0.00 mm and 0.09±0.01°, respectively. These small errors establish the capability of SDAs to detect small translations, and rotations. In conclusion, application of SDAs should be a useful tool for describing relative motion in joint kinematic studies.

Koerhuisa, C L, Winters, J C, van der Helm, F C T, & Hof, A L, 2003: Neck mobility measurement by means of the Flock of Birds electromagnetic tracking system. Clin. Biomech., 18(1):14-18.

Objective. To establish the accuracy and reliability of a six-degrees-of-freedom electromagnetic tracking device, the Flock of Birds, for measuring neck rotations and to identify the main sources of error.
Design. Ten human subjects made the same types of maximal neck rotation, both actively and passively: axial rotation in neutral position, from a flexed position and from an extended position, flexion /extension and lateral flexion. The same movements were mimicked in a 'dummy head' set-up.
Methods. One Flock of Birds receiver was mounted on the thorax, one on the head. By means of a third receiver, mounted on a stylus, bony landmarks on head and thorax were palpated. These served to define two anatomically based local coordinate systems, to which the rotations were referred.
Results. Measurements were accurate with a maximal measurement error of 2.5°. No significant difference between active and passive rotation was seen. The intra-subject variation was low within the same session, SD between 2° and 4°. Between sessions the variability was considerable, SD between 5° and 16°.
Conclusion. The Flock of Birds method is reliable and sufficiently precise. The variability in measured range of motion between sessions is a point of concern in interpreting follow-up studies in patients.
Relevance. A reduced range of neck motion is a major complaint in pathologies of the cervical spine or the shoulder. A method is described in which neck rotations are related to well-defined bony landmarks. In combined rotations, e.g. flexion combined with axial rotation, the measured range of motion can sometimes fluctuate strongly (up to 30°) between measurements, without apparent pathology.

LaScalza, S, Arico, J, & Hughes, R, 2003: Effect of metal and sampling rate on accuracy of Flock of Birds electromagnetic tracking system. J. Biomech., 36(1):141-144.

Electromagnetic tracking devices are used in many biomechanics applications. Previous studies have shown that metal located within the working field of direct current electromagnetic tracking devices produces significant errors. However, the effect of sampling rate on the errors produced in a metallic environment has never been studied. In this study, the accuracy of Ascension Technologies' Flock of Birds was evaluated at sampling rates of 20, 60, 100, and 140 Hz, in the presence of both aluminum and steel. Aluminum interference caused an increase in measurement error as the sampling rate increased. Conversely, steel interference caused a decrease in measurement error as the sampling rate increased. We concluded that the accuracy of the Flock of Birds tracking system can be optimized in the presence of metal by careful choice in sampling rate.

Poulin, F, & Amiot, L-P, 2002: Interference during the use of an electromagnetic tracking system under OR conditions. J. Biomech., 35(6):733-737.

Many computer-assisted surgery applications use electromagnetic tracking devices and several sources of interference may reduce the accuracy of this type of system in clinical situations. This study aims to quantify interference sources in an operating room (OR) and determine if their impact on the tracking system is excessive for applications requiring millimetric accuracy. Electromagnetic noise levels were measured in a controlled environment and compared with measurements in an OR. Errors generated by this noise remained below the 0.15 mm RMS level. OR equipment was also brought in proximity to the electromagnetic receivers and the errors generated by the ensuing interference were measured. Ferromagnetic and electrical devices can produce large interference (translation errors up to 8.4 mm RMS and rotation up to 166°). However, these devices can be identified and placed at sufficient distances to decrease the magnitude of their interference. In conclusion, in the absence of significant ferromagnetic or electromagnetic distortion caused by equipment often present in an OR, this electromagnetic tracking system provides valid relative measurements with millimetric accuracy to computer-assisted surgical applications. This distortion can be reduced by maximizing the distances to the interfering OR equipment and integrating noise-reducing algorithms in associated software.

van Ruijven, L J, Beek, M, Donker, E, & van Eijden, T M G J, 2000: The accuracy of joint surface models constructed from data obtained with an electromagnetic tracking device. J. Biomech., 33(8):1023-1028.

Electromagnetic tracking devices are widely used in biomechanics. In this article a method is evaluated to construct models of articular surfaces using an electromagnetic tracking device. First, the accuracy of the space tracker was examined and optimised. Then, from several joint surfaces random points were measured and eighth degree polynomials were fitted to these measurements. To check if the fit converged well, plots of cross sections of the model with corresponding data points were examined. The accuracy of the models was determined by comparing them with computed tomography data and by reproducibility tests. All the fits converged well to the data. The root mean square (RMS) error of the models varied from 0.07 to 0.18 mm, and was proportional to the size and complexity of the surface. This was mainly due to systematic errors made by the space tracker, which were also proportional to the size and complexity of the surface.

Meskers, C G M, Fraterman, H, van der Helm, F C T, Vermeulen, H M, & Rozing, P M, 1999: Calibration of the Flock of Birds electromagnetic tracking device and its application in shoulder motion studies. J. Biomech., 32(6):629-633.

In this paper the applicability in terms of measurement accuracy of the Flock of Birds six D.O.F. electromagnetic tracking device in shoulder research is investigated. Position measurements in a workspace of approximately 1 m3 were performed using a stylus. The andom error at the stylustip appeared to be 1.86, 1.98 and 2.54 mm for x-, y- and z-coordinate, respectively. The error caused by distortion of the magnetic field by metal in the concrete of especially the floor was 20.8, 22.2 and 20.4 mm for the x-, y- and z- coordinate, respectively. Calibration and leaving out the measurements closests to the floor lowered this error to 2.07, 2.38 and 2.35 mm. Orientation errors of the shoulder bones evolving from the measurement inaccuracy were estimated from repeated measurements of shoulder bony landmarks of ten subjects by means of the stylus. These errors were generally below 2°. This is lower than found for the same measurements using a spatial linkage digitizer. It is concluded that the Flock of Birds is a useful tool for shoulder kinematic studies.

Salvia, P, Le Pallec, J P, Baillon, R, David, J H, & Rooze, M, 1998: In vivo comparison between an instrumented spatial linkage and an electromagnetic tracking device during shoulder movement. J. Biomech., 31(Suppl. 1):16.

The aim of this study was to compare as a function of time shoulder rotation values obtained by two six degree of freedom measurement systems: a home-made instrumented spatial linkage (ISL) and a Flock of Birds (FOB), a magnetic tracking device using pulsed direct current magnetic field. The agreement between both devices for in plane motion was good. Most of the differences were included between mean ± 2.5°. R² was around 0.99. For out of plane motion, the relation grew poorer. To give clinical approach of agreement, the kinematics of both devices was not calculated relative to the same laboratory reference system but to separate ISL and FOB fixed reference systems. The poor out of plane results were probably due to misalignemnt of these reference systems.

Bull, A M J, & Amis, A A, 1997: Accuracy of an electromagnetic tracking device. J. Biomech., 30(8):857-858.

In a recent technical note Milne et al. [J. Biomech. 29(6):791-793] presented positional and rotational accuracy for Ascension Technology's Flock of Birds. Because of the increasing use of this device in biomechanics research, we would like to address some questions which come out of their work.
We were surprised at the errors of 1.8% of step size found in measuring position and chose to repeat - as closely as possible from the description - the positional accuracy tests. Our results are presented in Figs 2 and 4 - next to the data of Milne et al. in Figs 1 and 3. Two significant points can be made from the comparison: (1) the optimal operational range for measuring 25 mm steps found by Milne et al. (225 -640 mm) was confirmed by this work (217-723 mm), (2) the accuracy which we obtained when increasing the step size up to 152 mm was an order of magnitude better than in the work of Mime et al. (0.23% compared to 1.8% of the step size).
We hypothesise that a relevant factor in this comparison is the effect of transmitter power output. The Flock of Birds system maintains the strength of the magnetic field at the receiver by stepping up the transmitter power as the receiver becomes more remote, and we discovered that transients during this process affected the system adversely. This was overcome by using a power supply with a higher rating than that supplied by the vendors. A further point is that the data of Milne et al. shows a negative offset across the optimal operational range. This can be corrected by modifying the scaling factors employed in the vendor supplied software. Other factors may also be involved, such as electromagnetic environmental effects in the laboratory.
Because the use of multiple receivers is desirable in biomechanics research, the effect of using two and three receivers on the accuracy must also be assessed. The main point of this letter, however, is to show that the electromagnetic tracking device can be set-up to be an order of magnitude more accurate than has been suggested previously.

Milne, A D, Chess, D G, Johnson, J A, & King, G J W, 1996: Accuracy of an electromagnetic tracking device: A study of the optimal operating range and metal interference. J. Biomech., 29(6):791-793.

The positional and rotational accuracy of a direct-current magnetic tracking device commonly used in biomechanical investigations was evaluated. The effect of different metals was also studied to determine the possibility of interference induced by experimental test fixtures or orthopaedic implants within the working field. Positional and rotational data were evaluated for accuracy and resolution by comparing the device output to known motions as derived from a calibrated grid board or materials testing machine. The effect of different metals was evaluated by placing cylindrical metal samples at set locations throughout the working field and comparing the device readings before and after introducing each metal sample. Positional testing revealed an optimal operational range with the transmitter and receiver separation between 22.5 and 64.0 cm. Within this range the mean positional error was found to be 1.8% of the step size, and resolution was determined to be 0.25 mm. The mean rotational error over a 1-20° range was found to be 1 .6% of the rotational increment, with a rotational resolution of 0.1°. Of the metal alloys tested only mild steel produced significant interference, which was maximum when the sample was placed adjacent to the receiver. At this location the mild steel induced a positional difference of 5.26 cm and an angular difference of 9.75°. The device was found to be insensitive to commonly used orthopaedic alloys. In this study, the electromagnetic tracking device was found to have positional and rotational errors of less than 2%, when utilized within its optimal operating range. This accuracy combined with its insensitivity to orthopaedic alloys should make it suitable for a variety of musculoskeletal research investigations.

Anthropometric and Kinematic 3D Modeling

Hands and Fingers

Wan, H, Chen, F, & Han, X, 2009: A 4-layer flexible virtual hand model for haptic interaction. Proc. 2009 IEEE Int'l Conf. Virtual Environments, Human-Computer Interfaces & Measurement Systems (VECIMS 2009), 185-190.

Virtual hand interactions play key roles in virtual environments. The recent addition of force feedback to virtual reality simulations has enhanced their realism, especially when dexterous manipulation of virtual objects is concerned. In the past decades, much effort has been made on virtual hand modeling from the perspectives of computer animation and human computer interaction. However, much less attention is paid on haptic modeling of flexible virtual hand. In this paper, we propose a 4-layer flexible virtual hand model for virtual hand haptic interaction. The skin layer, kinematics layer, collision detection layer and haptic layer are integrated into a sophisticated virtual hand to simulate the human hand's natural anatomy in its appearance and motion, and to reflect the area contact feature of force feedback datagloves. The infrastructure and details of the flexible virtual hand model are discussed. Experimental results show that the proposed flexible virtual hand demonstrates good performance in virtual hand haptic applications.

van Nierop, O A, van der Helm, A, Overbeeke, K J, & Djajadiningrat, T J P, 2008: A natural human hand model. Visual Computer, 24(1):31-44.

We present a skeletal linked model of the human hand that has natural motion. We show how this can be achieved by introducing a new biology-based joint axis that simulates natural joint motion and a set of constraints that reduce an estimated 150 possible motions to twelve. The model is based on observation and literature.
To facilitate testing and evaluation, we present a simple low polygon count skin that can stretch and bulge. To evaluate we first introduce a hand-motion taxonomy in a two-dimensional parameter space based on tasks that are evolutionary linked to the environment. Second, we discuss and test the model.
The appendix shows motion sequences of the model and the real hand. Animations can be fetched from our website.

Rogers, M S, Barr, A B, Kasemsontitum, B, & Rempel, D M, 2008: A three-dimensional anthropometric solid model of the hand based on landmark measurements. Ergonomics, 51(4):511-526.

Hand anthropometry data are largely based on measurements of the hand in an outstretched hand posture and are, therefore, difficult to apply to tool gripping hand postures. The purpose of this project was to develop a representative, scalable hand model to be used with 3-D software drawing packages to aid in the ergonomic design of hand tools. Landmarks (66) on the palmar surface of the right hand of 100 subjects were digitised in four functional hand postures and, from these, 3-D surface models of a mean, 25th and 75th% hand were developed. The root mean square differences in hand length between the hand model and the digitised data for the 25th, 50th and 75th percentile hand were 11.4, 3.2 and 8.9 mm, respectively. The corresponding values for hand breadth were 2.0, 0.4 and 1.4 mm. There was good agreement between distances on the digitised hand and the hand model. The application of this research includes improved ergonomic hand tool design through the use of hand anthropometry reference values developed from the general population using grasping hand postures.

Dragulescu, D, Perdereau, V, Drouin, M, Ungureanu, L, & Menyhardt, K, 2007: 3D active workspace of human hand anatomical model.

Background: If the model of the human hand is created with accuracy by respecting the type of motion provided by each articulation and the dimensions of articulated bones, it can function as the real organ providing the same motions. Unfortunately, the human hand is hard to model due to its kinematical chains submitted to motion constraints. On the other hand, if an application does not impose a fine manipulation it is not necessary to create a model as complex as the human hand is. But always the hand model has to perform a certain space of motions in imposed workspace architecture no matter what the practical application does.
Methods: Based on Denavit-Hartenberg convention, we conceived the kinematical model of the human hand, having in mind the structure and the behavior of the natural model. We obtained the kinematical equations describing the motion of every fingertip with respect to the general coordinate system, placed on the wrist. For every joint variable, a range of motion was established. Dividing these joint variables to an appropriate number of intervals and connecting them, the complex surface bordering the active hand model workspace was obtained.
Results: Using MATLAB 7.0, the complex surface described by fingertips, when hand articulations are all simultaneously moving, was obtained. It can be seen that any point on surface has its own coordinates smaller than the maximum length of the middle finger in static position. Therefore, a sphere having the centre in the origin of the general coordinate system and the radius which equals this length covers the represented complex surface.
Conclusion: We propose a human hand model that represents a new solution compared to the existing ones. This model is capable to make special movements like power grip and dexterous manipulations. During them, the fingertips do not exceed the active workspace encapsulated by determined surfaces. The proposed kinematical model can help to choose which model joints could be eliminated in order to preserve only the motions important for a certain application. The study shows that all models, simplified or not, exhibit a pronounced similitude with the real hand motion, validated by the fingertips' computed trajectories. The results were used to design an artificial hand capable to make some of the hand's functions with a reduced set of degrees of freedom.

Lee, S-W, & Zhang, X, 2007: Biodynamic modeling, system identification, and variability of multi-finger movements. J. Biomech., 40(14):3215-3222.

A forward dynamic model of human multi-fingered hand movement is proposed. The model represents digits 2-5 in manipulative acts as a 12-degrees-of-freedom (DOF) system, driven by torque actuators at individual joints and controlled using a parsimonious proportional-derivative (PD) scheme. The control parameters as feedback gains along with an auxiliary parameter to modulate the joint torque magnitudes and cross -coupling can be empirically identified in an iterative procedure minimizing the discrepancy between the model-prediction and measurement. The procedure is guided and computationally accelerated by pre -knowledge of relations between the parameters and kinematic responses. An empirical test based on real grasping movement data showed that the model simulated the multi-finger movements with varied inter -joint temporal coordination accurately: the grand mean of the root-mean-square-errors (RMSE) across trials performed by 28 subjects was 3.25°. Analyses of the model parameters yielded new insights into intra- and inter-person variability in multi-finger movement performance, and distinguished the less variable motor control strategy from much more variable anthropometric and physiological factors.

Fu, A W, Keogh, E, Lau, L Y H, & Ratanamahatana, C A, 2005: Scaling and time warping in time series querying. Proc. 31st Int'l Conf. Very Large Data Dases, 649-660.

The last few years have seen an increasing understanding that Dynamic Time Warping (DTW), a technique that allows local flexibility in aligning time series, is superior to the ubiquitous Euclidean Distance for time series classification, clustering, and indexing. More recently, it has been shown that for some problems, Uniform Scaling (US), a technique that allows global scaling of time series, may just be as important for some problems. In this work, we note that for many real world problems, it is necessary to combine both DTW and US to achieve meaningful results. This is particularly true in domains where we must account for the natural variability of human action, including biometrics, query by humming, motion-capture/animation, and handwriting recognition. We introduce the first technique which can handle both DTW and US simultaneously, and demonstrate its utility and effectiveness on a wide range of problems in industry, medicine, and entertainment.

Lee, S-W, & Zhang, X, 2005: Development and validation of an optimization-based model for power-grip posture prediction. J. Biomech., 38(8):1591-1597.

An optimization-based model for power-grip posture prediction was proposed. The model was based on the premise that the hand prehensile configuration in a power grip best conforms to the object shape. This premise was embodied by an optimization procedure that minimized the sum of distances from the finger joints to the object surface. The model was evaluated against data from an experiment that measured the grasp postures of 28 subjects having diverse anthropometry. The intra- and inter-person variabilities in grip postures were empirically assessed and used as benchmark values for model evaluation. The evaluation showed that the root-mean-square (RMS) values of angle differences between the predicted and measured postures had a 13.7° grand mean (across all joints, subjects, and two cylindrical handles grasped), whereas the RMS values of the inter- and intra-person variabilities in measured postures had grand means of 13.0° and 4.4°, respectively. The model can be readily generalized to the prediction of postures in power-grasping objects of different shapes, and adapted for testing alternative prehensile strategies or performance criteria.

Tsang, W, Singh, K, & Fiume, E, 2005: Helping hand: an anatomically accurate inverse dynamics solution for unconstrained hand motion. Proc. 2005 ACM SIGGRAPH/Eurographics Symp. Computer Animation, 319-328.

We present a realistic skeletal musculo-tendon model of the human hand and forearm. The model permits direct forward dynamics simulation, which accurately predicts hand and finger position given a set of muscle activations. We also present a solution to the inverse problem of determining an optimal set of muscle activations to achieve a given pose or motion; muscle fatigue, injury or atrophy can also be specified, yielding different control solutions that favour healthy muscle. As there can be many (or no) solutions to this inverse problem, we demonstrate how the space of possible solutions can be filtered to an optimal representative. Of particular note is the ability of our model to take a wide array of joint interdependence into account for both forward and inverse problems. Given kinematic postures, the model can be used to validate, predict or fill in missing motion and improve coarsely specified motion with anatomic fidelity. Lastly, we address the visualization and understanding of the dynamically changing and spatially compact musculature using various interaction techniques.

Zhang, X, Lee, S-W, & Braido, P N, 2003: Determining finger segmental centers of rotation in flexion-extension based on surface marker measurement. J. Biomech., 36(8):1097-1102.

This paper describes the development of a novel algorithm for deriving finger segmental center of rotation (COR) locations during flexion-extension from measured surface marker motions in vivo. The algorithm employs an optimization routine minimizing the time-variance of the internal link lengths, and incorporates an empirically quantifiable relationship between the local movement of a surface marker around a joint (termed "surface marker excursion") and the joint flexion-extension. The latter relationship constrains and simplifies the optimization routine to make it computationally tractable. To empirically investigate this relationship and test the proposed algorithm, an experiment was conducted, in which hand cylinder-grasping movements were performed by 24 subjects (12 males and 12 females). Spherical retro-reflective markers were placed at various surface landmarks on the dorsal aspect of each subject's right (grasping) hand, and were measured during the movements by an opto-electronic system. Analysis of experimental data revealed a highly linear relationship between the "surface marker excursion" and the marker-defined flexion -extension angle: the average R2 in linear regression ranged from 0.89 to 0.97. The algorithm successfully determined the CORs of the distal interphalangeal, proximal interphalangeal, and metacarpophalangeal joints of digits 2-5 during measured motions. The derived CORs appeared plausible as examined in terms of the physical locations relative to surface marker trajectories and the congruency across different joints and individuals.

Albrecht, I, Haber, J, & Seidel, H-P, 2003: Construction and animation of anatomically based human hand models. Proc. 2003 ACM SIGGRAPH/Eurographics Symp. Computer Animation, 98-109.

The human hand is a masterpiece of mechanical complexity, able to perform fine motor manipulations and powerful work alike. Designing an animatable human hand model that features the abilities of the archetype created by Nature requires a great deal of anatomical detail to be modeled. In this paper, we present a human hand model with underlying anatomical structure. Animation of the hand model is controlled by muscle contraction values. We employ a physically based hybrid muscle model to convert these contraction values into movement of skin and bones. Pseudo muscles directly control the rotation of bones based on anatomical data and mechanical laws, while geometric muscles deform the skin tissue using a mass-spring system. Thus, resulting animations automatically exhibit anatomically and physically correct finger movements and skin deformations. In addition, we present a deformation technique to create individual hand models from photographs. A radial basis warping function is set up from the correspondence of feature points and applied to the complete structure of the reference hand model, making the deformed hand model instantly animatable.

ElKoura, G, & Singh, K, 2003: Handrix: animating the human hand. Proc. 2003 ACM SIGGRAPH/Eurographics Symp. Computer Animation, 110-119.

The human hand is a complex organ capable of both gross grasp and fine motor skills. Despite many successful high-level skeletal control techniques, animating realistic hand motion remains tedious and challenging. This paper presents research motivated by the complex finger positioning required to play musical instruments, such as the guitar. We first describe a data driven algorithm to add sympathetic finger motion to arbitrarily animated hands. We then present a procedural algorithm to generate the motion of the fretting hand playing a given musical passage on a guitar. The work here is aimed as a tool for music education and analysis. The contributions of this paper are a general architecture for the skeletal control of interdependent articulations performing multiple concurrent reaching tasks, and a procedural tool for musicians and animators that captures the motion complexity of guitar fingering.

Pullen, K, & Bregler, C, 2002: Motion capture assisted animation: texturing and synthesis. ACM Trans. Graphics, 21(3):501-508.

We discuss a method for creating animations that allows the animator to sketch an animation by setting a small number of keyframes on a fraction of the possible degrees of freedom. Motion capture data is then used to enhance the animation. Detail is added to degrees of freedom that were keyframed, a process we call texturing. Degrees of freedom that were not keyframed are synthesized. The method takes advantage of the fact that joint motions of an articulated figure are often correlated, so that given an incomplete data set, the missing degrees of freedom can be predicted from those that are present.

McDonald, J, Toro, J, Alkoby, K, Berthiaume, A, Carter, R, Chomwong, P, Christopher, J, Davidson, M J, Furst, J, Konie, B, Lancaster, G, Roychoudhuri, L, Sedgwick, E, Tomuro, N, & Wolfe, R, 2001: An improved articulated model of the human hand. Visual Computer, 17(3):158-166.

We present an improved anatomically based approach to modeling the human hand for use in the animation of the American Sign Language. The joint rotations in the model are based on the bone and muscle configurations of the hand, and a forward kinematic solution is used to position the hand. In particular, we investigate the rotations of the base joint of the thumb. This joint is a saddle joint with nontrivial rotational axes and centers, and must be treated with care in such a model. We take advantage of several correlations between joint rotations in the hand to reduce the number of degrees of freedom in the model and to provide a simple, natural, and interactive interface for American Sign Language handshape transcription.

Yasumuro, Y, Chen, Q, & Chihara, K, 1999: Three-dimensional modeling of the human hand with motion constraints. Image & Vision Computing, 17(2):149-156.

This paper describes a new approach for modeling the human hand by considering the dynamics and the natural constraints of the motion and the shape of hands. This hand model consists of a dynamic model and a surface model. The dynamic model is used to generate the posture of a hand. We show that the natural hand posture can be generated even when only a few hand parameters are available. The surface model is used to generate the hand shape based on the posture given by the dynamic model. The surface model is built based on the digitized three-dimensional shape of a real hand.

Rehg, J M, & Kanade, T, 1993: DigitEyes: Vision-Based human hand tracking. Technical Report CMU-CS-93-220, School of Computer Science, Carnegie Mellon University.

Passive sensing of human hand and limb motion is important for a wide range of applications from human -computer interaction to athletic performance measurement. High degree of freedom articulated mechanisms like the human hand are difficult to track because of their large state space and complex image appearance. This article describes a model-based hand tracking system, called DigitEyes, that can recover the state of a 27 DOF hand model from gray scale images at speeds of up to 10 Hz. We employ kinematic and geometric hand models, along with a high temporal sampling rate, to decompose global image patterns into incremental, local motions of simple shapes. Hand pose and joint angles are estimated from line and point features extracted from images of unmarked, unadorned hands, taken from one or more viewpoints. We present some preliminary results on a 3D mouse interface based on the DigitEyes sensor.

Davidoff, N A, & Freivalds, A, 1993: A graphic model of the human hand using CATIA. Int'l J. Industrial Ergonomics, 12(4):255-264.

Anthropometric measurements were made of the hands of 32 subjects. Data from this survey were used in the design of a graphic model of the male human hand. The model was created in the computer-aided design (CAD) package CATIA and reproduced any desired overall hand size. It was made up of 24 solid segments with 23 degrees of freedom at 17 moveable joints. Spans and grasping sizes of the model hand were examined. The representation of the model appeared realistic in several postures, but the model was found to allow some motions which would be impossible for an actual hand. The model could be used to help a designer visualize hand-held products and controls in use.

Rijpkema, H, & Girard, M, 1991: Computer animation of knowledge-based human grasping. SIGGRAPH Comput. Graph., 25(4):339-348.

The synthesis of human hand motion and grasping of arbitrary shaped objects is a very complex problem. Therefore high-level control is needed to perform these actions. In order to satisfy the kinematic and physical constraints associated with the human hand and to reduce the enormous search space associated with the problem of grasping objects, a knowledge based approach is used. A three-phased scheme is presented which incorporates the role of the hand, the object, the environment and the animator. The implementation of a hand simulation system HANDS is discussed.

Bone Structure from Surface Features

Rhee, T, Neumann, U, & Lewis, J P, 2006: Human hand modeling from surface anatomy. Proc. 2006 Symp. Interactive 3D Graphics & Games, 27-34.

The human hand is an important interface with complex shape and movement. In virtual reality and gaming applications the use of an individualized rather than generic hand representation can increase the sense of immersion and in some cases may lead to more effortless and accurate interaction with the virtual world. We present a method for constructing a person-specific model from a single canonically posed palm image of the hand without human guidance. Tensor voting is employed to extract the principal creases on the palmar surface. Joint locations are estimated using extracted features and analysis of surface anatomy. The skin geometry of a generic 3D hand model is deformed using radial basis functions guided by correspondences to the extracted surface anatomy and hand contours. The result is a 3D model of an individual's hand, with similar joint locations, contours, and skin texture.

Kurihara, T, & Miyata, N, 2004: Modeling deformable human hands from medical images. Proc. ACM SIGGRAPH/Eurographics Symp. Computer Animation, 355-363.

This paper presents a new method for constructing an example-based deformable human hand model from medical images. Realistic animation of human hands requires good estimates of the joint structure and properly weighted skeleton-driven surface deformation. For this purpose, we propose a method based on medical images of hands in several poses. Our method consists of the following 3 steps: First, using the measured bone shapes, we estimate the link structure (joint rotation centers) and the joint angles of each scan. Second, we construct a mutually consistent polygonal mesh of all the scans. For this purpose, a polygonal mesh of one pose, the base mesh, is deformed using skeletal subspace deformation, and then fitted interactively to the measured meshes from the other scans. Finally, the hand is deformed using a weighted pose space deformation. We demonstrate results of deformable hand models consisting of 100 ,000 triangle meshes derived from CT scans.

Arms and Shoulders

Grecu, V, Dumitru, N, & Grecu, L, 2009: A numerical approach for modeling the human upper limb. Proc. World Congress on Engineering (WCE 2009), 2:1719-1723.

The paper presents a dynamic model considering the human upper limb as a mechanic system with 6 degrees of freedom where the segments are moved by their own weight forces. The bones were modeled in Solid Works, the model of the upper limb obtained being very close as form to the real one. Based on this model, the calculus of mass proprieties was made. The differential equations of velocities obtained were solved using Lagrange formalism with help of Matlab programs. A dynamic model must provide a good approximation of total weight and mass distribution as well as transmissibility and amortization proprieties for bones, muscles and joints.

De Sapio, V, Warren, J, & Khatib, O, 2006: Predicting reaching postures using a kinematically constrained shoulder model. ed: Lennarcic, J, & Roth, B, Advances in Robot Kinematics, 209-218, Springer.

We present a new muscle effort criteria for predicting physiologically accurate upper limb motion in human subjects based on skeletal kinematics, muscle routing kinematics, and muscle strength characteristics. The new criteria properly accounts for the cross-joint coupling associated with the routing kinematics of multi -articular muscles. We also employ a new kinematically constrained model of the human shoulder complex, which is critical for the proper evaluation of our muscle effort criteria. Through a set of subject trials we have shown good correlation between natural reaching postures and our proposed criteria.

Full Body Models

He, L, 2005: Generation of human body models, M.Sc. Thesis, University of Auckland.

The goal of the research presented in this thesis is to design an integrated system for the automatic acquisition of a human body model, using input images from mutually orthogonal views. The model consists of two sets of free-form surface patches: the torso and its arms. We determine a neck joint on the torso and six joint positions on the arms (i.e., one location for each shoulder, elbow and wrist).
A conventional three-dimensional reconstruction technique called "shape from silhouettes" is applied in the project. However, one of the challenges of the project is to automatically separate the body segments during the reconstruction of the model from occluded silhouettes. The system suggests a body part localization procedure to determine the joint positions through the analysis of a feature view of body poses. Furthermore, this virtual human model is capable of simulating the motion of a real person using a-priori knowledge of the body. The human model will be specified using Virtual Reality Modelling Language (VRML) and visualized and rendered in a scene graph system, OpenSG.

Simonidis, C, Gärtner, S, & Do, M, 2009: Spezifikationen zu den Ganzkörpermenschmodellen im SFB 588, Technical Report, Institut für Technische Mechanik, Universität Karlsruhe.

Ganzkörpermodelle werden im Sonderforschungsbreich (SFB) 588 für die Untersuchung von Fragestellungen zur Analyse und Synthese menschlicher Bewegungen sowie zur Vorwärtssimulation und zur Erkennung von Bewegungen benötigt. Bedingt durch das große Spektrum an Anwendungsgebieten innerhalb des Gesamtprojekts wurden in der Vergangenheit mehrere anthropomorphe Modelle mit unterschiedlichem Detailgrad entwickelt. Hierbei werden zur Analyse der Bewegungen innerhalb der Biomechanik Modelle mit vielen Freiheitsgraden als auch detaillierte Einzelkomponenten des menschlichen Bewegungsapparates benötigt, die jedoch innerhalb der Bewegungssynthese in der Robotik keine größere Beachtung finden, da im Vergleich zum Menschen ein Roboter weniger mechanische Freiheitsgrade besitzt. Um die Arbeit zwischen den einzelnen Gruppen im SFB 588 zu koordinieren, ist es notwendig, verbindliche Standards zu schaffen. Auch soll eine Standardisierung der verwendeten Modelle die Möglichkeit eröffnen, Teilkomponenten des Gesamtsystems als auch Bewegungsdaten unterschiedlicher Aufnahmesysteme untereinander auszutauschen und vergleichbar zu machen.

Graphics, Animation and Rendering Techniques

van Waveren, J M P, 2005: Fast Skinning. <cache-www.intel.com/cd/00/00/29/37/293750_293750.pdf>.

Two approaches to matrix palette skinning are presented and optimized. Furthermore the Intel Streaming SIMD Extensions are used to exploit parallelism and get the most out of every clock cycle. The optimized routines are well over two times faster than the implementation in C on a Pentium 4.

James, D L, & Twigg, C D, 2005: Skinning Mesh Animations. ACM Trans. Graphics, 24(3):399-407.

We extend approaches for skinning characters to the general setting of skinning deformable mesh animations. We provide an automatic algorithm for generating progressive skinning approximations, that is particularly efficient for pseudo-articulated motions. Our contributions include the use of nonparametric mean shift clustering of high-dimensional mesh rotation sequences to automatically identify statistically relevant bones, and robust least squares methods to determine bone transformations, bone-vertex influence sets, and vertex weight values. We use a low-rank data reduction model defined in the undeformed mesh configuration to provide progressive convergence with a fixed number of bones. We show that the resulting skinned animations enable efficient hardware rendering, rest pose editing, and deformable collision detection. Finally, we present numerous examples where skins were automatically generated using a single set of parameter values.

Davies, L, 2004: Optimized CPU-based Skinning for 3D Games. <cache-www.intel.com/cd/00/00/17/21/172124_172124.pdf>.

Lifelike 3D character models play an increasingly important role in many computer games. Organic models, such as people, are more complex to render than rigid bodies because the mesh that defines the shape of the model constantly changes as the model animates. This animating mesh is referred to as a 'skin' since it's influenced by the underlying structure of the object; 'skinning' is the process of animating this mesh. Traditionally done on the CPU, as model complexity increased, skinning has been done on the video card using vertex shader class hardware. However, there are advantages to performing skinning on the CPU, which this paper highlights. It also details an optimal way of CPU-based skinning using the floating point Streaming SIMD Extensions (SSE) instructions found on the Intel(r) Pentium(r) III processors and above. This optimized solution offers greater than double the performance of the initial C implementation, as well as a flexible and efficient alternative to vertex shader skinning. In addition, we will discuss how the addition of multi-threading support improves this optimized CPU skinning solution, as well as the nuances involved with multi-threading the skinning algorithm.

Gourret, J, Magnenat-Thalmann, N, & Thalmann, D, 1989: Simulation of object and human skin deformations in a grasping task. ACM SIGGRAPH Comput. Graph., 23(3):21-30.

This paper addresses the problem of simulating deformations betwee n objects and the hand of a synthetic character during a grasping process. A numerical method based on finite element theory allows us to take into account the active forces of the fingers on the object and the reactive forces of the object on the fingers. The method improves control of synthetic human behavior in a task level animation system because it provides information about the environment of a synthetic human and so can be compared to the sense of touch. Finite element theory currently used in engineering seems one of the best approaches for modeling both elastic and plastic deformation of objects, as well as shocks with or without penetration between deformable objects. We show that intrinsic properties of the method based on composition /decomposition of elements have an impact in computer animation. We also state that the use of the same method for modeling both objects and human bodies improves the modeling both objects and human bodies improves the modeling of the contacts between them. Moreover, it allows a realistic envelope deformation of the human fingers comparable to existing methods. To show what we can expect from the method, we apply it to the grasping and pressing of a ball. Our solution to the grasping problem is based on displacement commands instead of force commands used in robotics and human behavior.

Komatsu, K, 1987: Human skin model capable of natural shape variation. The Visual Computer, 3(5):265-271.

In the production of character animation that treats living things moving at will, as in the case of humans and animals, it is important to express natural action and realistic body shape. If we can express these freely and easily, we will be able to apply character animation to many fields, such as the simulation of dances and sports, and electronic stand-ins. The computer graphics technique may be one of the most effective means to achieve such goals. We have developed human skin model capable of natural shape variation. This model has a skeleton structure, and free form surfaces cover the skeleton just like skin. The model permits continuous motion of every components of the skeleton according to actions. During such movements, the skin retains smoothness and naturalness. We are verifying the human skin model by producing several short animation pieces.

Standard Data Representations

Wu, G, van der Helm, F C T, Veeger, H E J, Mohsen Makhsous, M, Van Roy, P, Anglin, C, Nagels, J, Karduna, A R, McQuade, K, Wang, X, Werner, F W, & Buchholz, B, 2005: ISB recommendation on definitions of joint coordinate systems of various joints for the reporting of human joint motion, Part II: shoulder, elbow, wrist and hand. J. Biomech., 38(5):981-992.

In the past several years, the Standardization and Terminology Committee (STC) of the International Society of Biomechanics has been working to propose a set of standards for defining joint coordinate systems (JCS) of various joints based on Grood and Suntay's JCS of the knee joint (Grood and Suntay, 1983). The primary purpose of this work is to facilitate and encourage communication among researchers, clinicians, and all other interested parties.
The STC has established a total of nine sub-committees, involving nearly 30 people who have extensive experience (either research or clinical) in joint biomechanics, and had developed proposals for nine major joints in the body. These joints include: foot, ankle, hip, spine, shoulder, elbow, hand and wrist, TMJ, and whole body. The proposals are based on the ISB standard for reporting kinematic data published by Wu and Cavanagh (1995). The first set of these standards for the ankle joint, hip joint, and spine was published in Journal of Biomechanics in April 2002 (Wu et al., 2002). A response to comments to this set of standards was later published in 2003 (Allard et al., 2003).
In this publication, the proposed standards for the shoulder joint, elbow joint, and wrist and hand are included. For each joint, the standard is divided into the following sections: (1) Introduction, (2) Terminology, (3) Body segment coordinate systems, and (4) JCS and motion for the constituent joints. It is then up to the individual researcher to relate the marker or other (e.g. electromagnetic) coordinate systems to the defined anatomic system through digitization, calibration movements, or population-based anatomical relationships.
The two major values in using Grood and Suntay's JCS are: (1) conceptual, since it appears easier to communicate the rotations to clinicians when using individual axes embedded in the proximal and distal segments and (2) the inclusion of calculations for clinically relevant joint translations. Some confusion, however, has arisen over their statement that the JCS is sequence independent, whereas Euler or Cardan angle representations are not. It should be noted that the Grood and Suntay's convention, without the translations, is simply a linkage representation of a particular Cardan angle sequence; the floating axis is the second, i.e. rotated, axis in the Cardan sequence (Small et al., 1992 and Li et al., 1993, Baker, 2003). The angles are independent because the sequence is defined by the mechanism; a Cardan or Euler sequence is equally "independent" once the sequence is defined.

Wu, G, Siegler, S, Allard, P, Kirtley, C, Leardini, A, Rosenbaum, D, Whittle, M, D'Lima, D D, Cristofolini, L, Witte, H, Schmid, O, & Stokes, I, 2002: ISB recommendation on definitions of joint coordinate system of various joints for the reporting of human joint motion, Part I: ankle, hip, and spine. J. Biomech., 35(4):543-548.

The Standardization and Terminology Committee (STC) of the International Society of Biomechanics (ISB) proposes a general reporting standard for joint kinematics based on the Joint Coordinate System (JCS), first proposed by Grood and Suntay for the knee joint in 1983 (J. Biomech. Eng. 105 (1983) 136). There is currently a lack of standard for reporting joint motion in the field of biomechanics for human movement, and the JCS as proposed by Grood and Suntay has the advantage of reporting joint motions in clinically relevant terms.
In this communication, the STC proposes definitions of JCS for the ankle, hip, and spine. Definitions for other joints (such as shoulder, elbow, hand and wrist, temporomandibular joint (TMJ), and whole body) will be reported in later parts of the series. The STC is publishing these recommendations so as to encourage their use, to stimulate feedback and discussion, and to facilitate further revisions.
For each joint, a standard for the local axis system in each articulating bone is generated. These axes then standardize the JCS. Adopting these standards will lead to better communication among researchers and clinicians.

Sheehan, F T, & Mitiguy, P, 1999: In regards to the ISB recommendations for standardization in the reporting of kinematic data. J. Biomech., 32(10):1135-1136.

The ISB standards (Wu and Cavanagh, 1995) were intended to clarify the reporting of experimental results by standardizing the use of reference frames and orientation angles. However, the clarity is obscured by virtue of two di!erent standards being proposed. The differing standards arise from the fact that the orientation of a segment (e.g., the tibia) can be referenced to a global frame, "absolute orientation" or to a reference frame fixed in another segment (e.g., the femur), "relative orientation". However, there are occasions when the global frame is the relative frame. This happens, for example, when the femoral system is aligned with and fixed in the global frame. In such a case, the ISB standard propose that researchers report one set of values if the femur is considered to be a relative frame and a different set if the femur is considered to be the global frame. But these differing values correspond to the same orientation! We propose that a SINGLE standard should be adopted for reporting of all orientation angles, whether they are relative or absolute.

Wu, G, & Cavanagh, P R, 1995: ISB recommendations for standardization in the reporting of kinematic data. J. Biomech., 28(10):1258-1261.

Since 1990, the Standardization and Terminology Committee of the International Society of Biomechanics has been working towards a recommendation for standardization in the reporting of kinematic data. The paper, which is a result of those efforts (including broad input from members of the Society), is intended as a guide to the presentation of kinematic data in refereed publications and other materials. It is hoped that some uniformity in presentation will make publications easier to read and allow for the more straightforward comparison of data sets from different investigators. It is not intended to restrict individual investigators in the manner in which they collect or process their data. Rather, it could be viewed as a "output filter" applied to a variety of data formats to provide uniformity in the final product.
The ISB is cognizant of the various attempts at standardization that are being pursued by other organizations- such as CAMARC in Europe, the Clinical Gait Laboratory Group in the U.S.A., and the efforts of individual professional groups such as the Scoliosis Research Society. Where possible, we have sought unanimity with these groups, but on issues where the members of our society expressed strong opinions, we have-at timesstated a contrary view. One example in point is the use of center of mass-based segmental reference frames. Since such reference frames are needed for conventional dynamic analysis, we make the recommendation that such frames should be routinely used. We anticipate that extension to the present document in the future will include recommendations for joint coordinate systems and the definition of anatomical landmarks for locating other segmental reference frames.
The committee recognizes that standardization of the description of movement at individual joints is best left to those who are intimately involved in the study of those joints, and we have therefore appointed subcommittees for various joints to provide recommendations. Groups are currently active for the ankle, hand, shoulder, spine, temporomandibular joints, whole body and wrist; members with interests and explrtise in other joints are being actively sought. The initial recommendations of some of these groups have already been published in the ISB Newsletter, and once these recommendation have been discussed by the membership, a subsequent document on joint coordinate systems will be published.
The present recommendations are presented as a framework on which future progress can be based. We are grateful to former members of the Standardization and Terminology Committee (notably Professors John Paul, David Winter, and Don Grieve) and to the many ISB members who have commented on earlier drafts of this recommendation. The present paper owes much to the work of Sommer (1991).

Anthropometry Data

Nag, A, Nag, P K, & Hina Desai, H, 2003: Hand anthropometry of Indian women. Indian J. Med. Res., 117(6):260-269.

Background & objectives: Data on the physical dimension of the hand of Indian women are scanty. This information is necessary to ascertain human-machine compatibility in the design of manual systems for the bare and gloved hand, such as design and sizing of hand tools, controls, knobs and other applications in different kinds of precision and power grips. The present study was undertaken to generate hand anthropometric data of 95 women, working in informal industries (beedi, agarbatti and garment making).
Methods: Fifty one hand measurements of the right hand (lengths, breadths, circumferences, depths, spreads and clearances of hand and fingers) were taken, using anthropometric sliding and spreading calipers, measuring tape and handgrip strength dynamometer. The data were statistically analyzed to determine the normality of data and the percentile values of different hand dimensions, and simple and multiple regression analysis were done to determine better predictors of hand length and grip strength.
Results: The hand breadths, circumferences and depths were approximately normally distributed, with some deviation in case of the finger lengths. Hand length was significantly correlated with the fist, wrist and finger circumferences. The fist and wrist circumferences, in combination, were better predictors of hand length. The hand lengths, breadths and depths, including finger joints of the Indian women studied were smaller than those of American, British and West Indian women. The hand circumferences of the Indian women were also smaller than the American women. Grip strengths of Indian women (20.36±3.24 kg) were less than those of American, British and West Indian women. Grip strength was found to be statistically significant with hand dimensions, such as hand height perpendicular to wrist crease (digit 5), proximal interphalangeal joint breadth (digit 3) and hand spread across wedge 1.
Interpretation & conclusion: The women who are forced to frequently use cutters, strippers and other tools, which are not optimally designed to their hand dimensions and strength range, might have higher prevalence of clinical symptoms and disorders of the hand. In view of the human hand-tool interface requirements, the present data on Indian women would be useful for ergo-design applications of hand tools and devices.

Greiner, T M, 1992: Hand Anthropometry of U.S. Army Personnel. Technical Report Natick/TR-92/011, U.S. Army Natick Research Laboratory.

This report presents the results of the analysis of data on the hand gathered during the 1987-1988 anthropometric survey of Army personnel. Data are presented in the form of summary statistics and percentile tables. In addition, correlations, regressions, analyses of variance and principal components for sex and racial groups, nonmetric trait frequencies, and observer error magnitudes are reported. These data summaries are presented for a subset of the actual data base (1003 man and 1304 women) that match the working data bases summarized in the Anthropometric Survey's Final Report. Therefore, the hand working data bases match the demographic characteristics of the June 1988 Army, and are comprised of individuals who are in the Anthropometric Survey's working data bases. The dimensions given in this report include 64 hand measurements that were obtained using a special photometric system. An additional 22 dimensions, obtained through direct measurement during the Anthropometric Survey, were added to the hand data base. Therefore, a total of 86 dimensions are presented in this report. Measurement descriptions, visual indices, and a glossary are included to aid readers in locating dimensions and in interpreting presentations.

White, R M, 1981: Comparative Anthropometry of the Hand. Technical Report Natick/TR-81/010, U.S. Army Natick Research Laboratory.

Comparative anthropometric data on the human hand are presented and discussed in detail in this techical report. Since reliable and definitive data on the hands of the U.S. civilian population are lacking, anthropometric data on the hands of the U.S. military population may be utilized in analyses of handwear sizing.

Data are presented for ten hand measurements: Hand Length, Palm Length, Thumb CrotchLength, Hand Breadth, Hand Breadth at Thumb, Hand Circumference, Hand Circumference at Thumb, Fist Circumference, Wrist Circumference, and Hand Thickness. These hard measurements are defined and illustrated.
Detailed anthropometric data on the hands ot U.S. Army men and women are presented in the form of bivariate tables which depict the distribution of various categories of hand sizes, and show the interrelationships among hand dimensions.
Selected anthropometric data on the hands also are presented for a variety of foreign military populations in order to illustrate the range of variation in hand size to be found in different parts of the world.
In the final section, hands and handwear are examined in terms oi the sizing of handwear, and the development of tariffs for handwear is explained with illustrative examples.
An extensive bibliography concludes the report.

NASA, 1978: Anthropometric source book. Volume 1: Anthropometry for designers. Report Number NASA-RP-1024-VOL-1; S-479-VOL-1, NASA.

All the basic areas of anthropometry and its applications to the design of clothing, equipment, and workspaces for manned space flight are presented.

http://hdl.handle.net/2060/19790003563 (463 MB !!!)
parts of this are available at

ChI Intro http://www.engin.umich.edu/class/ioe491/Ref/NASA_1.pdf

Ch III McConville and Laubach http://ioe.engin.umich.edu/ioe491/Refs/NASA_3.pdf

Ch VIII McConville http://ioe.engin.umich.edu/ioe491/Refs/NASA_8_McConville.pdf

Ch IX Churchill http://ioe.engin.umich.edu/ioe491/Refs/NASA_9_Churchill.pdf

NASA, 1978: Anthropometric source book. Volume 2: A handbook of anthropometric data. Report Number NASA-RP-1024-VOL-2; S-479-VOL-2, NASA.

Volume 2 contains data resulting from surveys of 61 military and civilian populations of both sexes from the U.S., Europe, and Asia. Some 295 measured variables are defined and illustrated.

NASA, 1978: Anthropometric source book. Volume 3: Annotated bibliography of anthropometry. Report Number NASA-RP-1024-VOL-3; S-479-VOL-3, NASA.

Volume 3 is an annotated bibliography covering a broad spectrum of topics relevant to applied physical anthropology with emphasis on anthropometry and its applications in sizing and design.

Chandler, R F, Clauser, C E, McConville, J T, Reynolds, H M, & Young, J W, 1975: Investigation of inertial properties of the human body. USAF Technical Report AMRL-TR-74-137, U.S. Air Force Aerospace Medical Research Laboratory, Wright-Patterson Air Force Base.

Knowledge of the anthropometric parameters of the human body is essential for understanding of human kinetics and particularly for the design and testing of impact protective systems. Considerable information is available on the size, weight and center of mass of the body and its segments. This report supplements existing information with data regarding mass distribution characteristics of the human body as described by the principal moments on inertia and their orientation to body and segment anthropometry. The weight, center of mass location and principal moments of inertia of six cadavers were measured, the cadavers were then segmented and the mass, center of mass, moments of inertia and volume were measured on the fourteen segments from each cadaver. Standard and three-dimensronal anthropomehtry of the body and segments was also determined. This report describes the mathematical rationale and the techniques of measurement in detail. Results of the investigation are given as individual data values as well as summary statistics.

Kimbrough, R, 1972: An annotated bibliography of United States Air Force applied physical anthropology, January 1946 to March 1972. USAF Technical Report AMRL-TR-72-15, U.S. Air Force Aerospace Medical Research Laboratory, Wright-Patterson Air Force Base.

This report contains the titles, authors, publication/source information, and the abstracts of 122 technical reports and articles published by Anthropology Branch of the Aerospace Medical Research Laboratory between January 1946 and March 1972. It is a detailed document of the scope of the effort of the Air Force in the field of applied physical anthropology to provide the information on human body size and biomechanical characteristics of Air Force personnel required for the development and evaluation of Air Force systems, personal-protective equipment and clothing.

Garrett, J W, 1970: Anthropometry of the Hands of Male Air Force Flight Personnel. USAF Technical Report AMRL-TR-69-42, U.S. Air Force Aerospace Medical Research Laboratory, Wright-Patterson Air Force Base.

This report contains descriptions of and data on 56 anthropometric dimensions of the hands of 148 male Air Force flight personnel. Selected dimensional comparisons indicate that this sample is representative of the total group of Air Force flight personnel. Summary statistics presented include the means, standard deviations, ranges, selected percentiles, and coefficients of variation. Also included are data on the age, rank, major Air Command, and commissioned status of the sample; a complete matrix of intercorrelations among the anthropometric dimensions; bivariate tables; multiple regression equations; and nomographs for selected combinations of dimensions. A procurement table for the U.S. Air Force 12-size glove program revised to reflect the latest anthropometric data is presented.

Garrett, J W, 1970: Anthropometry of the Air Force Female Hand. USAF Technical Report AMRL-TR-69-26, U.S. Air Force Aerospace Medical Research Laboratory, Wright-Patterson Air Force Base.

The report describes 56 anthropometric dimensions measured on the hands of Air Force female personnel (Women in the Air Force, Nurse Corp, and Biomedical Science Corps), aged 18-56. Summary statistics including the means, standard deviations, ranges, selected percentiles, measures of distribution, and coefficients of variation are presented for the 56 dimensions. Also included are statistical variations by age, rank and Corps within the sample, a complete correlation matrix, bivariate tables, and nomographs for various selected combinations of dimensions.

Clauser, C E, McConville, J T & Young, J W, 1969: Weight, volume and center of mass segments of the human body. USAF Technical Report AMRL-TR-69-70, U.S. Air Force Aerospace Medical Research Laboratory, Wright-Patterson Air Force Base.

This study was designed to supplement existing knowledge of the weight, volume, and center of mass of segments of the human body and to permit their more accurate estimation on the living from anthropometric dimensions. Weight, volume and center of mass of 14 segments of the body were determined on 13 cadavers. Presented are descriptive statistics of these variables as well as a series of regression equations predicting these parameters from anthopometry. Included in the seven supporting appendices are reports of the studies of the mid-volume of segments as an approximation of their center of mass, relationships between standing and supine anthropometry, postmortem changes in gross body size, and comparisons between densities of fresh and preserved human tissues.

Oberg, T, & Karsznia, A, 1994: Joint angle parameters in gait: Reference data for normal subjects, 10-79 years of age. J. Rehabilitation Research & Development, 31(3):199-209.

Provides reference data on gait for joint angle parameters. Changes attributed to age; Absence of side and sex differences; Changes attributed to gait speed.

Oberg, T, & Karsznia, A, 1993: Basic gait parameters: Reference data for normal subjects, 10-79. J. Rehabilitation Research & Development, 30(2):210-223.

Provides reference data from normal subjects to be compared to basic gait parameters used in the evaluation of pathological gait. Subjects and procedures; Results; Relevance.

Hand Anthropometry Notes

OpenSim Models

Arnold, E M, Ward, S R, Lieber, R L, Delp, S L, 2010: Ann. Biomed. Eng., 38(2):269-279.

Computer models that estimate the force generation capacity of lower limb muscles have become widely used to simulate the effects of musculoskeletal surgeries and create dynamic simulations of movement. Previous lower limb models are based on severely limited data describing limb muscle architecture (i.e., muscle fiber lengths, pennation angles, and physiological cross-sectional areas). Here, we describe a new model of the lower limb based on data that quantifies the muscle architecture of 21 cadavers. The model includes geometric representations of the bones, kinematic descriptions of the joints, and Hill-type models of 44 muscle-tendon compartments. The model allows calculation of muscle-tendon lengths and moment arms over a wide range of body positions. The model also allows detailed examination of the force and moment generation capacities of muscles about the ankle, knee, and hip and is freely available at www .simtk.org.

Holzbaur, K R, Murray, W M, & Delp, S L, 2005: A model of the upper extremity for simulating musculoskeletal surgery and analyzing neuromuscular control. Ann. Biomed. Eng., 33(6):829-840.

Biomechanical models of the musculoskeletal system are frequently used to study neuromuscular control and simulate surgical procedures. To be broadly applicable, a model must be accessible to users, provide accurate representations of muscles and joints, and capture important interactions between joints. We have developed a model of the upper extremity that includes 15 degrees of freedom representing the shoulder, elbow, forearm, wrist, thumb, and index finger, and 50 muscle compartments crossing these joints. The kinematics of each joint and the force-generating parameters for each muscle were derived from experimental data. The model estimates the muscle-tendon lengths and moment arms for each of the muscles over a wide range of postures. Given a pattern of muscle activations, the model also estimates muscle forces and joint moments. The moment arms and maximum moment-generating capacity of each muscle group (e.g., elbow flexors) were compared to experimental data to assess the accuracy of the model. These comparisons showed that moment arms and joint moments estimated using the model captured important features of upper extremity geometry and mechanics. The model also revealed coupling between joints, such as increased passive finger flexion moment with wrist extension. The computer model is available to researchers at http://nmbl.stanford.edu.

The Multimod Application Framework

Viceconti, M, Taddei, F, Montanari, L, Testi, D, Leardini, A, Clapworthy, G J, & Van Sint Jan, S, 2007: Multimod Data Manager: A tool for data fusion. Computer Methods & Programs in Biomedicine, 87(2):148-159.

Nowadays biomedical engineers regularly have to combine data from multiple medical imaging modalities, biomedical measurements and computer simulations and this can demand the knowledge of many specialised software tools. Acquiring this knowledge to the depth necessary to perform the various tasks can require considerable time and thus divert the researcher from addressing the actual biomedical problems. The aim of the present study is to describe a new application called the Multimod Data Manager, distributed as a freeware, which provides the end user with a fully integrated environment for the fusion and manipulation of all biomedical data. The Multimod Data Manager is generated using a software application framework, called the Multimod Application Framework, which is specifically designed to support the rapid development of computer aided medicine applications. To understand the general logic of the Data Manager, we first introduce the framework from which it is derived. We then illustrate its use by an example-the development of a complete subject-specific musculo-skeletal model of the lower limb from the Visible Human medical imaging data to be used for predicting the stresses in the skeleton during gait. While the Data Manager is clearly still only at the prototype stage, we believe that it is already capable of being used to solve a large number of problems common to many biomedical engineering activities.

Viceconti, M, Zannoni, C, Testi, D, Petrone, M, Perticoni, S, Quadrani, P, Taddei, F, Imboden, S, & Clapworthy, G J, 2007: The multimod application framework: A rapid application development tool for computer aided medicine. Computer Methods & Programs in Biomedicine, 85(2)138-151.

This paper describes a new application framework (OpenMAF) for rapid development of multimodal applications in computer-aided medicine. MAF applications are multimodal in data, in representation, and in interaction. The framework supports almost any type of biomedical data, including DICOM datasets, motion -capture recordings, or data from computer simulations (e.g. finite element modeling). The interactive visualization approach (multimodal display) helps the user interpret complex datasets, providing multiple representations of the same data. In addition, the framework allows multimodal interaction by supporting the simultaneous use of different input-output devices like 3D trackers, stereoscopic displays, haptics hardware and speech recognition/synthesis systems. The Framework has been designed to run smoothly even on limited power computers, but it can take advantage of all hardware capabilities. The Framework is based on a collection of portable libraries and it can be compiled on any platform that supports OpenGL, including Windows, MacOS X and any flavor of Unix/linux.

Van Sint Jan, S, Viceconti, M, & Clapworthy, G J, 2004: Modern visualisation tools for research and education in biomechanics. Proc. 8th IEEE Int'l Conf. Information Visualisation (IV'04), 9-14.

The DataManager presented in this paper allows the multimodal visualisation of heterogeneous data originating from the biomedical field and, more particularly, biomechanics. In the latter, the aim is to increase our understanding of the musculo-skeletal system, and to achieve this, numerous disparate data must be collected and combined. Previously, no software tool fully allowed such integration, but the DataManager and its development environment (the MAF) now provide an answer to that problem. This paper presents the current visualisation and data processing tools available from the DataManager. Its usefulness for research, educational and clinical activities will be demonstrated. The system developers hope that the data management mechanisms available within the software will stimulate data sharing between scientists and will encourage them to participate in enhancing the system by integrating their own software tools.

Viceconti, M, Astolfi, L, Leardini, A, Imboden, S, Petrone, M, Quadrani, P, Taddei, F, Testi, D, & Zannoni, C, 2004: The multimod application framework. Proc. 8th IEEE Int'l Conf. Information Visualisation (IV'04), 15-20.

This paper presents the Multimod Application Framework, a software framework for the rapid development of computer-aided medicine applications. This framework, distributed under an open source licence, is being developed as part of the Multimod Project, a multi-national research endeavour partially supported by the European Commission through the Fifth Framework Programme. This application framework provides an effective re-use model for visualisation and data-processing algorithms that may be incorporated into the framework with moderate overhead and then made available to the biomedical research community as part of a complete set of applications.

Krokos, M, Savenko, A, Clapworthy, G J, Lin, H, Mayoral, R, Viceconti, M, & Van Sint Jan, S, 2004: Real-time visualisation within the Multimod Application Framework. Proc. 8th IEEE Int'l Conf. Information Visualisation (IV'04), 21-26.

This work gives an overview of real-time visualisation algorithms developed under the EC-funded project Multimod to support a novel paradigm for the virtual representation of musculo-skeletal structures. These algorithms are fully integrated into the Multimod Application Framework (MAF), an open-source freely -available software framework for the rapid development of medical visualisation applications. MAF is based on the visualisation toolkit (VTK) and other specialised toolkits, e.g. for image registration and segmentation, collision detection or numerical computation. MAF provides a range of high-level components that can be easily combined for rapid construction of visualisation applications that support synchronised views. The majority of algorithms available within the standard underlying MAF toolkits were frequently either too slow or too general for our purposes. We have thus implemented computationally efficient versions of existing algorithms, e.g. for surface and volume rendering, and more importantly, developed new techniques, e.g. for X-ray rendering and designing volume rendering transfer functions. To achieve interactive rendering we have employed a scheme for space partitioning. The emphasis is on exploiting the characteristics of medical datasets (e.g. density value homogeneity) but further utilising the hardware -accelerated capabilities of modern graphics cards. In this context, calculations are moved into hardware as appropriate while avoiding dependency on specialised features of particular manufacturers so as to ensure real code portability.

Anatomy, Kinesiology and Biomechanics

Hands and Fingers

Foumani, M, Strackee, S D, Jonges, R, Blankevoort, L, Zwinderman, A H, Carelsen, B, & Streekstra, G J, 2009: In-vivo three-dimensional carpal bone kinematics during flexion-extension and radio-ulnar deviation of the wrist: Dynamic motion versus step-wise static wrist positions. J. Biomech., 42(16):2664-2671.

An in-vivo approach to the measurement of three-dimensional motion patterns of carpal bones in the wrist may have future diagnostic applications, particularly for ligament injuries of the wrist. Static methods to measure carpal kinematics in-vivo only provide an approximation of the true kinematics of the carpal bones. This study is aimed at finding the difference between dynamically and statically acquired carpal kinematics.
For eight healthy subjects, static and a dynamic measurements of the carpal kinematics were performed for a flexion-extension and a radio-ulnar deviation movement. Dynamic scans were acquired by using a four-dimensional X-ray imaging system during an imposed cyclic motion. To assess static kinematics of the wrists, three-dimensional rotational X-ray scans were acquired during step-wise flexion-extension and radio-ulnar deviation. The helical axis rotations and the rotation components. i.e. flexion-extension, radio -ulnar deviation and pro-supination were the primary parameters. Linear mixed model statistical analysis was used to determine the significance of the difference between the dynamically and statically acquired rotations of the carpal bones.
Small and in most cases negligible differences were observed between the dynamic motion and the step-wise static motion of the carpal bones. The conclusion is that in the case of individuals without any pathology of the wrist, carpal kinematics can be studied either dynamically or statically. Further research is required to investigate the dynamic in-vivo carpal kinematics in patients with dynamic wrist problems.

Moore, D C, Crisco, J J, Trafton, T G, & Leventhal, E L, 2007: A digital database of wrist bone anatomy and carpal kinematics. J. Biomech., 40(11)2537-2542.

The skeletal wrist consists of eight small, intricately shaped carpal bones. The motion of these bones is complex, occurs in three dimensions, and remains incompletely defined. Our previous efforts have been focused on determining the in vivo three-dimensional (3-D) kinematics of the normal and abnormal carpus. In so doing we have developed an extensive database of carpal bone anatomy and kinematics from a large number of healthy subjects. The purpose of this paper is to describe that database and to make it available to other researchers. CT volume images of both wrists from 30 healthy volunteers (15 males and 15 females) were acquired in multiple wrist positions throughout the normal range of wrist motion. The outer cortical surfaces of the carpal bones, radius and ulna, and proximal metacarpals were segmented and the 3 -D motion of each bone was calculated for each wrist position. The database was constructed to include high-resolution surface models, measures of bone volume and shape, and the 3-D kinematics of each segmented bone. The database does not include soft tissues of the wrist. While there are numerous digital anatomical databases, this one is unique in that it includes a large number of subjects and it contains in vivo kinematic data as well as the bony anatomy.

Lee, S W, & Zhang, X, 2007: Biodynamic modeling, system identification, and variability of multi-finger movements. J. Biomech., 40(14):3215-3222.

A forward dynamic model of human multi-fingered hand movement is proposed. The model represents digits 2-5 in manipulative acts as a 12-degrees-of-freedom (DOF) system, driven by torque actuators at individual joints and controlled using a parsimonious proportional-derivative (PD) scheme. The control parameters as feedback gains along with an auxiliary parameter to modulate the joint torque magnitudes and cross -coupling can be empirically identified in an iterative procedure minimizing the discrepancy between the model-prediction and measurement. The procedure is guided and computationally accelerated by pre -knowledge of relations between the parameters and kinematic responses. An empirical test based on real grasping movement data showed that the model simulated the multi-finger movements with varied inter -joint temporal coordination accurately: the grand mean of the root-mean-square-errors (RMSE) across trials performed by 28 subjects was 3.25°. Analyses of the model parameters yielded new insights into intra- and inter-person variability in multi-finger movement performance, and distinguished the less variable motor control strategy from much more variable anthropometric and physiological factors.

Leonard, L, Sirkett, D, Mullineux, G, Giddins, G E B, & Miles, A W: Development of an in-vivo method of wrist joint motion analysis. Clin. Biomech., 20(2):166-171.

Background. A clinically applicable method of plotting wrist joint motion in three-dimensions has not been described. Computer modelling has been used to improve joint arthroplasty elsewhere in the body. We aimed to develop a method of measuring, and modelling, wrist joint motion that could potentially be used to improve the kinematic performance of wrist arthroplasty designs.
Methods. An electromagnetic system was used to record wrist motion in three-dimensions. A small pilot study attempted to assess repeatability. A larger group of volunteers with normal wrists was also studied. An iterative computer model, using a two-axis hinge, was developed. One output from this model, the offset of the two axes of motion, is presented as an example of the possible applications of this method of analysis.
Findings. For any one individual, in the pilot study, the offset of the axes calculated was relatively reproducible. Between individuals the difference in the offset of the axes was more marked. In 99 normal sets of data the mean axis offset was 6.8 mm (range 28 mm to -21 mm) A positive value represented the radio-ulnar deviation axis placed distal to the flexion-extension axis.
Interpretation. The three-dimensional motion plots generated using this method could be used clinically to follow disease progression or recovery following surgery. The computer modelling method described has potential applications, if further refined, to wrist joint arthroplasty design.

Valero-Cuevas, F J, 2005: An integrative approach to the biomechanical function and neuromuscular control of the fingers. J. Biomech., 38(4):673-684.

The exquisite mechanical functionality and versatility of the human hand emerges from complex neuro -musculo-skeletal interactions that are not completely understood. I have found it useful to work within a theoretical/experimental paradigm that outlines the fundamental neuro-musculo-skeletal components and their interactions. In this integrative paradigm, the laws of mechanics, the specifications of the manipulation task, and the sensorimotor signals define the interactions among hand anatomy, the nervous system, and manipulation function. Thus, our collaborative research activities emphasize a firm grounding in the mechanics of finger function, insistence on anatomical detail, and meticulous characterization of muscle activity. This overview of our work on precision pinch (i.e., the ability to produce and control fingertip forces) presents some of our findings around three Research Themes: Mechanics-based quantification of manipulation ability; Anatomically realistic musculoskeletal finger models; and Neural control of finger muscles. I conclude that (i) driving the fingers to some limit of sensorimotor performance is instrumental to elucidating motor control strategies; (ii) that the cross-over of tendons from flexors to extensors in the extensor mechanism is needed to produce force in every direction, and (iii) the anatomical routing of multiarticular muscles makes co-contraction unavoidable for many tasks. Moreover, creating realistic and clinically useful finger models still requires developing new computational means to simulate the viscoelastic tendinous networks of the extensor mechanism, and the muscle-bone-ligament interactions in complex articulations. Building upon this neuromuscular biomechanics paradigm is of immense clinical relevance: it will be instrumental to the development of clinical treatments to preserve and restore manual ability in people suffering from neurological and orthopedic conditions. This understanding will also advance the design and control of robotic hands whose performance lags far behind that of their biological counterparts.

Valero-Cuevas, F J, Johanson, M E, & Towles, J D, 2003: Towards a realistic biomechanical model of the thumb: the choice of kinematic description may be more critical than the solution method or the variability/uncertainty of musculoskeletal parameters. J. Biomech., 36(7):1019-1030.

A biomechanical model of the thumb can help researchers and clinicians understand the clinical problem of how anatomical variability contributes to the variability of outcomes of surgeries to restore thumb function. We lack a realistic biomechanical model of the thumb because of the variability/uncertainty of musculoskeletal parameters, the multiple proposed kinematic descriptions and methods to solve the muscle redundancy problem, and the paucity of data to validate the model with in vivo coordination patterns and force output. We performed a multi-stage validation of a biomechanical computer model against our measurements of maximal static thumbtip force and fine-wire electromyograms (EMG) from 8 thumb muscles in each of five orthogonal directions in key and opposition pinch postures. A low-friction point-contact at the thumbtip ensured that subjects did not produce thumbtip torques during force production. The 3-D, 8-muscle biomechanical thumb model uses a 5-axis kinematic description with orthogonal and intersecting axes of rotation at the carpometacarpal and metacarpophalangeal joints. We represented the 50 musculoskeletal parameters of the model as stochastic variables based on experimental data, and ran Monte Carlo simulations in the "inverse" and "forward" directions for 5000 random instantiations of the model. Two inverse simulations (predicting the distribution of maximal static thumbtip forces and the muscle activations that maximized force) showed that: the model reproduces at most 50% of the 80 EMG distributions recorded (eight muscle excitations in 5 force directions in two postures); and well-directed thumbtip forces of adequate magnitude are predicted only if accompanied by unrealistically large thumbtip torques (0.64±0.28 N m). The forward simulation (which fed the experimental distributions of EMG through random instantiations of the model) resulted in misdirected thumbtip force vectors (within 74.3±24.5° from the desired direction) accompanied by doubly large thumbtip torques (1.32±0.95 N m). Taken together, our results suggest that the variability and uncertainty of musculoskeletal parameters and the choice of solution method are not the likely reason for the unrealistic predictions obtained. Rather, the kinematic description of the thumb we used is not representative of the transformation of net joint torques into thumbtip forces /torques in the human thumb. Future efforts should focus on validating alternative kinematic descriptions of the thumb.

Sancho-Bru, J L, Perez-Gonzalez, A, Vergara-Monedero, M, & Giurintano, D, 2001: A 3-D dynamic model of human finger for studying free movements. J. Biomech., 34(11):1491-1500.

The purpose of this work is to develop a 3D inverse dynamic model of the human finger for estimating the muscular forces involved during free finger movements. A review of the existing 3D models of the fingers is presented, and an alternative one is proposed. The validity of the model has been proved by means of two simulations: free flexion-extension motion of all joints, and free metacarpophalangeal (MCP) adduction motion. The simulation shows the need for a dynamic model including inertial effects when studying fast movements and the relevance of modelling passive forces generated by the structures studying free movements, such as the force exerted by the muscles when they are stretched and the passive action of the ligaments over the MCP joint in order to reproduce the muscular force pattern during the simulation of the free MCP abduction-adduction movements.

Giurintano, D J, Hollister, A M, Buford, W L, Thompson, D E & Myers, L M, 1995: A virtual five-link model of the thumb. Med. Eng. Phys., 17(4):297-303.

Most researchers have modelled the thumb as three rigid links with connections of two universal joints (carpometacarpal joint and metacarpo-phalangeal joint), and a hinge joint (interphalangeal joint). Although this produces the required number of degrees of freedom, the resulting motion is not anatomically accurate. In this work, the thumb is modelled as a five-link manipulator with the virtual links connected by hinge joints - one for each degree of freedom of the thumb. The axes of the hinges are not orthogonal to one another, to the long axis of the bones or to the anatomic planes. Four static positions of hand function were analysed - key pinch, screwdriver hold, tip pinch, and wide grasp. The virtual five-link model of the thumb predicted similar muscle recruitment patterns to published EMG data. The force at the distal surface of the trapezium is between 6 and 24 times the applied load depending on the posture.

Cooney, W P, Lucca, M J, Chao, E Y, & Linscheid, R L, 1981: The kinesiology of the thumb trapeziometacarpal joint. J. Bone & Joint Surgery, 63(9):1371-1381.

To measure the motions of the trapeziometacarpal joint of the thumb quantitatively, a roentgenographic method was developed and tested using T-shaped metal markers, a special cassette-holder, and biplane roentgenograms. Two experiments were performed. In the first one, the metal markers were fixed to the trapezium and third metacarpal in ten cadaver specimens, and a fixed spatial relationship between the trapezium and the third metacarpal was identified roentgenographically. This relationship was that the reference axes of the trapezium were aligned at median angles of 48 degrees of flexion, 38 degrees of abduction, and 80 degrees of pronation with reference to the reference axes of the third metacarpal. In the second experiment, in the dominant hand of nine male and ten female subjects (average age, twenty-six years) T-shaped markers were fixed to the skin overlying the third metacarpal and the metacarpal and phalanges of the thumb. Using the same roentgenographic technique and coordinate systems employed in the first study, the average total motions of the trapeziometacarpal joint (determined as motions of the first metacarpal with reference to the third metacarpal) were 53 degrees of flexion-extension, 42 degrees of abduction-adduction, and 17 degrees of axial rotation (pronation-supination). In addition, six functional positions of the thumb were studied: rest, flexion, extension, abduction, tip pinch, and grasp. A position of adduction and flexion of the trapeziometacarpal joint was most common during thumb function, and both the trapeziometacarpal and metacarpophalangeal joints contributed to rotation of the thumb.

Arms and Shoulders

Klopcar, N, Tomšic, M, & Lenarcic, J, 2007: A kinematic model of the shoulder complex to evaluate the arm-reachable workspace. J. Biomech., 40(1):86-91.

Upper-arm evaluation including shoulder motion in physiotherapy has no three-dimensional tool for an arm -functioning evaluation, which hampers an uniform, objective comparison. Human shoulder complex models suffer from lack of shoulder girdle kinematic data. A kinematic shoulder-complex model with six degrees of freedom is proposed as the composition of the inner joint representing the shoulder-girdle joints and outer joint representing the glenohumeral joint. The outer shoulder joint has three perpendicular rotations: adduction/abduction, retroflexion/flexion and internal/external rotation of the humerus. The inner shoulder joint has two rotations, depression/elevation and retraction/protraction, and one translation, which are all dependent on the elevation angle of the humerus. The human arm-reachable workspace that represents the area within reach of the wrist is calculated on the basis of the shoulder-complex model and the additional elbow-joint direct kinematics. It was demonstrated that cross-sections of the calculated workspace are in agreement with the measured arm-reachable workspace in all three anatomical planes. The arm-reachable workspace volume and graphics were calculated and a comparison of the arm's workspaces during a patient's shoulder treatment was made. The obtained numerical and graphical arm-reachable workspaces can be used for arm-functioning evaluations in rehabilitation and ergonomics.

Londi, F, Pennestri, E, Valentini, P P, & Vita, L, 2004: Control and virtual reality simulation of tendon driven mechanisms. Multibody System Dynamics, 12(2):133-145.

In this paper the authors present a control strategy for tendon driven mechanisms. The aim of the control system is to find the correct torques which the motors have to exert to make the end effector describe a specific trajectory. In robotic assemblies this problem is often solved with closed loop algorithm, but here a simpler method, based on a open loop strategy, is developed. The difficulties in the actuation are in keeping the belt tight during all working conditions. So an innovative solution of this problem is presented here. This methodology can be easily applied in real time monitoring or very fast operations. For this reason several virtual reality simulations, developed using codes written in Virtual Reality Markup Language, are also presented. This approach is very efficient because it requires a very low CPU computation time, small size files, and the manipulator can be easily put into different simulated scenarios.

Pennestri, E, Renzi, A, & Santonocito, P, 2002: Dynamic modeling of the human arm with video-based experimental analysis. Multibody System Dynamics, 7(4):389-406.

The quantitative assessment of human joint torque capability has many important applications. By means of a multibody approach, the authors described a formulation for 3D inverse dynamic analysis of a human arm during voluntary free movement. In particular, it is presented as a test case where the kinematics of the arm is obtained by means of a video-based human motion analysis system.

Weinberg, A M, Pietsch, I T, Helm, M B, Hesselbach, J, & Tscherne, H, 2000: A new kinematic model of pro- and supination of the human forearm. J. Biomech., 33(4):487-491.

We introduce a new kinematic model describing the motion of the human forearm bones, ulna and radius, during forearm rotation. During this motion between the two forearm extrem-positions, referred to as supination (palm up) and pronation (palm down), effects occur, that cannot be explained by the the established kinematic model of R. Fick from 1904. Especially, the motion of the ulna is not properly reproduced by Fick's model. During forearm rotation an evasive motion of the ulna is observed by various authors, using magnetic resonance imaging (MRI) technology. Our new kinematic model also simulates this evasive motion. Furthermore, the model is enlarged to include angulations of the forearm bones. Using these results the influence of forearm fractures on the range of forearm motion can be predicted. This knowledge can be used by surgeons to choose the optimal therapy in re-establishing free forearm mobility.

Wang, X, 1999: A behavior-based inverse kinematics algorithm to predict arm prehension postures for computer-aided ergonomic evaluation. J. Biomech., 32(5):453-460.

In this paper, the computational problem of inverse kinematics of arm prehension movements was investigated. How motions of each joint involved in arm movements can be used to control the end-effector (hand) position and orientation was first examined. It is shown that the inverse kinematics problem due to the kinematic redundancy in joint space is ill-posed only at the control of hand orientation but not at the control of hand position. Based upon this analysis, a previously proposed inverse kinematics algorithm (Wang et Verriest, 1998a) to predict arm reach postures was extended to a seven-DOF arm model to predict arm prehension postures using a separate control of hand position and orientation. The algorithm can be either in rule-based form or by optimization through appropriate choice of weight coefficients. Compared to the algebraic inverse kinematics algorithm, the proposed algorithm can handle the non-linearity of joint limits in a straightforward way. In addition, no matrix inverse calculation is needed, thus avoiding the stability and convergence problems often occurring near a singularity of the Jacobian. Since an end-effector motion-oriented method is used to describe joint movements, observed behaviors of arm movements can be easily implemented in the algorithm. The proposed algorithm provides a general frame for arm postural control and can be used as an efficient postural manipulation tool for computer-aided ergonomic evaluation.

Chéze, L, Gutierrez, C, San Marcelino, R, & Dimnet, J, 1996: Biomechanics of the upper limb using robotic techniques. Human Mov. Sci., 15(3):477-496.

Motion biomechanics in living subjects is very often obtained from data provided by optoelectronic systems. They describe 3-dimensional trajectories of either emitting or reflective markers fixed upon subject body segments. Is it possible to use these trajectories to define the 3-dimensional kinematics of articular groups while carrying out a task? In this paper, the methodology to determine the movements of the internal structure of an upper limb from external marker trajectories is described. Each body segment (trunk, arm, forearm, hand) is labeled by reflective markers. The individual trajectories are numerically treated so that each segment can be considered as a solid for which laws of solid mechanics can be applied. A mechanism is introduced to represent the functional anatomy of the upper limb carrying out a task. The laws of displacements of this mechanism restitute the motion of body segments known from the external markers trajectories. The structure of the mechanism is defined from pretestings to determine the flexion and rotation axes of body segments, the centers of rotation and their corresponding scattering. A robotic approach is used so as to numerically describe the upper limb whilst carrying out a task. The mechanical structure is a complex structure of longitudinal links representing human body segments. These links are connected together by the mean of articular groups. Each of these groups have 3 angular degrees of freedom (df). Experimental testings have been performed using anatomic specimens of the upper limb. They focus upon geometrical constraints existing between the 3 angular df of each articular group. The corresponding laws of motion obtained on cadavers are assumed to be representative of the bony structure of living subjects. The structure of the mechanism is then modified to take into account this phenomenon and to introduce the disturbing effects due to the soft tissues.

Pontonnier, C, & Dumont, G, 2009: Motion analysis of the arm based on functional anatomy. Modelling the Physiological Human, Lecture Notes in Computer Science, 5903:137-149.

This article presents a biomechanical model of the right arm, developed with respect to the functional anatomy of the human. This model is developed as a motion analysis tool. The main application issued from this model is the muscle forces estimation of the flexion/extension and internal rotation of the forearm joints. This estimation is based on an inverse dynamics method, improved with additionnal constraints such as co-contraction factor between flexors and extensors of a joint. The article first presents the related work, then presents the biomechanical model developed, and at last some results obtained for a sample movement of the arm.

Full Body Models

Teu, K K, & Kim, W, 2006: The analysis of golf swing as a kinematic chain using dual Euler angle algorithm . J. Biomech., 39(7):1227-1238.

The manner in which anatomical rotation from an individual segment contributes to the position and velocity of the endpoint can be informative in the arena of many athletic events whose goals are to attain the maximal velocity of the most distal segment. This study presents a new method of velocity analysis using dual Euler angles and its application in studying rotational contribution from upper extremity segments to club head speed during a golf swing. Dual Euler angle describes 3D movement as a series of ordered screw motions about each orthogonal axis in a streamlined matrix form-the dual transformation matrix- and allows the translation and rotation component to be described in the same moving frame. Applying this method in biomechanics is a novel idea and the authors have previously applied the methodology to clinical studies on its use in displacement analysis. The focus of this paper is velocity analysis and applications in sports biomechanics. In this study, electrogoniometers (Biometrics, UK) with a frequency of 1000 Hz were attached to a subject during the execution of the swing to obtain the joint angles throughout the motion. The velocity of the club head was then analyzed using the dual velocity which specifies the velocity distribution of a rigid body in screw motion at any point in time as the dual vector. The contributions of each segment to the club-head velocity were also compared. In order to evaluate this method, the calculated position and velocity of the club head were compared to the values obtained from video image analysis. The results indicated that there is good agreement between calculated values and video data, suggesting the suitability of using the Dual Euler method in analyzing a kinematic chain motion.

Silva, M PT, Ambró sio, J A C, & Pereira, M S, 1997: Biomechanical model with joint resistance for impact simulation. Multibody System Dynamics, 1(1):65-84.

Based on a general methodology using naturalco-ordinates, a three-dimensional whole body responsemodel for the articulated human body is presented inthis paper. The joints between biomechanical segmentsare defined by forcing adjacent bodies to share commonpoints and vectors that are used in their definition.A realistic relative range of motion for the bodysegments is obtained introducing a set of penaltyforces in the model rather than setting up newunilateral constraints between the system components.These forces, representing the reaction momentsbetween segments of the human body model when thebiomechanical joints reach the limit of their range ofmotion, prevent the biomechanical model from achievingphysically unacceptable positions. Improved efficiencyin the integration process of the equations of motionis obtained using the augmented Lagrange formulation.The biomechanical model is finally applied indifferent situations of passive human motion such asthat observed in vehicle occupants during a crash orin an athlete during impact.

Laurie, H, & Penne, R, 2004: Projective geometry of human motion, with an application to injury risk. SIAM J. Appl. Math., 65(2):702-719.

We give an exposition of Plücker vectors for a system of joint axes in projective 3-space. We use Plücker vectors to analyze dependencies among joint axes and in particular to show that two rotational joints rigidly joined by a bar and each with 3 degrees of freedom always form a 5-dimensional system. We introduce the concept of reduced redundancy in a dependent set of projective Lines and argue that reduced redundancy in the axes of a body position increases injury risk. We apply this to a simple two-joint model of bowling in cricket and show by analysis of some experimental data that reduced redundancy around ball release is observed in some cases.

Hand Prosthetics

Chen, C-H, Naidu, D S, Perez-Gracia, A, Schoen, M P, 2009: Hybrid control strategy for five-fingered smart prosthetic hand. Proc. 48th IEEE Conf. Decision & Control / 28th Chinese Control Conf. (CDC/CCC 2009), 5102-5107.

This paper presents a hybrid of soft computing or control technique of adaptive neuro-fuzzy inference system (ANFIS) and hard computing or control technique of finite-time linear quadratic optimal control for the 14 degrees of freedom (DOFs), five-fingered smart prosthetic hand. In particular, ANFIS is used for inverse kinematics, and the optimal control is used for feedback linearized dynamics to minimize tracking error. The simulations of this hybrid controller, when compared with the proportional-integral-derivative ( PID) controller showed enhanced performance.

Chen, C-H, Naidu, D S, Perez-Gracia, A, Schoen, M P, 2009: A hybrid adaptive control strategy for a smart prosthetic hand. Proc. 31st Ann. Int'l Conf. IEEE Eng. Med. Biol. Soc., 5056-5059.

This paper presents a hybrid of a soft computing technique of adaptive neuro-fuzzy inference system ( ANFIS) and a hard computing technique of adaptive control for a two-dimensional movement of a prosthetic hand with a thumb and index finger. In particular, ANFIS is used for inverse kinematics, and the adaptive control is used for linearized dynamics to minimize tracking error. The simulations of this hybrid controller, when compared with the proportional-integral-derivative (PID) controller showed enhanced performance. Work is in progress to extend this methodology to a five-fingered, three-dimensional prosthetic hand.

Mathematical and Statistical Methods

Bayesian Analysis

Berger, J O, 1994: An overview of robust Bayesian analysis. Test, 3(1):5-124.

Robust Bayesian analysis is the study of the sensitivity of Bayesian answers to uncertain inputs. This paper seeks to provide an overview of the subject, one that is accessible to statisticians outside the field. Recent developments in the area are also reviewed, though with very uneven emphasis.

Berger, J O, Bernardo, J M, & Sun, D, 2009: Natural induction: An objective Bayesian approach. Revista de la Real Academia de Ciencias Exactas, Fisicas y Naturales. Serie A. Matematicas, 103(1):125-135.

The statistical analysis of a sample taken from a finite population is a classic problem for which no generally accepted objective Bayesian results seem to exist. Bayesian solutions to this problem may be very sensitive to the choice of the prior, and there is no consensus as to the appropriate prior to use.

This paper uses new developments in reference prior theory to justify and generalize Perks (1947) ([15]) 'rule of succession' - determining the probability that a new element from a population will have a property, given that all n previous elements from a random sample possessed the property - and to propose a new objective Bayesian solution to the 'law of natural induction' problem - determining the probability that all elements in a finite population have the property, given that all previous elements had the property.

The prior used for the first problem is the reference prior for an underlying hypergeometric probability model, a prior first suggested by Jeffreys (1946) ([10]) and recently justified on the basis of an exchangeability argument in Berger, Bernardo and Sun (2009) ([4]). The reference prior in the second problem arises as a modification to this prior that results from declaring the quantity of interest to be whether or not all the elements in the finite population have the property under scrutiny.

Girón, F J, & Moreno, E, 2009: Natural induction: An objective Bayesian approach - Comments. Revista de la Real Academia de Ciencias Exactas, Fisicas y Naturales. Serie A. Matematicas, 103(1):137-139.

Lindley, D V, 2009: Natural induction: An objective Bayesian approach - Comments. Revista de la Real Academia de Ciencias Exactas, Fisicas y Naturales. Serie A. Matematicas, 103(1):141.

Liseo, B, 2009: Natural induction: An objective Bayesian approach - Comments. Revista de la Real Academia de Ciencias Exactas, Fisicas y Naturales. Serie A. Matematicas, 103(1):143-144.

Raman, K, 2009: Natural induction: An objective Bayesian approach - Comments. Revista de la Real Academia de Ciencias Exactas, Fisicas y Naturales. Serie A. Matematicas, 103(1):145-148.

Rober, C P, 2009: Natural induction: An objective Bayesian approach - Comments. Revista de la Real Academia de Ciencias Exactas, Fisicas y Naturales. Serie A. Matematicas, 103(1):149-150.

Rueda, R, 2009: Natural induction: An objective Bayesian approach - Comments. Revista de la Real Academia de Ciencias Exactas, Fisicas y Naturales. Serie A. Matematicas, 103(1):151.

Zabell, S L, 2009: Natural induction: An objective Bayesian approach - Comments. Revista de la Real Academia de Ciencias Exactas, Fisicas y Naturales. Serie A. Matematicas, 103(1):153-155.

Berger, J O, Bernardo, J M, & Sun, D, 2009: Natural induction: An objective Bayesian approach - Rejoinder. Revista de la Real Academia de Ciencias Exactas, Fisicas y Naturales. Serie A. Matematicas, 103(1):157-159.

Berger, J O, Bernardo, J M, & Sun, D, 2009: The formal definition of reference priors. Ann. Statist., 37(2):905-938.

Reference analysis produces objective Bayesian inference, in the sense that inferential statements depend only on the assumed model and the available data, and the prior distribution used to make an inference is least informative in a certain information-theoretic sense. Reference priors have been rigorously defined in specific contexts and heuristically defined in general, but a rigorous general definition has been lacking. We produce a rigorous general definition here and then show how an explicit expression for the reference prior can be obtained under very weak regularity conditions. The explicit expression can be used to derive new reference priors both analytically and numerically.

Fernandes, A D, & Gloor, G B, 2010: Mutual information is critically dependent on prior assumptions: would the correct estimate of mutual information please identify itself? Bioinformatics, 26(9):1135-1139.

Motivation: Mutual information (MI) is a quantity that measures the dependence between two arbitrary random variables and has been repeatedly used to solve a wide variety of bioinformatic problems. Recently, when attempting to quantify the effects of sampling variance on computed values of MI in proteins, we encountered striking differences among various novel estimates of MI. These differences revealed that estimating the 'true' value of MI is not a straightforward procedure, and minor variations of assumptions yielded remarkably different estimates.

Results: We describe four formally equivalent estimates of MI, three of which explicitly account for sampling variance, that yield non-equal values of MI given exact frequencies. These MI estimates are essentially non-predictive of each other, converging only in the limit of implausibly large datasets. Lastly, we show that all four estimates are biologically reasonable estimates of MI, despite their disparity, since each is actually the Kullback-Leibler divergence between random variables conditioned on equally plausible hypotheses.

Conclusions: For sparse contingency tables of the type universally observed in protein coevolution studies, our results show that estimates of MI, and hence inferences about physical phenomena such as coevolution, are critically dependent on at least three prior assumptions. These assumptions are: (i) how observation counts relate to expected frequencies; (ii) the relationship between joint and marginal frequencies; and (iii) how non-observed categories are interpreted. In any biologically relevant data, these assumptions will affect the MI estimate as much or more-so than observed data, and are independent of uncertainty in frequency parameters.

Broad, C D, 1918: On the relation between induction and probability. Mind, 27(4):389-404.

In the present paper I propose to try to prove three points, which, if they can be established, are of great importance to the logic of inductive inference. They are (1) that unless inductive conclusions be expressed in terms of probability all inductive inference involves a formal fallacy; (2) that the degree of belief which we actually attach to the conclusions of well-established inductions cannot be justified by any known principle of probability, unless some further premise about the physical world be assumed; and (3) that it is extremely difficult to state this premise so that it shall be at once plausible and non-tautologous. I believe that the first two points can be rigorously established without entering in detail into the difficult problem of what it is that probability-fractions actually measure The third point is more doubtful, and I do not profess to have reached at present any satisfactory view about it.

Zabell, S L, 1989: The rule of succession. Erkenntnis, 31(2-3):283-321.

Laplace's rule of succession states, in brief, that if an event has occurred m times in succession, then the probability that it will occur again is (m+1)/(m+2). The rule of succession was the classical attempt to reduce certain forms of inductive inference - "pure inductions" (De Morgan) or "eductions" (W. E. Johnson) - to purely probabilistic terms. Subjected to varying forms of ridicule by Venn, Keynes, and many others, it often served as a touchstone for much broader issues about the nature and role of probability.
This paper will trace the evolution of the rule, from its original formulation at the hands of Bayes, Price, and Laplace, to its generalizations by the English philosopher W. E. Johnson, and its perfection at the hands of Bruno de Finetti. By following the debate over the rule, the criticisms of it that were raised and the defenses of it that were mounted, it is hoped that some insight will be gained into the achievements and limitations of the probabilistic attempt to explain induction. Our aim is thus not purely - or even primarily - historical in nature.

Zabell, S L, 1992: Predicting the unpredictable. Synthese, 90(2):205-232.

A major difficulty for currently existing theories of inductive inference involves the question of what to do when novel, unknown, or previously unsuspected phenomena occur. In this paper one particular instance of this difficulty is considered, the so-called sampling of species problem.
The classical probabilistic theories of inductive inference due to Laplace, Johnson, de Finetti, and Carnap adopt a model of simple enumerative induction in which there are a prespecified number of types or species which may be observed. But, realistically, this is often not the case. In 1838 the English mathematician Augustus De Morgan proposed a modification of the Laplacian model to accommodate situations where the possible types or species to be observed are not assumed to be known in advance; but he did not advance a justification for his solution.
In this paper a general philosophical approach to such problems is suggested, drawing on work of the English mathematician J. F. C. Kingman. It then emerges that the solution advanced by De Morgan has a very deep, if not totally unexpected, justification. The key idea is that although exchangeable random sequences are the right objects to consider when all possible outcome-types are known in advance, exchangeable random partitions are the right objects to consider when they are not. The result turns out to be very satisfying. The classical theory has several basic elements: a representation theorem for the general exchangeable sequence (the de Finetti representation theorem), a distinguished class of sequences (those employing Dirichlet priors), and a corresponding rule of succession (the continuum of inductive methods). The new theory has parallel basic elements: a representation theorem for the general exchangeable random partition (the Kingman representation theorem), a distinguished class of random partitions (the Poisson-Dirichlet process), and a rule of succession which corresponds to De Morgan's rule.

Hernando, A, & de Ledesma, L, 2009: A technique for dynamically measuring and modifying relevance while problem solving. Revista de la Real Academia de Ciencias Exactas, Fisicas y Naturales. Serie A. Matematicas, 103(1):111-124.

Although there is general agreement that efficiency of problem resolution is strongly related to the problem representation adopted, computer problem solvers have been traditionally designed to keep the same representation throughout the whole of the problem solving process. A system able to change representation whilst the actual problem solving process occurs has advantages over traditional ones, not only because a representation change can improve the efficiency of problem resolution (as already proven through much research), but also because the choice of the most suitable representation may be decisively enhanced after learning about the problem during its resolution process. A natural and interesting way of performing representation changes is related to detecting irrelevant elements which can be removed out of the problem representation. In this paper, we deal formally with a new technique for assigning and changing the relevance of the elements involved in the representation during the problem resolution, on behalf of their respective importance so as to actually solve the problem.

Berger, J O, 2003: Could Fisher, Jeffreys and Neyman have agreed on testing? Statist. Sci., 18(1)1-31.

Ronald Fisher advocated testing using p-values; Harold Jeffreys proposed use of objective posterior probabilities of hypotheses; and Jerzy Neyman recommended testing with fixed error probabilities. Each was quite critical of the other approaches. Most troubling for statistics and science is that the three approaches can lead to quite different practical conclusions.

We focus on discussion of the conditional frequentist approach to testing, which is argued to provide the basis for a methodological unification of the approaches of Fisher, Jeffreys and Neyman. The idea is to follow Fisher, in using p-values to define the 'strength of evidence' in data, and to follow his approach of conditioning on strength of evidence; then follow Neyman by computing Type I and Type II error probabilities, but do so conditional on the strength of evidence in the data. The resulting conditional frequentist error probabilities equal the objective posterior probabilities of the hypotheses advocated by Jeffreys.

Sellke, T, Bayarri, M J, & Berger, J O, 2001: Calibration of P-values for testing precise null hypotheses. The American Statistician, 55(1):62-71.

P-values are the most commonly used tool to measure evidence against a hypothesis or hypothesized model. Unfortunately, they are often incorrectly viewed as an error probability for rejection of the hypothesis or, even worse, as the posterior probability that the hypothesis is true. The fact that these interpretations can be completely misleading when testing precise hypotheses is first reviewed, through consideration of two revealing simulations. Then two calibrations of a P-value are developed, the first being interpretable as odds and the second as either a (conditional) frequentist error probability or as the posterior probability of the hypothesis.

Aitkin, M, Boys, R J, & Chadwick, T, 2005: Bayesian point null hypothesis testing via the posterior likelihood ratio. Statistics & Computing, 15(3):217-230.

This paper gives an exposition of the use of the posterior likelihood ratio for testing point null hypotheses in a fully Bayesian framework. Connections between the frequentist P-value and the posterior distribution of the likelihood ratio are used to interpret and calibrate P-values in a Bayesian context, and examples are given to show the use of simple posterior simulation methods to provide Bayesian tests of common hypotheses.

Aitkin, M, 1997: The calibration of P-values, posterior Bayes factors and the AIC from the posterior distribution of the likelihood. Statistics & Computing, 7(4):253-261.

The posterior distribution of the likelihood is used to interpret the evidential meaning of P-values, posterior Bayes factors and Akaike's information criterion when comparing point null hypotheses with composite alternatives. Asymptotic arguments lead to simple re-calibrations of these criteria in terms of posterior tail probabilities of the likelihood ratio. (Prior) Bayes factors cannot be calibrated in this way as they are model -specific.

Cooleyd, C A, & MacEachern, S N, 1999: Prior elicitation in the classification problem. Can. J. Statist., 27(2)299-313.

Results are developed concerning the asymptotic behaviour of the Bayes classification rule as the number of unclassified observations grows without bound. It is shown that unclassified observations serve only to estimate the individual population parameters in an unlabeled sense and do not provide information about the labels that are attached to the populations. Prior construction is approached through investigation of prior odds over regions of the joint parameter space (across all populations) deemed likely to contain the true joint parameter vector. It is shown that consideration of these prior odds can lead to more robust a posteriori classification of individual observations.

Loftus, G R, & Masson, M E J, 1994: Using confidence intervals in within-subject designs. Psychonomic Bulletin & Review, 1(4):476-490.

We argue that to best comprehend many data sets, plotting judiciously selected sample statistics with associated confidence intervals can usefully supplement, or even replace, standard hypothesis-testing procedures. We note that most social science statistics textbooks limit discussion of confidence intervals to their use in between-subject designs. Our central purpose in this article is to describe how to compute an analogous confidence interval that can be used in within-subject designs. This confidence interval rests on the reasoning that because between-subject variance typically plays no role in statistical analyses of within-subject designs, it can legitimately be ignored; hence, an appropriate confidence interval can be based on the standard within-subject error term-that is, on the variability due to the subject × condition interaction. Computation of such a confidence interval is simple and is embodied in Equation 2 on p. 482 of this article. This confidence interval has two useful properties. First, it is based on the same error term as is the corresponding analysis of variance, and hence leads to comparable conclusions. Second, it is related by a known factor (v2) to a confidence interval of the difference between sample means; accordingly, it can be used to infer the faith one can put in some pattern of sample means as a reflection of the underlying pattern of population means. These two properties correspond to analogous properties of the more widely used between-subject confidence interval.

Verdinelli, I, & Wasserman, L, 1991: Bayesian analysis of outlier problems using the Gibbs sampler. Statistics & Computing, 1(2):105-117.

We consider the Bayesian analysis of outlier models. We show that the Gibbs sampler brings considerable conceptual and computational simplicity to the problem of calculating posterior marginals. Although other techniques for finding posterior marginals are available, the Gibbs sampling approach is notable for its ease of implementation. Allowing the probability of an outlier to be unknown introduces an extra parameter into the model but this turns out to involve only minor modification to the algorithm. We illustrate these ideas using a contaminated Gaussian distribution, at-distribution, a contaminated binomial model and logistic regression.

Hypothesis Testing - Statistical Power and Bonferroni

Nakagawa, S, 2004: A farewell to Bonferroni: the problems of low statistical power and publication bias. Behavioral Ecology, 15(6):1044-1045.

Recently, Jennions and Møller (2003) carried out a meta-analysis on statistical power in the field of behavioral ecology and animal behavior, reviewing 10 leading journals including Behavioral Ecology. Their results showed dismayingly low average statistical power (note that a meta-analytic review of statistical power is different from post hoc power analysis as criticized in Hoenig and Heisey, 2001). The statistical power of a null hypothesis (Ho) significance test is the probability that the test will reject Ho when a research hypothesis (Ha) is true. Knowledge of effect size is particularly important for statistical power analysis (for statistical power analysis, see Cohen, 1988; Nakagawa and Foster, in press). There are many kinds of effect size measures available (e.g., Pearson's r, Cohen's d, Hedges's g), but most of these fall into one of two major types, namely the r family and the d family (Rosenthal, 1994). The r family shows the strength of relationship between two variables while the d family shows the size of difference between two variables. As a benchmark for research planning and evaluation, Cohen (1988) proposed 'conventional' values for small, medium, and large effects: r =.10,.30, and.50 and d =.20,.50, and.80, respectively (in the way that p values of.05,.01, and.001 are conventional points, although these conventional values of effect size have been criticized; e.g., Rosenthal et al., 2000).

Garamszegi, LZ, 2006: Comparing effect sizes across variables: generalization without the need for Bonferroni correction. Behavioral Ecology, 17(4):682-687.

Studies in behavioral ecology often investigate several traits and then apply multiple statistical tests to discover their pairwise associations. Traditionally, such approaches require the adjustment of individual significance levels because as more statistical tests are performed the greater the likelihood that Type I errors are committed (i.e., rejecting H0 when it is true) (Rice 1989). Bonferroni correction that lowers the critical P values for each particular test based on the number of tests to be performed is frequently used to reduce problems associated with multiple comparisons (Cabin and Mitchell 2000). However, this procedure dramatically increases the risk of committing Type II errors as it results in a high risk of not rejecting a H0 when it is false. To reach 80% statistical power, it is necessary to have huge sample sizes to detect medium (r = 0.3 or d = 0.5; sensu Cohen 1988) or small (r = 0.1 or d = 0.2; sensu Cohen 1988) strength effects (e.g., say N = 128 or N = 788, respectively, for a 2-sample t-test), but sample size is often limited when studying behavior.

Westfall, P H, Johnson, W O, & Utts, J M, 1997: A Bayesian perspective on the Bonferroni adjustment. Biometrika, 84(2):419-427.

Bayes/frequentist correspondences between the p-value and the posterior probability of the null hypothesis have been studied in univariate hypothesis testing situations. This paper extends these comparisons to multiple testing and in particular to the Bonferroni multiple testing method, in which p-values are adjusted by multiplying by k, the number of tests considered. In the Bayesian setting, prior assessments may need to be adjusted to account for multiple hypotheses, resulting in corresponding adjustments to the posterior probabilities. Conditions are given for which the adjusted posterior probabilities roughly correspond to Bonferroni adjusted p-values.

Cohen, J, 1992: A power primer. Psychol. Bull., 112(1):155-159.

One possible reason for the continued neglect of statistical power analysis in research in the behavioral sciences is the inaccessibility of or difficulty with the standard material. A convenient, although not comprehensive, presentation of required sample sizes is provided. Effect-size indexes and conventional values for these are given for operationally defined small, medium, and large effects. The sample sizes necessary for .80 power to detect effects at these levels are tabled for 8 standard statistical tests: (1) the difference between independent means, (2) the significance of a product-moment correlation, (3) the difference between independent rs, (4) the sign test, (5) the difference between independent proportions, (6) chi-square tests for goodness of fit and contingency tables, (7) 1-way analysis of variance (ANOVA), and (8) the significance of a multiple or multiple partial correlation.

Hoenig, J M, & Heisey, D M, 2001: The abuse of power. The American Statistician, 55(1):19-24.

It is well known that statistical power calculations can be valuable in planning an experiment. There is also a large literature advocating that power calculations be made whenever one performs a statistical test of a hypothesis and one obtains a statistically nonsignificant result. Advocates of such post-experiment power calculations claim the calculations should be used to aid in the interpretation of the experimental results. This approach, which appears in various forms, is fundamentally flawed. We document that the problem is extensive and present arguments to demonstrate the flaw in the logic.

Sphericity - Greenhouse-Geisser, Huynh-Feldt, and Box

Greenhouse, S W, & Geisser, S, 1959: On methods in the analysis of profile data. Psychometrika, 24(2):95-112.

This paper is concerned with methods for analyzing quantitative, non-categorical profile data, e.g., a battery of tests given to individuals in one or more groups. It is assumed that the variables have a multinormal distribution with an arbitrary variance-covariance matrix. Approximate procedures based on classical analysis of variance are presented, including an adjustment to the degrees of freedom resulting in conservativeF tests. These can be applied to the case where the variance-covariance matrices differ from group to group. In addition, exact generalized multivariate analysis methods are discussed. Examples are given illustrating both techniques.

Geisser, S, 1965: A Bayes approach for combining correlated estimates. J. Amer. Statist. Assoc., 60(310):602-607.

A Bayes solution is supplied for an estimation problem involving a sample from a multivariate normal population having an arbitrary unknown covariance matrix, but a vector mean whose components are all equal. Assuming that a particular unnormed prior density is a convenient expression for displaying prior ignorance, it is then demonstrated that a posterior interval for this common mean can be based on Student's t distribution. If prior information can be conveniently represented by a natural conjugate prior density, the posterior interval will also depend on Student's t. An extension is made to the case of estimating the constant difference between two parallel profiles.

Geisser, S, & Kappenman, R F, 1971: A posterior region for parallel profile differentials. Psychometrika, 36(1):71-78.

A simple Bayesian solution for the joint estimation of several parallel profile differentials for multivariate normal populations is obtained. This generalizes a previous result for a single profile differential.

Huynh, H S, & Feldt, L, 1976: Estimation of the Box correction for degrees of freedom from sample data in randomized block and split-plot designs. J. Educational Statistics, 1(1):69-82.

It has been suggested that when the variance assumptions of a repeated measures ANOVA are not met, the df of the mean square ratio should be adjusted by the sample estimate of the Box correction factor, epsilon. This procedure works well when epsilon is low, but the estimate is seriously biased when this is not the case. An alternate estimate is proposed which is shown by Monte Carlo methods to be less biased for moderately large epsilon.

Functional Analysis of Variance

Ramsay, J O, 1982: When the data are functions. Psychometrika, 47(4):379-396.

A datum is often a continuous function x(t) of a variable such as time observed over some interval. One or more such functions are observed for each subject or unit of observation. The extension of classical data analytic techniques designed for p-variate observations to such data is discussed. The essential step is the expression of the classical problem in the language of functional analysis, after which the extension to functions is a straightforward matter. A schematic device called the duality diagram is a very useful tool for describing an analysis and for suggesting new possibilities. Least squares approximation, descriptive statistics, principal components analysis, and canonical correlation analysis are discussed within this broader framework.

Guo, W, 2002: Functional mixed effects models. Biometrics, 58(1):121-128.

In this article, a new class of functional models in which smoothing splines are used to model fixed effects as well as random effects is introduced. The linear mixed effects models are extended to non-parametric mixed effects models by introducing functional random effects, which are modeled as realizations of zero -mean stochastic processes. The fixed functional effects and the random functional effects are modeled in the same functional space, which guarantee the population-average and subject-specific curves have the same smoothness property. These models inherit the flexibility of the linear mixed effects models in handling complex designs and correlation structures, can include continuous covariates as well as dummy factors in both the fixed or random design matrices, and include the nested curves models as special cases. Two estimation procedures are proposed. The first estimation procedure exploits the connection between linear mixed effects models and smoothing splines and can be fitted using existing software. The second procedure is a sequential estimation procedure using Kalman filtering. This algorithm avoids inversion of large dimensional matrices and therefore can be applied to large data sets. A generalized maximum likelihood (GML) ratio test is proposed for inference and model selection. An application to comparison of cortisol profiles is used as an illustration.

Hall, P, & Heckman, N E, 2002: Estimating and depicting the structure of a distribution of random runctions. Biometrika, 89(1):145-158.

We suggest a nonparametric approach to making inference about the structure of distributions in a potentially infinite-dimensional space, for example a function space, and displaying information about that structure. It is suggested that the simplest way of presenting the structure is through modes and density ascent lines, the latter being the projections into the sample space of the curves of steepest ascent up the surface of a functional-data density. Modes are always points in the sample space, and ascent lines are always one-parameter structures, even when the sample space is determined by an infinite number of parameters. They are therefore relatively easily depicted. Our methodology is based on a functional form of an iterative data-sharpening algorithm.

Abramovich, F, & Angelini, C, 2006: Testing in mixed-effects FANOVA models. J. Statistical Planning & Inference, 136(12):4326-4348.

We consider the testing problem in the mixed-effects functional analysis of variance models. We develop asymptotically optimal (minimax) testing procedures for testing the significance of functional global trend and the functional fixed effects based on the empirical wavelet coefficients of the data. Wavelet decompositions allow one to characterize various types of assumed smoothness conditions on the response function under the nonparametric alternatives. The distribution of the functional random-effects component is defined in the wavelet domain and captures the sparseness of wavelet representation for a wide variety of functions. The simulation study presented in the paper demonstrates the finite sample properties of the proposed testing procedures. We also applied them to the real data from the physiological experiments.

Barrientos-Marina, F, Ferraty, F, & Vieu, P, 2010: Locally modelled regression and functional data. J. Nonparametric Statistics, 22(5):617-632.

The general framework of this paper deals with the nonparametric regression of a scalar response on a functional variable (i.e. one observation can be a curve, surface, or any other object lying into an infinite -dimensional space). This paper proposes to model local behaviour of the regression operator (i.e. the link between a scalar response and an explanatory functional variable). To this end, one introduces a functional approach in the same spirit as local linear ideas in nonparametric regression. The main advantage of this functional local method is to propose an explicit expression of a kernel-type estimator which makes its computation easy and fast while keeping good predictive performance. Asymptotic properties are stated, and a functional data set illustrates the good behaviour of this functional locally modelled regression method.

Cuesta-Albertos, J A, & Febrero-Bande, M, 2010: A simple multiway ANOVA for functional data. Test, 19(3):537-557.

We propose a procedure to test complicated ANOVA designs for functional data. The procedure is effective, flexible, easy to compute and does not require a heavy computational effort. It is based on the analysis of randomly chosen one-dimensional projections. The paper contains some theoretical results as well as some simulations and the analysis of some real data sets. Functional data include multidimensional data, so the paper contains a comparison between the proposed procedure and some usual MANOVA tests.

Spline Fitting

Wegman, E J, & Wright, I W, 1983: Splines in statistics. J. Amer. Statist. Assoc., 78(382):351-365.

This is a survey article that attempts to synthesize a broad variety of work on splines in statistics. Splines are presented as a nonparametric function estimating technique. After a general introduction to the theory of interpolating and smoothing splines, splines are treated in the nonparametric regression setting. The method of cross-validation for choosing the smoothing parameter is discussed and the general multivariate regression/surface estimation problem is addressed. An extensive discussion of splines as nonparametric density estimators is followed by a discussion of their role in time series analysis. A comparison of the spline and isotonic regression methodologies leads to a formulation of a hybrid estimator. The closing section provides a brief overall summary and formulates a number of open/unsolved problems relating to splines in statistics.

Nychka, D, 1995: Splines as local smoothers. Ann. Statist., 23(4):1175-1197.

A smoothing spline is a nonparametric curve estimate that is defined as the solution to a minimization problem. One problem with this representation is that it obscures the fact that a spline, like most other nonparametric estimates, is a local, weighted average of the observed data. This property has been used extensively to study the limiting properties of kernel estimates and it is advantageous to apply similar techniques to spline estimates. Although equivalent kernels have been identified for a smoothing spline, these functions are either not accurate enough for asymptotic approximations or are restricted to equally spaced points. This paper extends this previous work to understand a spline estimate's local properties. It is shown that the absolute value of the spline weight function decreases exponentially away from its center. This result is not asymptotic. The only requirement is that the empirical distribution of the observation points be sufficiently close to a continuous distribution with a strictly positive density function. These bounds are used to derive the asymptotic form for the bias and variance of a first order smoothing spline estimate. The arguments leading to this result can be easily extended to higher order splines.

Hanson, M H, & Kooperberg, C, 2002: Spline adaptation in extended linear models, with discussion. Statistical Science, 17(1):2-51.

In many statistical applications, nonparametric modeling can provide insights into the features of a dataset that are not obtainable by other means. One successful approach involves the use of (univariate or multivariate) spline spaces. As a class, these methods have inherited much from classical tools for parametric modeling. For example, stepwise variable selection with spline basis terms is a simple scheme for locating knots (breakpoints) in regions where the data exhibit strong, local features. Similarly, candidate knot configurations (generated by this or some other search technique), are routinely evaluated with traditional selection criteria like AIC or BIC. In short, strategies typically applied in parametric model selection have proved useful in constructing flexible, low-dimensional models for nonparametric problems. Until recently, greedy, stepwise procedures were most frequently suggested in the literature. Research into Bayesian variable selection, however, has given rise to a number of new spline-based methods that primarily rely on some form of Markov chain Monte Carlo to identify promising knot locations. In this paper, we consider various alternatives to greedy, deterministic schemes, and present a Bayesian framework for studying adaptation in the context of an extended linear model (ELM). Our major test cases are Logspline density estimation and (bivariate) Triogram regression models. We selected these because they illustrate a number of computational and methodological issues concerning model adaptation that arise in ELMs.

Silverman, B W, 1985: Some aspects of the spline smoothing approach to non-parametric regression curve fitting. J. Roy. Statist. Soc. Ser. B, 47(1):1-52.

Non-parametric regression using cubic splines is an attractive, flexible and widely- applicable approach to curve estimation. Although the basic idea was formulated many years ago, the method is not as widely known or adopted as perhaps it should be. The topics and examples discussed in this paper are intended to promote the understanding and extend the practicability of the spline smoothing methodology. Particular subjects covered include the basic principles of the method; the relation with moving average and other smoothing methods; the automatic choice of the amount of smoothing; and the use of residuals for diagnostic checking and model adaptation. The question of providing inference regions for curves-and for relevant properties of curves-is approached via a finite-dimensional Bayesian formulation.

Winsberg, S, & Ramsay, J O, 1980: Monotonic transformations to additivity using splines. Biometrika, 67(3):669-674.

A class of monotonic integral transformations derived from B-splines is fitted to the independent and dependent variables in multiple regression so that the resulting additive relationship is optimized. The fit is achieved by maximizing a log likelihood criterion with inequality constraints on the parameters. Some examples of the analysis of artificial and real data are offered. An algorithm which works reliably is outlined.

Winsberg, S, & Ramsay, J O, 1981: Analysis of pairwise preference data using integrated B-splines. Psychometrika, 46(2):171-186.

Pairwise preference data are represented as a monotone integral transformation of difference on the underlying stimulus-object or utility scale. The class of monotone transformations considered is that in which the kernel of the integral is a linear combination of B-splines. Two types of data are analyzed: binary and continuous. The parameters of the transformation and the underlying scale values or utilities are estimated by maximum likelihood with inequality constraints on the transformation parameters. Various hypothesis tests and interval estimates are developed. Examples of artificial and real data are presented.

Ratto, M, & Pagano, A, 2010: Using recursive algorithms for the efficient identification of smoothing spline ANOVA models. Adv. Statist. Anal., 94(4):367-388.

In this paper we present a unified discussion of different approaches to the identification of smoothing spline analysis of variance (ANOVA) models: (i) the "classical" approach (in the line of Wahba in Spline Models for Observational Data, 1990; Gu in Smoothing Spline ANOVA Models, 2002; Storlie et al. in Stat. Sin., 2011) and (ii) the State-Dependent Regression (SDR) approach of Young in Nonlinear Dynamics and Statistics (2001). The latter is a nonparametric approach which is very similar to smoothing splines and kernel regression methods, but based on recursive filtering and smoothing estimation (the Kalman filter combined with fixed interval smoothing). We will show that SDR can be effectively combined with the "classical" approach to obtain a more accurate and efficient estimation of smoothing spline ANOVA models to be applied for emulation purposes. We will also show that such an approach can compare favorably with kriging.

Yue, R-X, & Hickernell, F J, 2002: Designs for smoothing spline ANOVA models. Metrika, 55(3):161-176.

Smoothing spline estimation of a function of several variables based on an analysis of variance decomposition (SS-ANOVA) is one modern nonparametric technique. This paper considers the design problem for specific types of SS-ANOVA models. As criteria for choosing the design points, the integrated mean squared error (IMSE) for the SS-ANOVA estimate and its asymptotic approximation are derived based on the correspondence between the SS-ANOVA model and the random effects model with a partially improper prior. Three examples for additive and interaction spline models are provided for illustration. A comparison of the asymptotic designs, the 2d factorial designs, and the glp designs is given by numerical computation.

de Boor, C R, 1986: B(asic)-spline basics. Technical Report, Accession Number ADA172773, University of Wisconsin.

This report contains the lecture notes for the first of four lectures which comprise the course entitled The extension of B-spline curve algorithms to surfaces given at SIGRAPH'86. It is an elaboration and extension of the MRC report 2896 by de Boor and Hollig, in which the basic B-spline theory is developed from the reccurence relation rather than the original definition in terms of divided differences of the truncated power. This avoids what, to the people in CAGD, amounts to a detour through the theory of divided differences. Somewhat surprisingly. The resulting development is no longer than the standard one, and in some respects seems even more direct. It does bring to the fore the dual functionals and stresses the point that B-splines are best treated in terms of their linear span.

Piegl, L, 1991: On NURBS: a survey. IEEE Computer Graphics and Applications, 11(1):55-71.

Nonuniform rational B-spline (NURBS) curves and surfaces, which are based on rational and B-splines, are defined. The important characteristics of NURBS that have contributed to their wide acceptance as standard tools for geometry representation and design are summarized. Their application to representing conic sections and commonly used surfaces, designing curves and surfaces, and modifying shapes is examined.

Curve Registration

Ramsay, J O, 2000: Functional components of variation in handwriting. J. Am. Statist. Assoc., 95(1):9-15.

Functional data analysis techniques are used to analyze a sample of handwriting in Chinese. The goals are (a) to identify a differential equation that satisfactorily models the data's dynamics, and (b) to use the model to classify handwriting samples taken from differential individuals. After preliminary smoothing and registration steps, a second-order linear differential equation, for which the forcing function is small, is found to provide a good reconstruction of the original script records. The equation is also able to capture a substantial amount of the variation in the scripts across replication. The cross-validated classification process is 100% effective for the samples analyzed.

Ramsay, J O, & Li, X, 1998: Curve registration. J. Roy. Statist. Soc. Ser. B, 60(2):351-363.

Functional data analysis involves the extension of familiar statistical procedures such as principal components analysis, linear modelling and canonical correlation analysis to data where the raw observation x_i is a function. An essential preliminary to a functional data analysis is often the registration or alignment of salient curve features by suitable monotone transformations h_i of the argument t, so that the actual analyses are carried out on the values x_i(h_i(t)). This is referred to as dynamic time warping in the engineenng literature. In effect, this conceptualizes variation among functions as being composed of two aspects: horizontal and vertical, or domain and range. A nonparametric function estimation technique is described for identifying the smooth monotone transformations h_i and is illustrated by data analyses. A second-order linear stochastic differential equation is proposed to model these components of variation.

Ramsay, J O, Wang, X, & Flanagan, R, 1995: A functional data analysis of the pinch force of human fingers. Applied Statistics, 44(1):17-30.

The ability of the human thumb and forefinger to adapt the pinch force to the static and dynamic characteristics of the object being grasped is one of the marvels of human physiology. We analyse a sample of records of the force applied during a brief squeeze by functional data analysis tech- niques in which familiar statistical concepts are adapted to observations that are functional in character. Except for scale, a graph of these force impulses closely resembles a log-normal density function, and this has a plausible physiological rationale. Specially adapted smoothing spline approximations along with a functional version of principal components analysis reveal that the residual variation is essentially one dimensional in structure, and that the force functions can be described by a simple linear differential equation incorporating the effects of drag or viscosity in the joints and muscles involved.

Ramsay, J O, Munhall, K G, Gracco, V L, & Ostry, D J, 1996: Functional data analyses of lip motion. J. Acoust. Soc. Am., 99(6):3718-3727.

The vocal tract's motion during speech is a complex patterning of the movement of many different articulators according to many different time functions. Understanding this myriad of gestures is important to a number of different disciplines including automatic speech recognition, speech and language pathologies, speech motor control, and experimental phonetics. Central issues are the accurate description of the shape of the vocal tract and determining how each articulator contributes to this shape. A problem facing all of these research areas is how to cope with the multivariate data from speech production experiments. In this paper techniques are described that provide useful tools for describing multivariate functional data such as the measurement of speech movements. The choice of data analysis procedures has been motivated by the need to partition the articulator movement in various ways: end effects separated from shape effects, partitioning of syllable effects, and the splitting of variation within an articulator site from variation from between sites. The techniques of functional data analysis seem admirably suited to the analyses of phenomena such as these. Familiar multivariate procedures such as analysis of variance and principal components analysis have their functional counterparts, and these reveal in a way more suited to the data the important sources of variation in lip motion. Finally, it is found that the analyses of acceleration were especially helpful in suggesting possible control mechanisms. The focus is on using these speech production data to understand the basic principles of coordination. However, it is believed that the tools will have a more general use.

Time Warping

Gervini, D, & Gasser, T, 2004: Self-modeling warping functions. J. Roy. Statist. Soc. Ser. B, 66(4):959-971.

The paper introduces a semiparametric model for functional data. The warping functions are assumed to be linear combinations of q common components, which are estimated from the data (hence the name 'self -modelling'). Even small values of q provide remarkable model flexibility, comparable with nonparametric methods. At the same time, this approach avoids overfitting because the common components are estimated combining data across individuals. As a convenient by-product, component scores are often interpretable and can be used for statistical inference (an example of classification based on scores is given).

Complete Books

Functional Data Analysis - Ramsay & Silverman, 2005

Ramsay, J O, & Silverman, B W, 2005: Front matter. Functional Data Analysis, 2nd ed., Springer Series in Statistics, i-xix, Springer.

Ramsay, J O, & Silverman, B W, 2005: Introduction. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 1-18, Springer.

Ramsay, J O, & Silverman, B W, 2005: Tools for exploring functional data. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 19-35, Springer.

Ramsay, J O, & Silverman, B W, 2005: From functional data to smooth functions. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 37-58, Springer.

Ramsay, J O, & Silverman, B W, 2005: Smoothing functional data by least squares. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 59-79, Springer.

Ramsay, J O, & Silverman, B W, 2005: Smoothing functional data with a roughness penalty. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 81-109, Springer.

Ramsay, J O, & Silverman, B W, 2005: Constrained functions. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 111-126, Springer.

Ramsay, J O, & Silverman, B W, 2005: The registration and display of functional data. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 127-145, Springer.

Ramsay, J O, & Silverman, B W, 2005: Principal components analysis for functional data. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 147-172, Springer.

Ramsay, J O, & Silverman, B W, 2005: Regularized principal components analysis. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 173-185, Springer.

Ramsay, J O, & Silverman, B W, 2005: Principal components analysis of mixed data. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 187-199, Springer.

Ramsay, J O, & Silverman, B W, 2005: Canonical correlation and discriminant analysis. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 201-215, Springer.

Ramsay, J O, & Silverman, B W, 2005: Functional linear models. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 217-222, Springer.

Ramsay, J O, & Silverman, B W, 2005: Modelling functional responses with multivariate covariates. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 223-245, Springer.

Ramsay, J O, & Silverman, B W, 2005: Functional responses, functional covariates and the concurrent model. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 247-259, Springer.

Ramsay, J O, & Silverman, B W, 2005: Functional linear models for scalar responses. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 261-277, Springer.

Ramsay, J O, & Silverman, B W, 2005: Functional linear models for functional responses. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 279-296, Springer.

Ramsay, J O, & Silverman, B W, 2005: Derivatives and functional linear models. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 297-306, Springer.

Ramsay, J O, & Silverman, B W, 2005: Differential equations and operators. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 307-326, Springer.

Ramsay, J O, & Silverman, B W, 2005: Fitting differential equations to functional data: principal differential analysis. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 327-348, Springer.

Ramsay, J O, & Silverman, B W, 2005: Green's functions and reproducing kernels. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 349-357, Springer.

Ramsay, J O, & Silverman, B W, 2005: More general roughness penalties. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 359-377, Springer.

Ramsay, J O, & Silverman, B W, 2005: Some perspectives on FDA. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 379-384, Springer.

Ramsay, J O, & Silverman, B W, 2005: Back matter. Functional Data Analysis, 2nd ed., Springer Series in Statistics, 385-426, Springer.

Applied Functional Data Analysis - Ramsay & Silverman, 2002

Ramsay, J O, & Silverman, B W, 2002: Front matter. Applied Functional Data Analysis: Methods and Case Studies, Springer Series in Statistics, i-x, Springer.

Ramsay, J O, & Silverman, B W, 2002: Introduction. Applied Functional Data Analysis: Methods and Case Studies, Springer Series in Statistics, 1-16, Springer.

Ramsay, J O, & Silverman, B W, 2002: Life course data in criminology. Applied Functional Data Analysis: Methods and Case Studies, Springer Series in Statistics, 17-40, Springer.

Ramsay, J O, & Silverman, B W, 2002: The nondurable goods index. Applied Functional Data Analysis: Methods and Case Studies, Springer Series in Statistics, 41-56, Springer.

Ramsay, J O, & Silverman, B W, 2002: Bone shapes from a paleopathology study. Applied Functional Data Analysis: Methods and Case Studies, Springer Series in Statistics, 57-67, Springer.

Ramsay, J O, & Silverman, B W, 2002: Modeling reaction-time distributions. Applied Functional Data Analysis: Methods and Case Studies, Springer Series in Statistics, 69-81, Springer.

Ramsay, J O, & Silverman, B W, 2002: Zooming in on human growth. Applied Functional Data Analysis: Methods and Case Studies, Springer Series in Statistics, 83-99, Springer.

Ramsay, J O, & Silverman, B W, 2002: Time warping handwriting and weather records. Applied Functional Data Analysis: Methods and Case Studies, Springer Series in Statistics, 101-114, Springer.

Ramsay, J O, & Silverman, B W, 2002: How do bone shapes indicate arthritis? Applied Functional Data Analysis: Methods and Case Studies, Springer Series in Statistics, 115-130, Springer.

Ramsay, J O, & Silverman, B W, 2002: Functional models for test items. Applied Functional Data Analysis: Methods and Case Studies, Springer Series in Statistics, 131-143, Springer.

Ramsay, J O, & Silverman, B W, 2002: Predicting lip acceleration from electromyography. Applied Functional Data Analysis: Methods and Case Studies, Springer Series in Statistics, 145-156, Springer.

Ramsay, J O, & Silverman, B W, 2002: The dynamics of handwriting printed characters. Applied Functional Data Analysis: Methods and Case Studies, Springer Series in Statistics, 157-170, Springer.

Ramsay, J O, & Silverman, B W, 2002: A differential equation for juggling. Applied Functional Data Analysis: Methods and Case Studies, Springer Series in Statistics, 171-181, Springer.

Ramsay, J O, & Silverman, B W, 2002: Back matter. Applied Functional Data Analysis: Methods and Case Studies, Springer Series in Statistics, 183-190, Springer.

Functional Data Analysis with R and MatLab - Ramsay, Hooker, & Graves, 2009

Ramsay, J O, Hooker, G, & Graves, S, 2009: Front matter. Functional Data Analysis with R and MatLab, i-x, Springer.

Ramsay, J O, Hooker, G, & Graves, S, 2009: Introduction to functional data analysis. Functional Data Analysis with R and MatLab, 1-19, Springer.

Ramsay, J O, Hooker, G, & Graves, S, 2009: Essential comparisons of the MatLab and R languages. Functional Data Analysis with R and MatLab, 21-27, Springer.

Ramsay, J O, Hooker, G, & Graves, S, 2009: How to specify basis systems for building functions. Functional Data Analysis with R and MatLab, 29-44, Springer.

Ramsay, J O, Hooker, G, & Graves, S, 2009: How to build functional data objects. Functional Data Analysis with R and MatLab, 45-58, Springer.

Ramsay, J O, Hooker, G, & Graves, S, 2009: Smoothing: computing curves from noisy data. Functional Data Analysis with R and MatLab, 59-82, Springer.

Ramsay, J O, Hooker, G, & Graves, S, 2009: Descriptions of functional data. Functional Data Analysis with R and MatLab, 83-97, Springer.

Ramsay, J O, Hooker, G, & Graves, S, 2009: Exploring variation: functional principal and canonical components analysis. Functional Data Analysis with R and MatLab, 99-115, Springer.

Ramsay, J O, Hooker, G, & Graves, S, 2009: Registration: aligning features for samples of curves. Functional Data Analysis with R and MatLab, 117-130, Springer.

Ramsay, J O, Hooker, G, & Graves, S, 2009: Functional linear models for scalar responses. Functional Data Analysis with R and MatLab, 131-146, Springer.

Ramsay, J O, Hooker, G, & Graves, S, 2009: Linear models for functional responses. Functional Data Analysis with R and MatLab, 147-177, Springer.

Ramsay, J O, Hooker, G, & Graves, S, 2009: Functional models and dynamics. Functional Data Analysis with R and MatLab, 179-195, Springer.

Ramsay, J O, Hooker, G, & Graves, S, 2009: Back matter. Functional Data Analysis with R and MatLab, 197-208, Springer.

Human-Like Biomechanics - Ivancevic & Ivancevic, 2006

Ivancevic, V G, & Ivancevic, T T, 2006: 00 - Front Matter. Human-Like Biomechanics, i-xviii, Springer.

Ivancevic, V G, & Ivancevic, T T, 2006: 01 - Introduction. Human-Like Biomechanics, 5-62, Springer.

Human-like biomechanics is a modern scientific approach to human-like motion dynamics and control. Its human perspective has been developed in the work of the present authors. The dynamics of human motion is extremely complex, multi-dimensional, highly nonlinear and hierarchical. Human skeleton has more than two hundred rigid bones, connected by rotational joints, witch have up to three axes of rotation. Nevertheless, in classical biomechanics the main analytical tool was translational vector geometry (see Figure 1.1). The skeleton is driven by a synergistic action of its 640 skeletal muscles. Each of these muscles has its own excitation and contraction dynamics, in which neural action potentials are transformed into muscular force vectors.

Ivancevic, V G, & Ivancevic, T T, 2006: 02 - Geometric Basis of Human-Like Biomechanics. Human-Like Biomechanics, 63-189, Springer.

The core of geometrodynamics is the concept of the manifold, the stage where our covariant force law, Fi = mgijaj,works. To get some dynamical feeling before we dive into more serious geometry, let us consider a simple 3DOF biomechanical system (e.g., a representative point of the center of mass of the human body) determined by three generalized coordinates qi = {q1, q2, q3}. There is a unique way to represent this system as a 3D manifold, such that to each point of the manifold there corresponds a definite configuration of the biomechanical system with coordinates qi; therefore, we have a geometric representation of the configurations of our biomechanical system. For this reason, the manifold is called the configuration manifold. If the biomechanical system moves in any way, its coordinates are given as the functions of the time. Thus, the motion is given by equations of the form: qi = qi(t). As t varies we observe that the system's representative point in the configuration manifold describes a curve and qi = qi(t)are the equations of this curve.

Ivancevic, V G, & Ivancevic, T T, 2006: 03 - Mechanical Basis of Human-Like Biomechanics. Human-Like Biomechanics, 191-288, Springer.

This chapter studies various aspects of modern mechanics as is currently used in biomechanics. It includes both Lagrangian and Hamiltonian variations on the central theme of our covariant force law, Fi = mgijaj. We start with the basics of Lagrangian and Hamiltonian formalisms. After that we move on to the general variational principles of holonomic mechanics. Next we depart to nonholonomic. At the end, we present the current research in biomechanics given in the framework of Lie-Lagrangian and Lie-Hamiltonian functors.

Ivancevic, V G, & Ivancevic, T T, 2006: 04 - Topology of Human-Like Biomechanics. Human-Like Biomechanics, 289-312, Springer.

In this chapter we develop the basics of algebraic topology as is used in modern biomechanics. It includes both tangent (Lagrangian) and cotangent (Hamiltonian) topological variations on the central theme of our covariant force law, Fi = mgijaj.

Ivancevic, V G, & Ivancevic, T T, 2006: 05 - Nonlinear Control in Human-Like Biomechanics. Human-Like Biomechanics, 313-369, Springer.

In this Chapter we develop the basics of nonlinear control theory as is used in modern human-like biomechanics. It includes control variations on the central theme of our covariant force law, Fi = mgijaj, and its associated covariant force functor F* : TT*M => TTM.

Ivancevic, V G, & Ivancevic, T T, 2006: 06 - Covariant Biophysics of Electro-Muscular Stimulation. Human-Like Biomechanics, 371-390, Springer.

In this chapter we develop covariant biophysics of electro-muscular stimulation, as an externally induced generator of our covariant muscular forces, Fi = mgijaj. The so-called functional electrical stimulation (FES) of human skeletal muscles is used in rehabilitation and in medical orthotics to externally stimulate the muscles with damaged neural control. However, the repetitive use of electro-muscular stimulation, besides functional, causes also structural changes in the stimulated muscles, giving the physiological effect of muscular training.

Ivancevic, V G, & Ivancevic, T T, 2006: 07 - Back Matter. Human-Like Biomechanics, 391-468, Springer.