Encoding and Updating of Depth

Sources and Representation of Depth Information

In the previous two lectures we considered the encoding and updating of visual direction for action, but this leave out a significant variable: depth. It is necessary to know the distance of the goal in order to program vergence movements (which we will leave beyond the scope of this review) and of course reach.

Most people associate depth perception with binocular disparity. If one can correctly match, point-by-point, the images of the two retinas, then geometry dictates that they will be slightly deviated based on the variable distance of the targets and fixed distance between the two eyes. This provides an egocentric depth cue. As in directional coding, in normal circumstances this is powerfully supplemented by allocentric cues, the so-called monocular depth cues such as relative size, occlusion, and convergence of lines. But if we strip these away, disparity still requires additional cues to be useful. One needs to know the binocular vergence angle, because disparity provides a measure of depth relative to the plane of fixation. Moreover, one needs to know the 3-D orientations of the eyes and head in order to solve how simple points correspond on the two retinas.

This implies a synthesis of information from the two eyes, which begs a question avoided in the previous lectures: what is the location of reference for an eye-fixed reference frame? For some manual behaviors, it is not enough to know direction relative to the eye in spherical coordinates, it is necessary to know where the centre of that sphere is. Are these coordinates both eye-fixed and eye-centered, and if so, which eye? In some cases, such as sighting a gun or pointing (where the fingertip tends to align with the line of gaze) it is expedient to treat the eyes according to their separate locations and one is required to choose a ‘dominant eye’ – although that dominance is not fixed, it can be reversed by simply changing horizontal target direction relative to the head (). In other cases it may be expedient to refer back to a fused cyclopean eye located roughly at the bridge of the nose. These options have been debated rancorously, but it seems likely the brain is capable of using both strategies, depending on the task.

How does this information enter the Action system?

Physiological studies of visual depth have mainly focused on depth perception, but those studies that have looked at action suggest that posterior parietal cortex plays a special role in depth coding (Ferraina). First, damage to PPC produces both deficits in perception of relative and absolute depth (), as well as deficits in the depth of reaching (). Single unit recordings in areas such as LIP and V6A show many neurons that are sensitive to visual disparity and/or vergence angle. This is information is relayed downstream, but shows up to a lesser extent in more anterior cortical structures and the superior colliculus ().

*Moreover, LIP also contains neurons that encode changes in vergence angle in their saccade-related activity (perhaps move to next lecture).

Updating Depth Information

Less is known about spatial updating for depth than spatial updating for direction. Gaze centered updating of target direction appears to hold for different target depths (), but this is not the same thing as updating depth.

However, a recent study used a variation of the paradigm described in the last lecture to show that similar principles hold for depth updating. If one considers the binocular fixation point as the 3-D depth equivalent to the gaze point, then one can measure errors for reaching to targets relative to that depth. As for directional updating, when the depth of the binocular fixation point changes between seeing and reaching toward the target, subjects show a pattern of errors consistent with the final fixation position, suggesting that reach plans are updated in both direction and depth relative to binocular fixation.

A related concept is the depth-dependent updating of target locations during translational motion of the head. Most readers will be familiar with motion parallax: the observation that the direction of near targets (such as people) changes much faster than that of far targets (such as the sun) as one moves through space. In order to update such objects as goals for action during head translations, the brain would have to internally simulate this complex non-linear geometry. Remarkably, the brain is able to this for programming both saccade direction (Medendorp, Angelaki), and reaches (), and even (although with more variable success) for eye translations due to head rotation (see lecture 2). To do this in gaze-centered coordinates, the brain must account use an updater circuit that receives information about self motion and then uses intrinsic information about depth information of each target to determine how much it would be remapped.

Conclusion

Synthesizing information from the past 3 lectures –which form a key block of this course— we can conclude that the brain actively forms internal representations of reach and saccade goals, and then updates these during self motion, using similar mechanisms.