Cerebral Cortex Advance Access originally published online on October 12, 2005
Cerebral Cortex 2006 16(8):1168-1180; doi:10.1093/cercor/bhj058
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Haptic Feature Extraction
Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
Address correspondence to John F. Soechting, Department of Neuroscience, 6-145 Jackson Hall, 321 Church street SE, Minneapolis, MN 55455, USA. Email: soech001{at}umn.edu.
 |
Abstract |
|---|
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
This study examined the process by which the shape of a haptically explored object is synthesized from the geometric characteristics of simpler constituent elements, such as arcs and ellipses. Subjects traced the outlines of virtual objects by means of whole arm movements. Each object consisted of the union of a large central ellipse and two smaller circles, extending upward and outward from the top left and right sides of the base. The sizes of the two circles and the eccentricity of the elliptical base were varied. After exploring the object's contour in the absence of vision, subjects reproduced the sensed shape by means of freehand drawing. Speed and force were modulated during the exploratory phase in a manner that suggested that subjects reacted to rather than predicted changes in curvature. Also, subjects typically devoted more time to exploring the part of the contour encompassing the two smaller circles. During drawing, individual features of the explored shape were reproduced with varying degrees of fidelity. Aspects related to the size and location of the smaller circles were reproduced better than was the eccentricity of the ellipse forming the base. Since subjects spent proportionally less time exploring the base, these results suggest that subjects selectively focused attention to regions of high spatial contrast and that the exploratory strategy introduced distortions in the haptically sensed shapes.
Key Words: arm movements • drawing • haptic exploration • human • movement sequences • somatosensory
|
Introduction |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Identifying the shape of unfamiliar objects by means of exploratory movements may involve a sequential process, the first step being the characterization of segments of the object's contour, such as arcs and straight edges. In such a scheme, the overall shape of an object could be synthesized by mentally conjoining individual segments. This process requires that each of the segments be characterized in terms of specific features, such as their lengths, orientations and curvatures. While this hypothesis has not been tested experimentally, it parallels proposed schemes for visual object recognition (cf. Marr and Nishihara, 1978
; Biederman, 1987; Riesenhuber and Poggio, 2000). It has also been proposed that complex movements such as handwriting or drawing are generated as a sequence of simpler movements. For example, Hollerbach (1981) showed that script could be synthesized by controlling the phase and amplitude of two-dimensional sinusoidal oscillations in a step-wise fashion. There is also evidence for a segmentation of drawing movements (Soechting and Terzuolo, 1987) and a virtual segmentation of fingerspelling movements in American Sign Language (Jerde et al., 2003). Modeling studies (cf. Doeringer and Hogan, 1998) and cortical activity during complex motor tasks such as drawing and copying (Averbeck et al., 2002, 2003) provide further evidence for a segmentation of limb movements in free space.
If this idea is to be a plausible scheme for haptic sensing, then the first requirement is that features of individual segments can be discriminated with good accuracy. This is indeed the case when whole-arm movements are used to make haptic discriminations of parameters such as the relative lengths of line segments or the angle between them (Hogan et al., 1990; Fasse et al., 2000; Voisin et al., 2002). This is also true when subjects are asked to make absolute judgments, concerning for example the convexity or concavity of a line segment or its orientation relative to cardinal axes (Henriques and Soechting, 2003) or the eccentricity of a circle (Henriques and Soechting, 2003; Soechting and Poizner, 2005). In most of these tasks, subjects did exhibit consistent biases, implying that their haptic sense of space was distorted. For example, subjects reported lines with a radius of curvature of 
2 m as being straight and ellipses with an aspect ratio of 1.07 as being circular (Henriques and Soechting 2003).
As a first step in exploring the idea that the haptic sensing of complex shapes is constructed from such primitives, we examined the extent to which subjects could reproduce the shape of quadrilaterals (Henriques et al., 2004). In that study, subjects explored the contours of a quadrilateral with their eyes closed and then reproduced its shape by means of a freehand drawing movement. The subjects' errors were consistent with errors in judging the length or orientation of individual line segments. However, not all of the features of a quadrilateral can be specified independently of each other. For example, specifying three internal angles serves to determine the fourth one. Similarly, specifying the lengths of three segments and two internal angles determines the length of the fourth segment.
To eliminate this constraint, in the present work we examined the extent to which subjects could characterize the shape of convex objects formed by the intersection of circles and ellipses. Our results were consistent with the premise outlined above in that subjects were successful in reproducing the features of the constituent elements, such as the size and shape of individual components. However, not all features were reproduced with equal fidelity. We suggest that the distortions in haptic perception resulted from a selective focusing of attention during this task.
|
Materials and Methods |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Overview
Subjects explored the contours of various shapes by grasping the handle of a manipulandum (Interactive Motion Technologies) that was constrained to move in the horizontal plane. They performed these exploratory movements with their eyes closed; otherwise they were free to use any strategy they desired. The contours of the objects were simulated by programming torque motors of the manipulandum to generate an elastic force field (cf. Fasse et al., 2000; Henriques et al., 2004). If the handle was inside the contour, no force was generated, but if the handle penetrated the contour, an elastic restoring force in the direction perpendicular to the contour was generated. [We asked subjects to explore the inside of a convex contour because it is easier to maintain contact with the boundary from this direction (Fasse et al., 2000)]. Once the subjects were satisfied that they had fully explored the shape, the force field was turned off. They were then asked to reproduce the shape by means of a freehand drawing, using either the manipulandum in the horizontal plane or a touchsensitive computer monitor (Elo Entuitive 2125C Touchmonitor) oriented vertically.
Eight subjects participated in these experiments. All gave their informed consent to procedures that were approved by the Institutional Review Board of the University of Minnesota. Two of the eight subjects were left-handed; all subjects used their dominant arm in this task. Four subjects used the manipulandum during the reproduction phase of the task, whereas the other four used the touchscreen. None of the subjects had prior knowledge of the shapes that they explored or of the experimental design. They were not afforded any practice trials, but they had the option to reject a trial if they were not satisfied with the reproduction.
Object Shapes
The virtual objects were generated from the union of a large, central circle or ellipse that formed a base, and two smaller circles (Fig. 1). The base was either a circle of radius 10 cm (top row, Fig. 1) or an ellipse whose aspect ratio (the ratio of the lengths of the major to minor axes) was 1.5. The major axis was either oriented medio-laterally (middle row, Fig. 1), forming a wide ellipse, or antero-posteriorly (bottom row, Fig. 1), forming a tall ellipse. The size of each elliptical base was chosen so that it encompassed the same area as the circle.
|
The two smaller circles (lobes) were centered on the perimeter of the base, in the upper quadrants, at ±45° from the antero-posterior axis (see Fig. 1, top left). They could vary in size, with a radius of either 5, 3.5 or 2 cm. The size of the left and right lobes could vary independently of each other. Accordingly, there were 27 different possible shapes (3 bases x 3 left lobes x 3 right lobes). For convenience they were numbered in a three-digit code, the first number corresponding to the base, and the next two denoting the sizes of the left and right lobes. Figure 1 presents nine examples, with one base per row, and one combination of lobe sizes per column. Each shape was presented to the subject three times, for a total of 81 trials per session. The order of presentation was randomized.
To control the robot manipulandum, we defined the contour by its radial distance from the center (filled circles in Fig. 1) as a function of the polar angle in increments of 0.01 radians (i.e. at 628 discrete locations). The sharp edges at the intersection of the lobes with the base were smoothed by digital filtering and the contour was simulated by an elastic force field (stiffness 2 N/mm).
Experimental Procedures
The subjects were seated, and at the start of each trial they grasped the manipulandum's handle (10 cm x 3 cm diameter, oriented vertically) with their dominant hand and with their eyes closed. The robot first slowly guided the handle to the origin (the base's center) which was located in the midsagittal plane, 30 cm from the subject's trunk. When the handle was at the origin, this force was turned off and a tone provided a signal for the subject to begin exploring the contour. Whenever the handle penetrated the contour, the robot produced a force proportional to the depth of penetration in the direction perpendicular to the contour. Damping (5 N/mm/s) was also added to increase stability.
The subjects were free to explore the contour until they were confident of its shape, signaling the end of the exploratory phase verbally. They next reproduced the shape by means of a freehand drawing. For the first four subjects, this phase of the trial also involved the use of the manipulandum. The handle was again guided to the center, all forces were turned off and a tone provided the signal for the reproduction phase. The subjects were instructed to draw the entire shape once with their eyes closed. For the other four subjects, the reproduction phase involved a touch-sensitive computer monitor (21'', 0.3 mm spatial resolution), oriented vertically and laterally to the right of the subject. After rotating their trunk through 90°, with eyes open they used their fingertip to draw the shape's outline on the screen, leaving a visible trace. They had the option to reject the drawing (by touching a REJECT button) and to repeat it until they were satisfied. (Most subjects were generally satisfied on the first try.) Each session lasted 3 h and subjects typically took several breaks.
Data Acquisition and Analysis
During the exploration phase of each trial, the x- and y- components of the handle's position and velocity (vx, vy) and the robot's force (fx, fy) were recorded at 200 Hz. When the reproduction was performed using the manipulandum, position and velocity were recorded at the same sampling frequency and stored in a separate file. The x, y positions of the finger were recorded at 50 Hz when the touchscreen was used for reproduction.
Subsequent data analysis dealt primarily with characterizing the reproduced shape drawn by the subject. This was done in two ways: fitting ellipses to various segments of the drawing and measuring the relative location and/or size of various landmarks in the drawing (Fig. 5). These measures were subsequently correlated with the corresponding parameters obtained from the subject's actual trajectory during the exploratory phase to delineate the accuracy with which various features of the explored shape were reproduced.
|
Since the simulated border was elastic, the actual shape traced by the subject could exceed the nominal shape (i.e. the border at which the elastic force was zero). Furthermore, as described in Results, subjects typically made several passes around the contour. Therefore, as a first step in the analysis, for each subject we computed the average traced contour, and the speed and force along the contour for each of the 81 trials. For this purpose, the data were binned and interpolated in polar coordinates in 1° increments, and the radius from the center to the handle position, and speed and force magnitude were computed for each pass and averaged. The number of passes, defined as every instance in which the handle was moved at least ±1° from a given bin, was also computed. Landmarks, such as the intersection of the two lobes with the base were identified interactively and ellipses were fitted to the segment corresponding to each of the two lobes and to the two segments defining the base using the algorithm described by Fitzgibbon et al. (1999)
. For each ellipse, we computed the length of the major and minor axes (ef1 and ef2, ele1 and ele2, see Fig. 5) and their orientation with respect to the x-axis.
The size of each of the two lobes was defined to be half the area of the ellipse for that lobe:
 | (1) |
 | (2) |
Similar measures were computed for the reproduced shapes. Seven of the eight subjects began the reproduction at the intersection of one of the lobes with the base. In those instances, we used the continuous segment from the start to the other lobe to define the shape of the base. One of the eight subjects began the reproduction at the bottom of the base. For that subject, we used two segments (the left and right halves of the base) in the fitting procedure. One subject (whose data are shown in Fig. 6) in the first few trials began the reproduction at the 3 o'clock position, but then switched her strategy to begin at the end of the right lobe. In those few instances, we neglected the last portion of the reproduced base in the fitting procedure. Since the estimate of the size of the smallest lobe (2 cm radius) obtained by fitting an ellipse to the explored traces was prone to error, in subsequent analysis we used equation (2) as an estimate of the size of the lobes. (The two measures gave equivalent results for the reproduced drawings.) We also fitted a second ellipse to the base neglecting the short segment connecting the two lobes at the top.
|
We were more interested in distortions of the shape than in the overall size of the reproduction, and there were no explicit instructions to the subject concerning size. Therefore, for each trial, we scaled the reproduction so that it encompassed the same area as the explored shape. Specifically, we normalized the values of each of the parameters by the ratio of the sizes of the reproduced and of the explored shapes. For this purpose, the size was computed as the sum of the size of the base (
ef1ef2) and the size of each of the lobes (equation 2). Standard methods of statistical analysis [regression, between-subjects analysis of variance (ANOVA)] were used to compare the extent to which various features in the reproduced shapes were related to features of the explored shape. Specifically we used a four-way ANOVA (including subjects as one of the independent factors) to assess whether the various measures describing the reproduced shape depended statistically on the size of the two lobes and the aspect ratio of the base. |
Results |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Shape Exploration
Subjects typically explored the reference shape by making several sweeps along the perimeter of the contour. Generally, these exploratory movements were not strictly unidirectional (i.e. clockwise or counterclockwise), but included reversals in direction. Furthermore, the number of sweeps along particular portions of the contour was not uniform. Specifically, the upper half of the contour, including the two lobes, was explored more extensively than was the lower half.
This is illustrated by a typical example in the left column in Figure 2. The outline of the average shape that was actually traced (a wide ellipse with two medium sized lobes) is shown by the dark heavy lines. Each of the lighter lines denotes one sweep at that particular location in the form of a polar histogram. Thus, on average, this subject passed through the 0° location (the point midway between the two lobes) six times, whereas he passed through the 180° location only twice. The behavior illustrated in this panel of Figure 2 was typical of the behavior of all subjects. Two subjects explored the shapes more extensively than did the other six, sweeping by each point on the contour at least 6–8 times. The corresponding value for the other subjects was 2–3 times. Nevertheless, the asymmetry depicted in the amount of time spent at each location was consistent among all subjects. This can be seen in the right panel of Figure 2, which depicts the normalized number of sweeps at different polar directions along the contour. The data points represent the grand mean for all trials for all subjects, with the error bars indicating the SEM.
|
Statistical analysis showed that the normalized number of sweeps at points at different directions from the center depended only on the directional location [ANOVA, F(7,1512) = 51.4, P < 0.001), but not on the shape of the base or the size of the two lobes (P > 0.05). The proportional number of sweeps at 0, 45 and 315° was greater than was the corresponding number at all other directions (post-hoc Bonferroni comparisons).
Given the finite stiffness of the boundary, the actual contours traced by the subjects slightly exceeded the nominal contours, i.e. the ones at which force was zero. For example, the average diameter of the circular base was 10.2 ± 0.1 cm (mean ± SD), and the widths of the lobes averaged 10.7, 7.8 and 4.5 cm (compared to nominal values of 10, 7 and 4 cm). The aspect ratios of the bases as traced did not differ from their nominal values (1.0, 1.5 and 0.67).
Hand speed and the force exerted during the exploration were also nonuniform along the contour. Variations in speed (Fig. 3) and in force (Fig. 4) are depicted in polar plots for several exemplary shapes. Speed invariably reached a maximum at 180° and a minimum at 0°. For all subjects this maximum speed averaged 38 cm/s, with maxima for individual subjects ranging from 60.9 ± 7.2 to 26.9 ± 2.9 cm/s. The three plots in Figure 3 depict the average traced contour and the average speed along the contour from one subject for the three shapes illustrated in the rightmost column in Figure 1. From these examples, it is clear that variations in speed along the base (i.e. from 90 to 270°) depended little on the shape of the base. In contrast, in the upper half, the speed in the middle of the larger lobes was larger than was the speed in the middle of the smaller lobes (compare the results for the left and right lobes for each of the shapes in Fig. 3). In the intervals from 270 to 90° (in the clockwise direction, through 0°), for all subjects, normalized speed did depend on the shape of the base, being largest for tall ellipses (ANOVA, P < 0.02). Statistically, it also increased with the size of the lobe at 45 and 315° (P < 0.001), as indicated schematically by the upward arrows in the plot of averaged normalized speed in the upper right panel of Figure 3. The size of the left lobe also affected speed in adjacent portions of the contour; speed was slowest for the largest left lobe at 270 and 360° (downward arrows).
|
|
During unconstrained arm movements, it is well known that speed is related to radius of curvature through the ‘two-thirds power law’ (Lacquaniti et al. 1983
; Viviani and Cenzato 1985). Thus hand speed should be fastest along the straightest portions of a trajectory and slowest along the portions that are most curved. The constrained exploratory movements in our experimental conditions did not conform to this relation. For the wide ellipse, where the radius of curvature is largest at 180°, speed and radius of curvature did tend to be positively correlated as subjects traced the base's contour. However, when they explored the tall ellipse, speed and radius of curvature were negatively correlated, speed being maximal and the radius of curvature reaching a minimum at 180°. The force exerted by the subjects against the contour was also nonuniform and reflected the particular shape (Fig. 4). As was the case for speed, force was generally largest at the bottom of the base (180°), with other peaks at the top of the base (0°) and at the centers of the two lobes. Compared to speed, the modulation in force was more congruent with the shape of the base, force at 90, 135 and 225° being larger for the wide elliptical base than it was for the narrow one (P < 0.001). Consequently, the force profile in the lower half (90–270°) resembles the shape of the base (compare the lower two plots in Fig. 4). Thus, in general, force was largest when the tangent to the contour changed direction faster (e.g. at the bottom of the tall ellipse) and when the subject moved faster. Peak force averaged 14.2 N for all subjects, with a range from 6.2 ± 0.7 N for the subject who moved the slowest to 23.9 ± 4.5 N for the subject whose peak speed was the largest.
At 0 (360), 90 and 270° the magnitude of the force depended significantly on the size of one or both lobes (as indicated by the arrows in the plot in Fig. 4) as well as on the shape of the base. To the contrary, the size of either lobe did not affect the force magnitude at 45 or 315°, i.e. at the midpoints of the two lobes. Thus, as the subjects moved back and forth along the upper half of the perimeter, the force amplitude along this part of the shape depended on adjacent portions of the contour. This suggests that subjects were reacting to changes in shape rather than predicting them. A similar explanation also holds for the force profile when the base was explored; force was larger when the curvature was larger. Taken together, the speed and force profiles suggest a strategy of planned straight-line or circular movements encountering a resistance in areas of high curvature.
Shape Reproduction
Based on interviews at the conclusion of the session, the subjects recognized that the shapes consisted of a main oval with two lobes, whose size could vary. They also reported that the shape of the oval varied from wide to tall. None guessed that there were only three discrete sizes or eccentricities; each tended to overestimate this number. Thus they were successful in capturing the overall gestalt of the haptically explored shapes, also evidenced by the fact that their freehand reproductions could be well fitted with an ellipse representing the base and two other ellipses, located laterally in the upper two quadrants, representing the two lobes (Figs 5 and 6). This was true when they drew the shape on a vertically oriented touchscreen with eyes open (Fig. 5 and Fig. 6, left column) or when they reproduced it with eyes closed using the manipulandum (Fig. 6, right column). In the latter case, however, they were not always successful in generating a closed figure by returning to the starting point of the drawing (denoted by the arrows in Fig. 6). For example, in the lower two cases illustrated in Figure 6 (right column), the left lobe began inside the ellipse forming the base. In many instances, the segment connecting the two lobes appeared to be a continuation of the ellipse forming the base (Fig. 5). However in some cases, the lower segment curved inward at the points where it met the two lobes; in others, the curvature of the segment connecting the two lobes did not match the curvature of the elliptical fit (Fig. 6, top right panel).
Except for one right-handed subject, the reproductions were always performed with a clockwise motion, although the starting point varied among subjects. For example, in the left panels in Figure 6, motion began in the middle on the right (on the x-axis), whereas in the examples in the right panels, the left lobe was drawn first.
However, irrespective of the mode of reproduction, some of the specific features of the shape were reproduced with more fidelity than were others. In the following we will focus on this aspect of the reproductions. As described in Materials and Methods and illustrated in Figures 5 and 6, we fitted ellipses to segments of the reproduced shape, demarcating each segment at points where the direction of motion changed abruptly. In Figures 5 and 6, adjacent segments are denoted by heavy solid lines and heavy dashed lines. The two segments indicated by the solid line were taken to represent the contour of the base, and were fitted with one ellipse (see light, dashed lines). We also fitted an ellipse using data only from the lower portion of the base. In most instances (63% touchscreen, 52% manipulandum), this ellipse was identical (by visual inspection) to the one obtained by including the segment connecting the two lobes at the top. Each of the segments indicated by the heavy dashed lines (representing the two lobes) was also fitted with an ellipse. For each ellipse, we computed the length of the major and minor axes (e.g. ef1 and ef2) and their orientation relative to the x- and y-axes. We also computed the size of each lobe from its width (e.g. wle) and its height (hle, the maximum perpendicular distance from the line defining the width to the perimeter), the height of each lobe above the crown (difference in ymax for the lobe and ymax of the segment connecting the two lobes), and the distance between the two lobes.
The reproduction illustrated in Figure 5 corresponds to a circular base, with a large left lobe and a small right lobe (shape 113 in Fig. 1). Similarly, the reproductions in Figure 6 (which are from two other subjects) correspond to the shapes plotted in the far right column of Figure 1. It is clear that all three subjects captured some features quite well. Specifically, the relative sizes of the two lobes correspond quite well with their relative sizes in the reference shape, as does their location with respect to the base. The relative distance between the two lobes was also captured quite well by two of the three subjects (Fig. 5 and Fig. 6, left column), but not as well by the third (Fig. 6, right column). The reproduction in Figure 5 is slanted to the left. However, this was not a common finding, even for this subject. The reproductions in Figure 6 are more typical in this regard.
The illustrations in Figure 6 are also representative in another respect. Whereas subjects were successful in characterizing the size and location of the two lobes quite well, they were much less accurate in reproducing the shape of the base. Note that the reproductions in the middle row of Figure 6 correspond to wide reference bases, while the ones in the lower row correspond to narrow, tall reference bases. We characterized the shape of the base by computing the aspect ratio, which we defined as the ratio of the major and minor axes of the ellipse for the base. (The numerator was the axis that most closely paralleled the x-axis and the denominator the one closest to the y-axis.) Following this convention, the aspect ratios for the explored shapes were 1 for the circle, 1.5 for the wide ellipse and 0.67 for the tall ellipse. The range of aspect ratios for the reproductions shown in Figure 6 is greatly reduced. For the touchscreen reproductions, the values were 0.86 (shape 113), 0.89 (213) and 0.76 (313), and they were 1.14 (113), 1.03 (213) 1.03 (313) when the shapes were reproduced using the manipulandum.
Figure 7 illustrates the overall trends for all subjects. On average, the slope of the regression between the aspect ratio of the explored and reproduced bases was 0.434 (r2 = 0.271) for the manipulandum and 0.222 (r2 = 0.143) for the touchscreen. The plots in Figure 7 also show the overall bias that was apparent in Figure 6: the aspect ratios tended to be less than unity when the shapes were drawn on the touchscreen and greater than one when they were reproduced using the manipulandum. An ANOVA found that the aspect ratio for the touchscreen reproductions depended on the shape of the reference base [F(2,216) = 44.1, P < 0.001], but not on the size of either lobe. When the manipulandum was used in the reproduction, the aspect ratio depended on the size of the left lobe [F(2,216) = 3.49, P < 0.03] as well as on the shape of the base [F(2,216) = 74.6, P < 0.001]. Accordingly, in the left panel of Figure 7, we show separate regressions for the three sizes of the left lobe. We also characterized the shape of the base by the ratio of its width to its height, computed from the extrema in x (height) and in y (width) of the segments constituting the base (solid lines in Fig. 6). This analysis gave essentially the same results, with an even weaker slope in the regression of this measure with the aspect ratio of the explored shape (slopes = 0.24 and 0.05).
|
There was some variability in the extent to which individual subjects captured the eccentricity of the base. For the four subjects who used the manipulandum to reproduce the shape, the slopes of the regression ranged from 0.24 to 0.79, the latter value being for the subject whose data are illustrated in Figure 6. For two of the subjects who used the touchscreen, the slope did not differ significantly from zero, whereas it was 0.42 (shown in Fig. 6) and 0.45 for the two others. The first two subjects spent proportionally the least amount of time exploring the base (normalized number of sweeps at 180° = 0.29 for both). For all eight subjects, the slope of the regression was positively and significantly (P < 0.01) correlated with the normalized number of sweeps at the bottom of the base (r2 = 0.758), but not (P = 0.2) with the actual number of sweeps at that location. In summary, consistent with their verbal reports, most (6 of 8) subjects did incorporate variations in the aspect ratio of the base into their reproductions, but they vastly underestimated the range of variation in this parameter.
By contrast, as shown in Figures 8 and 9, features that were located in the upper two quadrants of the explored shape were reproduced with much greater fidelity. Among the parameters we quantified were the relative size of each of the lobes (Fig. 8), their height above the crown of the base (Fig. 9, top panels) and the distance between the two lobes (Fig. 9, bottom panels). (For this analysis, we first scaled the size of the reproduced shape so that it equaled the size of the explored shape.) For each of these parameters, the slopes of the regressions between the parameters characterizing the reproduced and the explored shapes were substantially greater than the values for the aspect ratio of the base. Pooling the data from all subjects, the slopes ranged from 0.54 to 0.87 for this set of parameters.
|
|
Results for the sizes of the left and right lobes are shown in Figure 8. The data points show average values for the respective sizes, after the overall size of the reproduced shape was normalized to the size of the explored shape. In two instances (left lobe, touchscreen; right lobe, manipulandum) the size of the reproduced lobe also depended on the shape of the base and in these the values have been segregated according to this parameter. The slopes of the regression ranged from 0.611 to 0.870. For individual subjects, these values ranged from 0.451 to 1.108, with coefficients of determination that ranged from 0.31 to 0.78 (average r2 = 0.61).
The height of each lobe above the crown of the base and the distance between the two lobes were also reproduced with similar fidelity (Fig. 9). The height of each lobe above the crown in the reference shapes depended on the size of the lobe as well as on the shape of the base (see Fig. 1). An ANOVA showed that when the shape was reproduced with the manipulandum, the reproduced height depended on the size of that lobe and on the shape of the base, but not on the size of the other lobe [e.g. right lobe: F(2,297) = 69.9 right lobe; F(2,297) = 39.3 base, both P < 0.001; F(2,297) = 1.59 left lobe, P = 0.21]. When the reproductions were drawn using the touchscreen, there was also a weak dependence on the size of the other lobe [e.g. right lobe: F(2,297) = 60.8 right lobe; F(2,297) = 49.4 base, both P < 0.001; F(2,297) = 4.91 left lobe, P < 0.01). (The effect of the contralateral lobe on the height of the left lobe was even weaker.)
The slopes of the regressions between the explored and reproduced heights were comparable for the two lobes, with an average slope of 0.78 for the manipulandum and somewhat less (0.55) for the touchscreen. The distance between the two lobes depended on all three of the parameters that characterized the explored object: the eccentricity of the base and the size of the two lobes. The slopes of the regressions in this instance were 0.59 for the reproductions using the manipulandum and 0.72 when the touchscreen was used (Fig. 9, lower panels).
The extent to which the various features of the explored shapes were captured can be visualized in Figures 10 and 11, which depict an overlay of the nine reference shapes illustrated in Figure 1 (dashed lines) on the average shape drawn by the subjects. The plots in Figure 10 show results obtained when the manipulandum was used whereas those in Figure 11 were obtained when the touchscreen was used. To obtain these traces, we first computed average values for the parameters characterizing each ellipse (origin, major and minor axis and inclination), using all of the trials from the four subjects for that particular shape. We then computed the intersection of the three averaged ellipses. Before averaging, the values were scaled so that the area encompassed by the reproduction was the same as the area that was explored. The shaded area encompasses ±1 SEM.
|
|
The illustrations in Figures 10 and 11 reinforce the conclusions of the statistical analysis. The relative sizes of the two lobes were reproduced quite well. For example, in the left column of Figures 10 and 11, they are of about equal size, whereas the right lobe is clearly much smaller in the traces in the right column. Furthermore, the location of the two lobes relative to the base was reproduced quite well. This was true also for the distance between the lobes (the length of the arc of the base connecting them). This distance is smallest for reference shape 211 (Fig. 1) — a detail that was reproduced quite well. However, the eccentricity of the base was not captured nearly as well. In Figure 10, the bases in the middle row are clearly wider than the ones in the other rows, but not nearly to the extent as in the reference shapes. Finally, the illustrations demonstrate one other phenomenon. When the drawings were generated using the manipulandum, they were distorted (stretched) along the x-axis, while those generated using the vertically oriented touchscreen were elongated along the y-axis.
There were statistical differences in the slopes of the regressions between features of the explored shape and its reproduction and in the coefficients of determination for the various features [ANOVA, slope: F(5,42) = 3.812, P < 0.01, r2: F(5,42) = 5.149, P = 0.001]. For this analysis we computed the slopes of the regressions and the coefficients of determination separately for each subject (Fig. 12). As shown in this figure, the slope of the regression for the aspect ratio of the base is substantially smaller than is the slope of the regression for all of the other features. In a post-hoc analysis, we used a t-test with pairwise comparisons (adjusted for multiple comparisons), to compare the value for the aspect ratio with corresponding values for all other features (taking into account that the overall performance of some subjects was better than of others). According to this test, the slope of the regression for the aspect ratio differed from all of the other slopes (P-values ranging from 0.03 to <0.001). The coefficient of determination for the aspect ratio was also lower than it was for all other features (P < 0.005), with the exception of the distance between the two lobes (P = 0.12).
|
|
Discussion |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
The experiments described here were based on the idea that haptically explored complex shapes are synthesized from simpler geometric elements, such as straight lines, arcs or even ellipses. Such a process would entail first determining the length, curvature, orientation and location of each of these ‘primitives’ and then reconstituting the overall shape. The shapes that we presented to the subjects could be reconstituted in this manner. They consisted of circles of several sizes and an ellipse of several different eccentricities. The overall convex shape was provided by the union of two circles centered on the perimeter of the ellipse.
Although we did not test this idea directly, the shapes of the reproduced drawings were consistent with this point of view. We segmented the drawings at points where the contour changed substantially, i.e. at points where the curvature became large (Fig. 5). These segments could be well fitted with elliptical arcs (see Figs 5 and 6). In more than half the cases (e.g. Fig. 5), the segment connecting the two lobes appeared to be a continuation of the ellipse forming the base, suggesting that the subjects had internalized the shape according to rules similar to those used in constructing it. However, in other instances (Fig. 6, right column), the arc connecting the two lobes had a curvature that differed markedly from the curvature of the base. The present experiment was not designed to test whether or not subjects perceived this arc to be a continuation of the base, and this question remains unresolved.
When subjects explored the contour, it appeared that they were primarily reacting to changes in its curvature rather than predicting them, force being largest when the tangent to the contour changed direction faster. There was a lag in the modulation of the forces exerted against the boundary with respect to its curvature. For example, force depended on the radii of the two lobes at points to either side (0 and ±90°) but not at the apices of the two lobes (±45°, Fig. 4). (The upper half of the contour was typically explored bi-directionally in the counter-clockwise as well as in the clockwise direction.) The modulation in speed did not consistently mirror changes in the contour's curvature (Fig. 3), making it unlikely that this parameter would be useful in extracting information about the contour's properties (see also Soechting and Poizner, 2005). Modulations in the force profile more closely mirrored the outline (Fig. 4) and conceivably force information could have been utilized to characterize the contour (Robles-De-La-Torre and Hayward, 2001; Song et al., 2004).
We used two different means to characterize the haptically sensed shapes. When the manipulandum was used during reproduction, subjects conceivably could have merely reproduced the exploratory movement without actually synthesizing the shape, but they did not have visual feedback of the movement. Conversely, when they used the touchscreen, visual feedback was available, but the arm movement required for accurate reproduction was different (vertical rather than horizontal plane). The two modes of reproduction led to some differences and some similarities in the distortions of the reproduced shapes, in agreement with results we have reported previously (Henriques et al., 2004).
In the previous experiments, subjects explored the contours of various quadrilaterals and also reproduced them using either the manipulandum or the touchscreen. One distortion common to all quadrilaterals concerned their aspect ratio. Quadrilaterals reproduced with the manipulandum tended to be wider than tall, whereas the opposite tendency was found when they were drawn on the vertically oriented touchscreen. (This distortion may be related to the ‘tangential-radial effect’, as discussed in Henriques et al., 2004.) A similar trend was found in the present experiments (compare Figs 10 and 11, and see Fig. 7). Furthermore, the drawings of quadrilaterals reflected a tendency to the mean: long sides were drawn as being shorter and short sides were drawn as being longer. On average, the slope of the regression between normalized line length of the explored and reproduced quadrilaterals was 0.83 (manipulandum) and 0.90 (touchscreen). The same tendency also held for the internal angles between adjacent sides. In the present experiments, we computed the slopes of the regressions between the reproduced and the explored shape as characterized by several different parameters (see Fig. 12). Invariably, these slopes were less than unity; variations in the sizes of the two lobes and the eccentricity of the base in the reproduction were smaller than the corresponding variations in the explored shape. With the exception of the aspect ratio of the base, the slopes reported here were similar to those reported in Henriques et al. (2004).
In the absence of visual feedback, subjects typically were not successful in returning to the starting point of the reproduction (see Fig. 6, right column). With visual feedback, they generated a closed shape. When they used the touchscreen, the reproductions were smaller than the reference shape. (The workspace of the monitor was smaller than the workspace of the manipulandum.) Both of these differences were also found previously (Henriques et al., 2004).
The previous experiments (Henriques et al., 2004) also uncovered a serial effect in the reproduction of haptically explored shapes. When the order of exploration of an open shape consisting of three straight, non-colinear lines was strictly controlled, the lengths and orientations of the lines subsequently drawn freehand from memory depended on the characteristics of the preceding segments but not on those that followed. In the present experiments, we did not control the serial order of exploration. In seven of the eight subjects, the reproductions were consistently performed by clockwise motion and the exploration of the base was also primarily in the clockwise direction. However, the upper half of the shape was typically explored bidirectionally and the starting location of the reproduction varied from subject to subject.
Surprisingly, in the present experiments not all features of the reproduced shape were scaled equally. Variations in the eccentricity of the ellipse were underestimated to a much greater extent than were variations in the sizes of the two lobes (Figs 10 and 11). They were also underestimated when compared to the variations in the height of the two lobes above the crown and the distance between the two lobes (Fig. 12), parameters that depend on the eccentricity of the base as well as on the size of the two lobes. Visually, variations in the eccentricity of the base are as salient as variations in the sizes of the lobes (Fig. 1). We suspect that had we required subjects to copy shapes presented visually (cf. Averbeck et al., 2003; Chafee et al., 2005), they would have captured this feature quite well. A possible explanation lies in one difference that is inherent between visual and haptic shape exploration; the latter is obligatorily serial in nature (Henriques and Soechting, 2005) and it is likely that subjects focussed their attention on the upper half of the shape, containing the two lobes. This supposition is supported by the fact that this portion was explored to a considerably greater extent than was the lower half (Fig. 2). The curvature of the contour changed more rapidly and more often there than it did in the lower half. We suggest that regions of high curvature during haptic exploration are analogous to regions of high spatial contrast or of greater structure during visual exploration. Since Yarbus (1961), it is well known that such areas are foveated preferentially during visual exploration. Yet, visual attention directed to these areas does not necessarily lead to the distortions in the representation of the overall shape.
Conceivably, subjects devoted insufficient time to exploring the contours of the base to discriminate its eccentricity. In previous experiments, in which we used a two-alternative forced choice procedure (Henriques and Soechting, 2003; Soechting and Poizner, 2005), to test subjects' ability to discriminate the eccentricity of ellipses, subjects reliably identified ellipses with aspect ratios of 1.5 (the value used in the present experiments) as being wide or narrow. Changes in the speed profiles during the exploration did not affect the threshold of discrimination appreciably (Soechting and Poizner, 2005). Reliability of identification of a shape may improve as the number of sweeps increases (Sinclair and Burton, 1991). However, we do not believe that this factor alone accounts for the present results. First, in our previous experiment where the motion of the arm was controlled by the robot (Soechting and Poizner, 2005), subjects often correctly discriminated the eccentricity of ellipses with aspect ratios of 1.5 on the first sweep. Furthermore, in the present experiments, the fidelity of the reproduction (slope of the relation between the aspect ratios of the reproduced and explored bases) depended on the relative amount of time devoted to exploring the base, but not on the absolute number of sweeps.
In the visual system, neural responsiveness is typically enhanced at spatial locations that are attended to (cf. Seidemann and Newsome, 1999; Cook and Maunsell, 2002). If similar processes apply to haptic sensing, the sensitivity to various shape parameters could be modulated by the spatial locus to which attention is directed. However, it has also been shown that when one attends to a particular feature at a particular spatial location, the sensitivity of neurons tuned to that feature at other spatial locations is enhanced as well (Martinez-Trujillo and Treue, 2004). Our results are also consistent with such a feature-dependent focusing of attention. Assume that the upper half of the shape was attended to preferentially. The features that varied in the upper half of the shape were related to the size, but not the eccentricity of the two lobes. Conversely, the size of the base did not vary, but its eccentricity did. Thus it is possible that subjects exhibited a selective tuning to variations in size, rather than shape. Such an interpretation leads to the prediction that had we varied the eccentricity, but not the size of the two lobes, subjects would have reproduced the eccentricity of the base to a greater extent.
|
Acknowledgments |
|---|
This work was supported by NIH Grant NS-15018. We thank Ajay Shrestha for his assistance with some of the experiments.
|
References |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Averbeck BB, Chafee MV, Crowe DA, Georgopoulos AP (2002) Parallel processing of serial movements in prefontal cortex. Proc Nat Acad Sci USA 99:13172–13177.
Averbeck BB, Chafee MV, Crowe DA, Georgopoulos AP (2003) Neural activity in prefrontal cortex during copying gemetrical shapes. II. Decoding shape segments from neural ensembles. Exp Brain Res 150:142–153.[ISI][Medline]
Biederman I. (1987) Recognition-by-components: a theory of human image understanding. Psychol Rev 94:115–147.[CrossRef][ISI][Medline]
Chafee MV, Crowe DA, Averbeck BB, Georgopoulos AP (2005) Neural correlates of spatial judgement during object construction in parietal cortex. Cereb Cortex 15:1393–1413.
Cook EP, Maunsell JH (2002) Attentional modulation of behavioral performance and neuronal responses in middle temporal and ventral intraparietal areas of macaque monkey. J Neurosci 22:1994–2004.
Doeringer JA, Hogan N (1998) Serial processing in human movement production. Neural Netw 11:1345–1356.[Medline]
Fasse ED, Hogan N, Kay BA, Mussa-Ivaldi FA (2000) Haptic interaction with virtual objects. Biol Cybern 82:69–83.[CrossRef][ISI][Medline]
Fitzgibbon A, Pilu M, Fisher RB (1999) Direct least squares fitting of ellipses. IEEE Trans Pattern Anal Mach Intell 21:476–480.[CrossRef]
Henriques DY, Soechting JF (2003) Bias and sensitivity in the haptic perception of geometry. Exp Brain Res 150:95–108.[ISI][Medline]
Henriques DY, Soechting JF (2005) Approaches to the study of haptic sensing. J Neurophysiol 93:3036–3043.
Henriques DYP, Flanders M, Soechting JF (2004) Haptic synthesis of shapes and sequences. J Neurophysiol 91:1808–1821.
Hogan N, Kay BA, Fasse ED, Mussa-Ivaldi FA (1990) Haptic illusions: experiments on human manipulation and perception of ‘virtual objects’. Cold Spring Harb Symp Quant Biol 55:925–931.[Medline]
Hollerbach JM (1981) An oscillation theory of handwriting. Biol Cybern 39:139–156.[CrossRef]
Jerde TE, Soechting JF, Flanders M (2003) Coarticulation in fluent fingerspelling. J Neurosci 23:2383–2393.
Lacquaniti F, Terzuolo C, Viviani P (1983) The law relating the kinematic and figural aspects of drawing movements. Acta Psychol 54:115–130.[CrossRef][Medline]
Marr D, Nishihara HK (1978) Representation and recognition of the spatial organization of three-dimensional shapes. Proc R Soc Lond B BiolSci 200:269–294.
Martinez-Trujillo JC, Treue S (2004) Feature-based attention increases the selectivity of population responses in primate visual cortex. Curr Biol 14:744–751.[CrossRef][ISI][Medline]
Riesenhuber M, Poggio T (2000) Models of object recognition. Nat Neurosci 3:1199–1204.
Robles-De-La-Torre G, Hayward V (2001) Force can overcome object geometry in the perception of shape through active touch. Nature 412:445–448.[CrossRef][Medline]
Seidemann E, Newsome WT (1999) Effect of spatial attention on the responses of area MT neurons. J Neurophysiol 81:1783–1794.
Sinclair RJ, Burton H (1991) Tactile discrimination of gratings: psychophysical and neural correlates in human and monkey. Somatosens Mot Res 8:241–248.[ISI][Medline]
Soechting JF, Poizner H (2005) The use of motion cues in the haptic sense of circularity. Exp Brain Res 165:413–421.[Medline]
Soechting JF, Terzuolo CA (1987) Organization of arm movements. Motion is segmented. Neurosci 23:39–52.[CrossRef][ISI][Medline]
Song W, Flanders M, Soechting JF (2004) Effect of compliance on haptic perception of curvature. Somatosens Motor Res 21:177–182.[Medline]
Viviani P, Cenzato M (1985) Segmentation and coupling in complex movements. J Exp Psychol Hum Percept Perform 11:828–845.[CrossRef][ISI][Medline]
Voisin J, Benoit G, Chapman CE (2002) Haptic discrimination of object shape in humans: two-dimensional angle discrimination. Exp Brain Res 145:239–250.[Medline]
Yarbus AL (1961) Eye movements during the examination of complicated objects. Biofizika 6:52–56.[Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||