Class AccuracyMeasures

java.lang.Object

AccuracyMeasures

public class AccuracyMeasures
extends java.lang.Object

AccuracyMeasures

Summary

• A class or application to perform accuracy measurements on the path of a cursor during target selection tasks
• The accuracy measures computed are
  • TRE (target re-entries)
  • TAC (task axis crossings)
  • MDC (movement direction changes)
  • ODC (orthogonal direction changes)
  • MV (movement variability)
  • MO (movement offset)
  • ME (movement error)
• Full descriptions are in Accuracy Measures for Evaluating Computer Pointing Devices by MacKenzie, Kauppinen, and Silfverberg (2001).
• Two uses:
  • As a class embedded in custom-designed software
  • As a utility program executed from a command prompt

AccuracyMeasures Class

As a class embedded in custom-designed software, the AccuracyMeasures class file is placed in the same directory as other class files for the application. This API provides all the details necessary to use the AccuracyMeasures class. An AccuracyMeasures object requires the following arguments (data) which are provided either via the constructor or using the setData method:

• from – a Point2D.Double holding the coordinate of the designated beginning of the trial (center of the "from" target)
• to – a Point2D.Double holding the coordinate of the designated ending of the trial (center of the "to" target)
• width – a double representing the width of the (circular) target to select
• path – a Point2D.Double array holding the points representing the path of the cursor during the trial

Once the data are provided to the AccuracyMeasures object, the accuracy measures are immediately calculated. They are retrieved in the application using get-methods.

AccuracyMeasures Utility Program

As a utility program, AccuracyMeasures is executed from a command prompt. The following is the usage message, if invoked without arguments:

 
 Usage: java AccuracyMeasures -d|infile [-t] [-v]
 
    where -d = demo using internal data set
          infile = read data from 'infile'
          -t = table output
          -v = verbose output
 

There must be at least one command-line argument. This is -d for the demo option, or the name of a file containing the input data to process (see below).

A good way to get started is to execute AccuracyMeasures from the command prompt with the -d (demo) option:

  
PROMPT>java AccuracyMeasures -d
========================
Demo of AccuracyMeasures
========================
From = 856.5, 501.0
To = 515.0, 159.5
Amplitude = 483.0
Width = 60.0
Number of sample points = 149
Sample points...
(842.0, 499.0) (847.0, 501.0) (850.0, 503.0) (852.0, 504.0) (854.0, 505.0) (856.0, 506.0) (858.0, 507.0) (860.0, 507.0) 
(863.0, 508.0) (864.0, 508.0) (865.0, 507.0) (865.0, 505.0) (862.0, 501.0) (858.0, 496.0) (854.0, 490.0) (848.0, 483.0) 
(839.0, 474.0) (830.0, 465.0) (819.0, 455.0) (807.0, 445.0) (794.0, 434.0) (781.0, 423.0) (768.0, 412.0) (754.0, 401.0) 
(742.0, 391.0) (728.0, 380.0) (715.0, 368.0) (699.0, 355.0) (688.0, 345.0) (675.0, 334.0) (659.0, 321.0) (637.0, 302.0) 
(625.0, 292.0) (613.0, 282.0) (602.0, 273.0) (591.0, 265.0) (580.0, 257.0) (562.0, 247.0) (554.0, 242.0) (546.0, 238.0) 
(539.0, 235.0) (534.0, 232.0) (528.0, 230.0) (523.0, 228.0) (520.0, 227.0) (516.0, 226.0) (514.0, 225.0) (511.0, 224.0) 
(508.0, 223.0) (506.0, 222.0) (503.0, 221.0) (501.0, 219.0) (498.0, 218.0) (496.0, 217.0) (493.0, 216.0) (491.0, 215.0) 
(488.0, 213.0) (486.0, 212.0) (484.0, 210.0) (482.0, 209.0) (482.0, 208.0) (481.0, 207.0) (481.0, 206.0) (481.0, 205.0) 
(481.0, 204.0) (481.0, 203.0) (482.0, 202.0) (482.0, 202.0) (482.0, 202.0) (483.0, 201.0) (484.0, 201.0) (485.0, 200.0) 
(486.0, 200.0) (490.0, 201.0) (492.0, 201.0) (495.0, 201.0) (497.0, 201.0) (500.0, 201.0) (503.0, 200.0) (506.0, 199.0) 
(509.0, 199.0) (512.0, 197.0) (515.0, 195.0) (518.0, 193.0) (521.0, 190.0) (523.0, 187.0) (526.0, 182.0) (530.0, 174.0) 
(533.0, 168.0) (536.0, 160.0) (537.0, 156.0) (538.0, 151.0) (540.0, 144.0) (541.0, 141.0) (541.0, 140.0) (542.0, 140.0) 
(543.0, 141.0) (544.0, 142.0) (544.0, 145.0) (545.0, 148.0) (546.0, 154.0) (548.0, 160.0) (549.0, 168.0) (551.0, 176.0) 
(553.0, 185.0) (556.0, 195.0) (558.0, 203.0) (560.0, 211.0) (563.0, 220.0) (566.0, 230.0) (568.0, 238.0) (571.0, 244.0) 
(573.0, 250.0) (574.0, 253.0) (575.0, 258.0) (576.0, 262.0) (575.0, 270.0) (573.0, 269.0) (571.0, 268.0) (569.0, 266.0) 
(567.0, 265.0) (565.0, 263.0) (563.0, 260.0) (562.0, 258.0) (559.0, 255.0) (557.0, 251.0) (552.0, 244.0) (548.0, 237.0) 
(543.0, 232.0) (541.0, 229.0) (538.0, 226.0) (536.0, 223.0) (533.0, 220.0) (531.0, 217.0) (528.0, 213.0) (527.0, 211.0) 
(525.0, 207.0) (523.0, 204.0) (521.0, 201.0) (519.0, 198.0) (517.0, 195.0) (515.0, 191.0) (514.0, 189.0) (512.0, 186.0) 
(510.0, 184.0) (508.0, 181.0) (506.0, 179.0) (504.0, 177.0) (502.0, 175.0) 
Accuracy measures...
TRE = 1
TAC = 4
MDC = 4
ODC = 5
MV = 24.53
ME = 26.84
MO = 20.53
Done!
 

The demo uses embedded data for the from, to, width, and path arguments. In the demo, the data are echoed to the console in a human-readable format followed by the accuracy measures.

The data for the demo are available in a text file that is distributed with AccuracyMeasures. The file is AccuracyMeasures-demo.txt. The same output is generated if AccuracyMeasures is executed as follows:

 
 PROMPT>java AccuracyMeasures AccuracyMeasures-demo.txt -t
 (same output as above)
 

The accuracy measures in the demo indicate 1 target re-entry, 4 task axis crossings, 4 movement direction changes, and so on. Clearly, cursor movement for this trial was not a direct line between the "from" and "to" target points. With a little effort, a graphic rendering of the trial can be created in Excel. Below is an example. The chart shows the "from" target, the "to" target, the cursor path, and the task axis:

Click here to view the spreadsheet used to create the above chart. Using visual inspection, the accuracy measures can be assessed. One target re-entry is evident, as are 4 task axis crossings, 4 movement direction changes, and 5 orthogonal direction changes. The continuous measures of MV, ME, and MO are more difficult to quantify by visual inspection. They can be verified by calculation in the spreadsheet, if desired.

The -t (table) option provides tabular output. While nice for a demo, this format is awkward if analysing the data for numerous trials collected in an experiment. For this, the -t option is omitted:

 
 PROMPT>java AccuracyMeasures AccuracyMeasures-demo.txt
 Participant,Session,Block,Gain,Mode,Trial,A,W,TRE,TAC,MDC,ODC,MV,MO,ME
 P01,S01,B01,100,M1,2,500,60,1,3,4,4,24.530310,20.525079,26.836838
 

The output contains two lines: a header line and a data line. Each entry in the header identifies the type of information in the corresponding entry in the data. The entries are full-precision, comma-delimited for importing into a spreadsheet or statistics application for further analyses.

In the output above, there are 15 comma-delimited values on each line. The first 8 hold a "code" that associates test conditions with the 7 accuracy measures that follow.

The code entries in both lines are provided in the input data file. This allows the AccuracyMeasures utility to be as flexible as possible. If your experiment included "feedback mode" as independent variable, then "Feedback_Mode" can be provided as an entry in the code header. The corresponding entries in the data lines might contain "Audio" for trials using audio feedback or "Vibrotactile" for trials using vibrotactile feedback. The use of codes, including the number of codes, is entirely determined in the data provided to the AccuracyMeasures utility.

Input Data Format

To help describe the input data format, the data in AccuracyMeasures-demo.txt are copied below (slightly abbreviated):

 
# ========================================          
# Test data for the AccuracyMeasures class          
# ========================================          
Participant,Session,Block,Gain,Mode,Trial,A,W       
P01,S01,B01,100,M1,2,500,60                         
856.5,501.0                                         
515.0,159.5                                         
60                                                  
842,847,850,852,854,856,858,860,863,864,865,865,862, ...
499,501,503,504,505,506,507,507,508,508,507,505,501, ...

(data for the next trial begin here, after a blank line)
 

For each trial, the input data may begin with comments. Comment lines begin with "#". They are ignored. The first non-blank, non-comment line must contain the code header. The comma-delimited entries in the code header appear in the first line of output. The code header is provided once only – with the first trial of input. Following the code header is the code. This contains the comma-delimited codes associated with this trial. Five lines of data follow. The first three contain, in order, the from (x₁, y ₁), to (x₂, y ₂), and width data. The last two lines contain the cursor path data, first the x-points, then the y-points. Obviously, there must be the same number of comma-delimited entries in these two lines.

Following the data, there is a blank line to separate the data for this trial from the data for the next trial. This pattern is repeated for each trial to be processed.

Using the AccuracyMeasures utility requires input data as described above. As an example, in the paper FittsTilt: The Application of Fitts' Law To Tilt-based Interaction by MacKenzie and Teather (2012), accuracy measures were computed and analysed in an experiment using device tilt to control a cursor and select targets. Cursor path data were collected for every trial in the experiment. There were more than 10,000 trials, so a large amount of data were collected. The following three files show the general idea of how the AccuracyMeasures utility was used for this experiment. The files are abbreviated to show only the first 50 trials.

• AccuracyMeasures-input.txt - the raw data from the first 50 trials, in the required format.
• AccuracyMeasures-output.txt - the output from the AccuracyMeasures utility.
• AccuracyMeasures-spreadsheet.xlsx - the Excel spreadsheet into which the data above were imported.

Good luck. If you have any comments or suggestions, please get in touch (mack "at" yorku.ca).

Author:

Scott MacKenzie, 2013-2023

Constructor Summary

Constructors

Constructor and Description
AccuracyMeasures() Empty constructor.
AccuracyMeasures(int taskTypeArg, java.awt.geom.Point2D.Double fromArg, java.awt.geom.Point2D.Double toArg, double widthArg, java.awt.geom.Point2D.Double[] pathArg) Construct an AccuracyMeasures object.

Method Summary

Modifier and Type	Method and Description
java.awt.geom.Point2D.Double	getFrom() Returns the "from" point for this trial.
int	getMDC() Returns the number of movement direction changes.
double	getME() Returns the movement error.
double	getMO() Returns the movement offset.
double	getMV() Returns the movement variability.
int	getODC() Returns the number of orthogonal direction changes.
java.awt.geom.Point2D.Double[]	getPath() Returns the array of path points.
int	getTAC() Returns the number of task axis crossings.
java.awt.geom.Point2D.Double	getTo() Returns the "to" point for this trial.
java.awt.geom.Point2D.Double	getTransformedFrom() Returns the transformed from point for this trial.
java.awt.geom.Point2D.Double[]	getTransformedPath() Returns the transformed path array for this trial.
java.awt.geom.Point2D.Double	getTransformedTo() Returns the transformed to point for this trial.
int	getTRE() Returns the number target re-entries.
double	getWidth() Returns the target width.
void	setData(int taskTypeArg, java.awt.geom.Point2D.Double fromArg, java.awt.geom.Point2D.Double toArg, double widthArg, java.awt.geom.Point2D.Double[] pathArg) Set the data, or set new data, for this AccuracyMeasures object.
void	setMDCThreshold(int thresholdArg) Set the threshold for computing movement direction changes.
void	setODCThreshold(int thresholdArg) Set the threshold for computing orthogonal direction changes.
void	setTACThreshold(int thresholdArg) Set the threshold for computing task axis crossings.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AccuracyMeasures

public AccuracyMeasures()

Empty constructor. Must follow with setData.

AccuracyMeasures

public AccuracyMeasures(int taskTypeArg,
                        java.awt.geom.Point2D.Double fromArg,
                        java.awt.geom.Point2D.Double toArg,
                        double widthArg,
                        java.awt.geom.Point2D.Double[] pathArg)

Construct an AccuracyMeasures object.

Parameters:

taskTypeArg - the task type

fromArg - the from coordinate

toArg - the to coordinate

widthArg - the target width

pathArg - an array of coordinates for the cursor path

Method Detail

setData

public void setData(int taskTypeArg,
                    java.awt.geom.Point2D.Double fromArg,
                    java.awt.geom.Point2D.Double toArg,
                    double widthArg,
                    java.awt.geom.Point2D.Double[] pathArg)

Set the data, or set new data, for this AccuracyMeasures object.

Parameters:

taskTypeArg - the task type

fromArg - the target from coordinate

toArg - the target to coordinate

widthArg - the target width

pathArg - a double array containing the path points

setTACThreshold

public void setTACThreshold(int thresholdArg)

Set the threshold for computing task axis crossings. The threshold is the pixel distance the path must deviate from the task axis to be considered "off" the task axis. The default is 5.

NOTE: Execute setData after using this method (so the accuracy measures are computed using the new threshold.)

Parameters:

thresholdArg - the threshod for computing TAC

setMDCThreshold

public void setMDCThreshold(int thresholdArg)

Set the threshold for computing movement direction changes. The threshold is the y-axis distance between consecutive transitions from "moving left" to "moving right", or vice versa.

NOTE: Execute setData after using this method (so the accuracy measures are computed using the new threshold.)

Parameters:

thresholdArg - the threshold value

setODCThreshold

public void setODCThreshold(int thresholdArg)

Set the threshold for computing orthogonal direction changes. The threshold is the x-axis distance between consecutive transitions from "moving forward" to "moving backward", or vice versa.

NOTE: Execute setData after using this method (so the accuracy measures are computed using the new threshold.)

Parameters:

thresholdArg - the threshold value

getTransformedPath

public java.awt.geom.Point2D.Double[] getTransformedPath()

Returns the transformed path array for this trial. The transformation rotates and translates the path points, maintaining their relationship with the from and to points, such that the new from point is (0.0, 0.0) and the new to point is (x, 0.0), with x equal to the specified movement amplitude for the trial (i.e., in code, x = Math.hypot(from.x - to.x, from.y - to.y)). Note that the first and last points of the transformed path array will only be (0.0, 0.0) and (x, 0.0) if the path for this trial started at from and finished at to.

Returns:

a double array of the points in the transformed path

getTRE

public int getTRE()

Returns the number target re-entries.

A target re-entry is logged each time the path enters the target after the first entry. To implement this, a string pattern is created representing the state of the path for each sample point: "0" = outside the target, "1" = inside the target. Then, the pattern is smoothed, excluding single- or double-runs of 0s or 1s. The smoothed pattern is then scanned with TRE incremented for each occurrence of "01"(excluding the first occurrence).

Returns:

the number of target re-entries

getTAC

public int getTAC()

Returns the number of task axis crossings.

A task axis crossing is logged each time the path transitions from "above" the task axis to "below" the task axis. A threshold is used to prevent artificially inflating TAC for reasonably direct movements that oscillate about the task axis. The default threshold is 5. A point is "above" if it is > threshold units above the task axis or "below" if it is > threshold units below the task axis. A TAC is logged for each above-below or below-above transition. Transition detection includes hysteresis (i.e., the last state is maintained until the opposite threshold is crossed).

See also setThresholdTAC.

Returns:

the number of task axis changes

getMDC

public int getMDC()

Returns the number of movement direction changes.

A movement direction change is logged when the cursor path transitions from "moving left" to "moving right", or vice versa. To implement this, the movement direction for each pair of points in the path is coded using "1" if the path is moving left or "0" if the path moving right. An example pattern might be "1111001111011100010000111111". Then, a smoothing algorithm traverses the pattern to remove single- or double-runs of 0s or 1s. For the example, this yields "1111111111111100000000111111". Then, each transition point is compared with the next transition point. (The final point is considered a transition point.) If the difference in the y coordinates exceeds a threshold, an MDC is logged. The default threshold is 10. For the example, MDC = 2 (provided the threshold requirement is met for both transitions).

See also setThresholdMDC.

Returns:

the number of movement direction changes

getODC

public int getODC()

Returns the number of orthogonal direction changes.

An orthogonal direction change is logged when the cursor path transitions from "moving forward" to "moving backward", or vice versa. To implement this, the movement direction for each pair of points in the path is coded using "1" if the path is moving forward or "0" if the path is moving backward. An example pattern might be "1111001111011100010000111111". Then, a smoothing algorithm traverses the pattern to remove single- or double-runs of 0s or 1s. For the example, this yields "1111111111111100000000111111". Then, each transition point is compared with the next transition point. (The final point is considered a transition point.) If the difference in the x coordinates exceeds a threshold, an ODC is logged. The default threshold is 10. For the example, ODC = 2 (provided the threshold requirement is met for both transitions).

See also setThresholdODC.

Returns:

the number of orthogonal direction changes

getMV

public double getMV()

Returns the movement variability.

Returns:

the movement variability

getME

public double getME()

Returns the movement error.

Returns:

the movement error

getMO

public double getMO()

Returns the movement offset.

Returns:

the movement offset

getFrom

public java.awt.geom.Point2D.Double getFrom()

Returns the "from" point for this trial.

Returns:

the from point

getTo

public java.awt.geom.Point2D.Double getTo()

Returns the "to" point for this trial.

Returns:

the to point

getTransformedFrom

public java.awt.geom.Point2D.Double getTransformedFrom()

Returns the transformed from point for this trial. The transformed from point is (0.0, 0.0).

Returns:

the transformed from point

getTransformedTo

public java.awt.geom.Point2D.Double getTransformedTo()

Returns the transformed to point for this trial. The transformed to point is (x, 0.0) with x equal to the specified movement amplitude for this trial (i.e., in code, x = Math.hypot(from.x - to.x, from.y - to.y)).

Returns:

the transformed to point

getWidth

public double getWidth()

Returns the target width.

Returns:

the target width

getPath

public java.awt.geom.Point2D.Double[] getPath()

Returns the array of path points.

Returns:

a double array of path points