The Ariel Performance Analysis System (APAS)
considered as the most accurate motion analysis system in the world. The following are
third party studies and comments on the APAS accuracy: There are detailed studies to verify the accuracy of the APAS system. In
addition there are responses to questions posted by the Biomech-L list. The following are
few of the studies and the responses.
Response to Biomch-L
|
To: Multiple recipients of list BIOMCH-L Hi, We have the APSAS and CES here at McGill University
Physed Dept. We bought it in 1993. We had graduate students collect data on it but no one
here has done any validity studies on it.
My apologies for the delay to all of you requesting this
information Here is the summary of the replies I received:
Subject: summary
APAS accuracy Date: Sun, 25 Feb 1996 11:31:23 -0600 From: "Claudia A. Angeli (Lansing, MI/MSU)" |
Response No 1
I have an APAS & MacReflex system and use them both .
APAS uses the DLT so it's accuracy is dependent on the accuracy of the calibration. I
constructed my own frame which consisted of aluminum rods with reflective balls attached
hanging from the ceiling. I then had the frame surveyed and locations determined with
respect to my floor mounted force plate. The engineering firm said the accuracy of the
centroid locations are +/- .5 mm. By having the frame coordinated with respect to my force
plate coordinate system I eliminated having to take extra steps to determine where under
the foot my center of pressure was. I checked by system using a second calibration frame I
had constructed for outdoors and found the accuracy to be roughly +/- 0.5cm for a volume
of (1.23m X 2.45m X 1.84m) using 3cm diameter markers.
|
(This represent less then .25 percent error.) This was deemed more than acceptable for my purposes. I have requested to
Gideon that they make available the residual values that the DLT produces which would tell
you the accuracy of the calibration, but his reply has always been to check it with the
camera locations calculated by the DLT.
(This is done now with APAS revision 8.3. [by ADI
Inc.]) |
|
I also had the front middle of the camera lens location
calculated when they surveyed & APAS would always come back with coordinates within a
couple of cm, however, the DLT calculates the location of the camera film plane which is
pretty tough to determine by surveying. Thus, I assumed I had a good calibration if I was
close to the surveyed camera locations.
The bottom line is APAS uses the DLT & like any biomechanist worth their mass in dust,
you should use the most accurate calibration possible for the volume you are working with
and have some check as to the accuracy of each calibration performed. |
Response No. 2
We've had the Ariel system for a few years and we did check
the accuracy when we first got it. We concluded that the accuracy was acceptable as far as
the location of points in 2D. We didn't test the 3D.
Our tests showed that in 2D, the
joint coordinates as well as the segmental kinematics
were of acceptable accuracy.
The first test you can do is to take a grid of points on a
flat surface and determine how well those points are located. You can place the grid in
different orientations. That is for static analysis. For dynamic analysis take a
record-player turntable (hard to find now with CD's but they're around). Place one or more
markers on the turntable and turn it on. For 2D analysis test the system by orienting the
turntable perpendicular to the camera. For 3D analysis, you can do different trials where
you put the turntable in different planes relative to the camera. In either case, because
you know the speed of the turntable and location of the points, you know how the
kinematics should come out. Check that against Ariel's output. An old way to test
kinematic analysis is to drop an object. The position, velocity and acceleration of the
object is predictable based on Newton's law.
|
Recently however, there was a paper published that did
deal with accuracy of three dimensional and angular estimates on the APAS system. The
referecnce is as follows:
Klein, P.J. & DeHaven, J.J., (1995). Accuracy of
three-dimensional linear and angular estimates obtained with the ariel performance
analysis system. American Academy of Physical Medicine and Rehabilitation, 76,(183-189). |
Response No. 3
Many of our users have performed accuracy studies for
static, dynamic, angular measures, wide-angle-lens, and even underwater. Several of these
are posted on the Ariel Dynamics, Inc. Home Page at the following address:
/. You can access these studies under the heading of Product Comparison
In particular, please access the study that compares the APAS to the Acension Flock of Birds.
The APAS has proven to be extremely accurate by testing at the leading institutions in the
world:
- NASA Biomechanics Lab: 5
APAS Systems
- NASA Medical Sciences: 2 APAS Systems
- Stanford University: 1 APAS System
- Karolinska Institute (Sweden): 5 APAS Systems
- UCLA: 2 APAS Systems
- University of Jyvaskyla (Finland): 4 APAS Systems
Response No. 4
|
I will bring one or two things to your attention about
systems that use reflective markers of signiificant size as does the APAS system (among
others).
One problem with such system is partial obstruction of the marker. As the s/w calculates
the centroid, if a marker is partially blocked, the centroid will appear to move as it
becmes blocked.
Another problem is the calibration. This should take 2 forms, viz. correction of the
camera optics and calibration of the space for the DLT that is used by most passive marker
systems. Stay away from systems in which the calibration frame is not a rigid, accurate
structure. It is impossible to obtain accurate readings if the calibration system is bad. |
The better systems will also calculate lens corrections for
your cameras. The others will just try to incorporate these problems in the DLT, which is
not designed to handle such effects.
A simple(ish) way to *verify* the basic accuracy of a
system is to attach a pair of markers to a rigid bar of fixed length. Film the bar from a
number of positions (I would refer to the ASME B89 standard for these: it suggests 20
positions for a cubic volume that encompass the various edges and diagonals of a cube). Of
course, the bar should be the same length in all 20 positions. It won't be, but deviation
from the fixed length will give you a good idea as to how good or how poor the unit is. It
will also be obviouse if there is a problem with some positions of the bar.
Response No. 5.I have worked extensively with APAS for the last four
or five years with numerous sport and occupational applications. I've also done a number
of accuracy checks on various aspects of the system:
Positional Data
I digitised various frames (2 cubes and a triangular
pyramid) of known dimensions to check the accuracy of displacement data. Errors of less
than 1% were averaged inside the calibrated boundaries. Outside these boundaries,
however, errors started to increase markedly (so now we calibrate all over the place).
Linear velocity and acceleration Only basic tests done on
this parameter as it is pretty hard to set up a test rig that is accurate to the level
required. We used a ball drop experiment as well as a comparison of ball velocity with the
results recorded from a system we use for golf (which is very accurate). Errors in the
ball drop test were fairly small in terms of acceleration (about 9.81 m/s/s ) we achieved.
The ball speed comparison returned differences of about 1%. Once again, set-up was
important to these results with a high speed video image achieving better results than a
normal speed image.
Angular Displacement
|
Basic results less than 1% error (tested using the angles
present in the frames analysed for positional data).
A lot of the errors I've found can be reduced with good set-up procedures. We have learnt,
through quite a lot of testing in different situations and finding odd results here and
there, that thorough and thoughtful set-up is essential. Video presents a few problems as
well in the fact that it has less resolution than high speed film and normal speed video
is a bit slow for a lot of activities that we deal with. This is a problem with video
analysis in general, rather than APAS. All in all, we are confident in the results we
get. |
Response No. 6.
We purchased a NTSC APAS sytem in ... and a PAL System
in..... I used both systems extensively and was - of course - concerned with the
'accuracy'. Whilst the topic is too lengthy to put it in one mail, I briefly relate a few
thoughts to you: We tested APAS by placing objects in the gym (Hurdles, standards, boxes
...). Then we determined the precise 3-d- coordinates of points on these objects using a
theodolite connected to an HP 'totalizer' (I'm not sure of the english word, however it is
a system used by surveyors which is highly accurate). We also determined the camera
positions. Then we digitized points, the rationale being that the fundamental measure to
compute derivatives of are the x,y,z coordinetes - hence it doesn't matter wether these
points are moving or stationary. We also considered using mechanically defined systems
such as a pedulum or a free falling mass but saw no advantage (assuming the time base is
correct) over stationary points. We also had the advantage of using these known points
either as calibration or as 'unknown' points.
Now, 'accuracy' has to be clearly defined for the context.
This means, you should consider your object space size, your pixel resolution and the
quantity to be derived. We found for an object space of 6x6x3 meters mean position
errors of 0,2 to 1,5 cm. I consider this good.
Let me highlight just a few of points:
- We found the even distribution of control points covering at
least in the total object space, possibly a little beyond, to be very important. DLT does
not function when you 'calibrate' a large object space with a small control grid.
- Camera placement and number of cameras used are critical
parameters! We use three cameras, possibly in connection with a NAC HSV400 high speed
system. A recent study of horse trotting showed excessive 'Z' deviations if no frontal
view was incorporated.
- Our PAL cameras are phase locked. We use a strobe flash for
synchronization of the start frame.
- When smoothing, I try to find 'tale telling' variables, such
as the vertical acceleration of the CM as external validation of the results and the
smoothing.
- We make extensive use of the zoom feature.
My bottom line is that the system is
as accurate as one can expect from a video based DLT system. We used it successfully in a
variety of laboratory and field situations
|