The ANAM is not a conventional neuropsychological test battery. It
consists of a number of computer-mediated tasks (a good thing) that are
very sensitive to sustained attention and high levels of cognitive
abilities. It is best used when you want to decide who among the Navy
Seals should be selected as a Delta Force member. ANAM does a good job
discriminating impaired pilots from unimpaired. Even in this situation,
some pilots are likely deemed impaired when they could probably function
well as pilots. Since they are compared to a "norm" of other pilots,
they may appear disabled. The absolute level of cognitive abilities
need to competently fly an airplane is largely unknown. When you
administer this test to typical soldiers, an incorrect number appear
disabled (false positives). That is what I mean by "too sensitive": it
is extremely sensitive to small changes in high levels of ability. A
clinical test with the same problem is the PASAT. This results from the
design of the tests and may be a result of norming. I don't know enough
about the norms to make a judgment about them. The fact that the person
representing the tests described them as, " no better then flipping a
coin", suggests that someone did a validity study and found that the
specificity/sensitivity approached random levels among typical
soldiers. I wonder what they used as an external standard for TBI?
The fundamental problem with this whole approach was found when
psychological assessments were used to predict violent behavior among
people discharged from mental institutions. The predictive power was
low. It was low because the incidence of violence is so low. If I have
a "violence test" with an accuracy of 90%, I still end up identifying a
large number of people as potentially violent who will not engage in
violence. The ANAM, or any test battery, has the same problem. If the
base rate of TBI is low (<5%) then it will be impossible to detect with
a neuropsych battery that is insufficiently valid and reliable to detect
an effect that small. When you add in extraneous factors like
malingering and psychological depression, then it will appear that many
soldiers have cognitive impairment when they do not. Someone with PTSD
and no head injury will bomb the ANAM. Some of the tests are so
sensitive to attention that a poorly placed sneeze will lower your score.
A sensible program would be to assess with the ANAM and conventional
clinical tests any soldier who was rendered unconscious or had other
evidence of a head injury. The money being used to test all these
soldiers could be better spent on the rehabilitation of the ones who
have valid TBI.
Mike Williams
Drexel University
On 6/16/10 1:00 AM, Teaching in the Psychological Sciences (TIPS) digest
wrote:
Subject: Re:U.S. Tax Dollars at Work Part 982,542: Pretest But Don't Posttest
for Brain Injury
From: "Mike Palij"<[email protected]>
Date: Tue, 15 Jun 2010 07:21:38 -0400
X-Message-Number: 3
On Mon, 14 Jun 2010 23:09:17 -0700, Mike Wiliams wrote:
>The ANAM battery is far too sensitive for a general application like
>this.
What are the specificity and sensitivity for the ANAM? Also,
what do you mean by "too sensitive"? One interpretation is
that it is good at detecting cases with brain injury (sensitivity)
while the article suggests that it produces false positives
(i.e., 1 - specificity).
>In addition, the base rate of TBI among returning soldiers is
>so low that a screening with a test like this will be far too
>expensive for what it is intended to do.
Please explain this to me. From what I have heard, the rate of
TBI is much higher than (a) that experienced in previous wars
and (b) in the general population. If TBI can be researched in
these groups, why shouldn't it be researched in soldiers from
Iraq and Afghanistan?
If I interpret the article correctly, the pretesting and posttesting
was part of an ongoing study which can be interpreted as gathering
baseline data under different conditions and in different groups.
This seems to me like a worthwhile thing to do unless the ANAM
has really bad diagnostic accuracy which raises the question of why
it was chosen in the first place. In any event, the premature cancellation
of posttests means that the pretest data makes it much more difficult
to reach any conclusions at all (outside of supporting the confirmation
bias)..
>The obvious approach is to only test soldiers who have
>some history of head injury, especially those who were rendered
>unconscious. Do they really expect that soldiers serving in low
>risk assignments will come back with a brain injury and PTSD?
Although I agree that soldiers with a documented case of head
injury or concussion should be used but you seem to suggest
that multiple control groups should not be used. I assume that
prettesting will identify a certain percentage of people with
pre-existing problems -- are soldiers with pre-existing conditions
more likely to develop worsening symptoms after combat experience
relative to soldier with pre-existing conditions who were given
clerical or noncombat assignments? How to soldiers with pre-existing
problems differ from those soldiers who pretest "normal" but
manifest abnormalities on posttest? If pretest "normals" can be
divided into groups on the basis of combat experience (i.e.,
involved in combat vs. no involvement), type of wounding experiences
(i.e., brain involvement vs other body parts), and so on, how
do these differ on posttest as well as other variables, such as diagnosis
of TBI and PTSD? There are many ways to slice this data pie
and I think it premature to think that we know everything we
need to know about this situation.
As for whether soldiers with low risk assignments will come back with
brain injury and PTSD, we'll have some relevant data coming from
studies of people's reactions to the 9/11 attacks in NYC where comparisons
are being made between people immediately involved with the World
Trade Center (WTC) and some distance from the WTC site. If memory
serves, at least once study has shown that the incidence of PTSD
after 9/11 was related to distance from the WTC, with the rate decreasing
as distance increased. One did not have to be at the WTC site to
have developed PTSD.
>The other thing they must include is an assessment of malingering. The
>military disability support system is very ripe for abuse. It is a waste of
>tax dollars to conduct these assessments without checks on malingering.
Perhaps the testing program is set up very badly, with very little thought
as to how it should be implemented and what sorts of controls need to be
maintained (this seems to be a implication of what you are saying). But the
news article provides little info on these points and unless one has the
proposal
for the program and progress reports on how well it is being implemented,
it is very difficult to determine how well it is doing or poorly it is doing.
An alternative interpretation as to why the posttest program was stopped
is that it actually accurately detects a higher rate of valid cases of TBI then
the methods currently in use but because treating such cases is expensive,
it is better to keep the detection of such cases low instead of incurring the
expense of providing treatment. There is much more we need to know about
this situation including (a) if the ANAM is such a lousy screening instrument
why was it used and how did the military's IRB allow it and (b) a convincing
case for stopping the posttesting has not been made, in my opinion, and move
evidence concerning this point needs to be provided.
-Mike Palij
New York University
[email protected]
---
You are currently subscribed to tips as: [email protected].
To unsubscribe click here:
http://fsulist.frostburg.edu/u?id=13090.68da6e6e5325aa33287ff385b70df5d5&n=T&l=tips&o=3124
or send a blank email to
leave-3124-13090.68da6e6e5325aa33287ff385b70df...@fsulist.frostburg.edu
