No doubt this
was published to coincide with Gen. Hayden facing confirmation questioning
today as the new CIA chief. Perhaps he can answer questions just why TrailBlazer
was favored over ThinThread, and if there are any conflicts of interest
regarding contractors.
This is
another example of where political expediency and analysis collide, real
capability and options are compromised.
Color
highlights, italics and bullets, mine. Kwc
NSA killed system that
sifted phone data legally
Sources say project
was shelved in part because of bureaucratic infighting
By Siobhan Gorman, Baltimore Sun Reporter, May 17, 2006, 10:27 PM EDT
WASHINGTON -- The National Security Agency developed a pilot program in
the late 1990s that would have enabled it to gather and analyze massive amounts
of communications data without running afoul of privacy laws. But after the
Sept. 11 attacks, it shelved the project -- not because it failed to work --
but because of bureaucratic infighting and a sudden White House expansion of
the agency's surveillance powers, according to several intelligence officials.
The agency opted instead to adopt only one component of the program, which
produced a far less capable and rigorous program. It remains the backbone of
the NSA's warrantless surveillance efforts, tracking domestic and overseas
communications from a vast databank of information, and monitoring selected
calls.
Four intelligence officials knowledgeable about the program agreed to discuss
it with The Sun only if granted
anonymity because of the sensitivity of the subject.
The program the NSA rejected, called ThinThread,
was developed to handle greater volumes of information, partly in expectation
of threats surrounding the millennium celebrations. Sources say it bundled
together four cutting-edge surveillance tools. ThinThread would have:
§
Used more sophisticated methods of
sorting through massive phone and e-mail data to identify suspect
communications.
§
Identified U.S. phone numbers and other
communications data and encrypted
them to ensure caller privacy.
§
Employed an automated auditing system to monitor how
analysts handled the information, in order to prevent misuse and improve
efficiency.
§
Analyzed the data to identify
relationships between callers and chronicle their contacts. Only when evidence of a potential threat had been
developed would analysts be able to request decryption of the records.
An agency spokesman declined to discuss NSA operations.
"Given the nature of the work we do, it would be irresponsible to discuss
actual or alleged operational issues as it would give those wishing to do harm
to the U.S. insight and potentially place Americans in danger," said NSA
spokesman Don Weber in a statement to The Sun.
"However, it is important to note that NSA takes its legal
responsibilities very seriously and operates within the law."
In what intelligence experts describe as rigorous testing of ThinThread in
1998, the project succeeded at each task with high marks. For example, its
ability to sort through massive amounts of data to find threat-related
communications far surpassed the existing system, sources said. It also was
able to rapidly separate and encrypt U.S.-related communications to ensure
privacy.
But the NSA, then headed by Air Force Gen. Michael V. Hayden,
opted against both of those tools, as well as the feature that monitored
potential abuse of the records. Only the data analysis facet of the program
survived and became the basis for the warrantless surveillance program.
The decision, which one official attributed to "turf
protection and empire building," has undermined the
agency's ability to zero in on potential threats, sources say. In the wake of
revelations about the agency's wide gathering of U.S. phone records, they add,
ThinThread could have provided a simple solution to privacy concerns.
A number of independent studies, including a classified 2004 report from the
Pentagon's inspector-general, in addition to the successful pilot tests, found that
the program provided "superior processing, filtering and protection of
U.S. citizens, and discovery of important and previously unknown targets,"
said an intelligence official familiar with the program who described the
reports to The Sun. The Pentagon
report concluded that ThinThread's ability to sort through data in 2001 was far
superior to that of another NSA system in place in 2004, and that the program
should be launched and enhanced.
Hayden, the president's nominee to lead the CIA, is to appear Thursday before
the Senate Select Committee on Intelligence and is expected to face tough
questioning about the warrantless surveillance program, the collection of
domestic phone records and other NSA programs. While the furor over warrantless surveillance, particularly
collection of domestic phone records, has raised questions about the legality
of the program, there has been little or no discussion about how it might be
altered to eliminate such concerns.
ThinThread was designed to address two key challenges:
§
The NSA had more information than it could digest, and,
increasingly,
§
its targets were in contact with people in the US whose
calls the agency was prohibited from monitoring.
With the explosion of digital communications, especially phone calls over
the Internet and the use of devices such as BlackBerries, the NSA was
struggling to sort key nuggets of information from the huge volume of data it
took in.
By 1999, as some NSA officials grew increasingly concerned about
millennium-related security, ThinThread seemed in position to become an
important tool with which the NSA could prevent terrorist attacks. But it was
never launched. Neither was it put into effect after the attacks in 2001.
Despite its success in tests, ThinThread's information-sorting system was
viewed by some in the agency as a competitor to Trailblazer, a $1.2 billion program that was being developed
with similar goals. The NSA was committed to Trailblazer, which later ran into
trouble and has been essentially abandoned.
Both programs aimed to better sort through the sea of data to find key tips to
the next terrorist attack, but Trailblazer had more political support
internally because it was initiated by Hayden when he first arrived at the NSA,
sources said. NSA managers did not want to adopt the data-sifting
component of ThinThread out of fear that the Trailblazer program would be
outperformed and "humiliated," an intelligence official
said.
Without ThinThread's data-sifting assets, the warrantless surveillance program
was left with a sub-par tool for sniffing out information, and that has
diminished the quality of its analysis, according to intelligence officials. Sources say the NSA's existing
system for data-sorting has produced a database clogged with corrupted and
useless information.
The mass collection of relatively unsorted data, combined with system flaws
that sources say erroneously flag people as suspect, has produced numerous
false leads, draining analyst resources, according to two intelligence
officials. FBI agents have complained in published reports in The New York Times that NSA leads have
resulted in numerous dead ends.
The privacy protections offered by ThinThread were also abandoned in the
post-Sept. 11 push by the president for a faster response to terrorism.
Once President Bush gave the go-ahead for the NSA to secretly gather and
analyze domestic phone records -- an authorization that carried
no stipulations about identity protection -- agency officials regarded
the encryption as an unnecessary step and rejected it, according to two
intelligence officials knowledgeable about ThinThread and the warrantless
surveillance programs.
"They basically just disabled the [privacy] safeguards," said one
intelligence official. Another, a
former top intelligence official, said that without a privacy requirement,
"there was no reason to go back to something that was perhaps more
difficult to implement."
However two officials familiar with the program said the encryption feature
would have been simple to implement. One said the time required would have
involved minutes, not hours.
Encryption would have required analysts to be more disciplined in their
investigations, however, by forcing them to gather what a court would consider
sufficient information to indicate possible terrorist activity before
decryption could be authorized.
While it is unclear why the agency dropped the component that monitored for
abuse of records, one intelligence official noted that the feature was not
popular with analysts. It not only tracked the use of the database, but hunted
for the most effective analysis techniques, and some
analysts thought it would be used to judge their performance.
Within the NSA, the primary advocate for the ThinThread program was Richard Taylor, who headed the agency's
operations division. Taylor who has retired from the NSA, did not return calls
seeking comment.
Officials say that after the successful tests of ThinThread in 1998, Taylor
argued that the NSA should implement the full program. He later told the 9/11 Commission that ThinThread could have
identified the hijackers had it been in place before the attacks, according to
an intelligence expert close to the commission.
But at the time, NSA lawyers viewed the program as too aggressive. At that
point, the NSA's authority was limited strictly to overseas communications,
with the FBI responsible for analyzing domestic calls. The lawyers feared that
expanding NSA data collection to include communications in the United States
could violate civil liberties, even with the encryption function.
Taylor had an intense meeting with Hayden and NSA lawyers. "It was a very
emotional debate," recalled a former intelligence official.
"Eventually it was rejected by [NSA] lawyers."
After the 2001 attacks, the NSA lawyers who had blocked the program reversed their position and approved the
use of the program without the enhanced technology to sift out terrorist
communications and without the encryption protections.
The NSA's new legal analysis was based on the commander in
chief's powers during war, said former officials familiar with the program. The
Bush administration's defense has rested largely on that argument since the
warrantless surveillance program became public in December.
The strength of ThinThread's approach is that by encrypting
information on Americans, it is legal regardless of whether the country is at
war, according to one intelligence official. Officials familiar with Thin Thread say some within NSA were
stunned by the legal flip-flop. ThinThread "was designed
very carefully from a legal point of view, so that even in non-wartime, you
could have done it legitimately," the official said.
In a speech in January, Hayden said the warrantless surveillance program was
not only limited to al-Qaida communications, but carefully implemented with an
eye toward preserving the Constitution and rights of Americans. "As the director, I was the one
responsible to ensure that this program was limited in its scope and
disciplined in its application," he said.
http://www.baltimoresun.com/news/nationworld/bal-nsa517,0,5970724.story?coll=bal-home-headlines
