Last fall, U.S. Army Maj. Nidal Malik Hasan allegedly opened fire at Fort Hood, Tex., killing 13 people and wounding 30. In the days after the shootings, information emerged about Hasan's increasingly strange behavior at Walter Reed Army Medical Center, where he had trained as a psychiatrist and warned commanders that they should allow Muslims to leave the Army or risk "adverse events." He had also exchanged e-mails with a well-known radical cleric in Yemen being monitored by U.S. intelligence.
But none of this reached the one organization charged with handling counterintelligence investigations within the Army. Just 25 miles up the road from Walter Reed, the Army's 902nd Military Intelligence Group had been doing little to search the ranks for potential threats. Instead, the 902's commander had decided to turn the unit's attention to assessing general terrorist affiliations in the United States, even though the Department of Homeland Security and the FBI's 106 Joint Terrorism Task Forces were already doing this work in great depth.
The 902nd, working on a program the commander named RITA, for Radical Islamic Threat to the Army, had quietly been gathering information on Hezbollah, Iranian Republican Guard and al-Qaeda student organizations in the United States. The assessment "didn't tell us anything we didn't know already," said the Army's senior counterintelligence officer at the Pentagon.
Secrecy and lack of coordination have allowed organizations, such as the 902nd in this case, to work on issues others were already tackling rather than take on the much more challenging job of trying to identify potential jihadist sympathizers within the Army itself.
Beyond redundancy, secrecy within the intelligence world hampers effectiveness in other ways, say defense and intelligence officers. For the Defense Department, the root of this problem goes back to an ultra-secret group of programs for which access is extremely limited and monitored by specially trained security officers.
These are called Special Access Programs - or SAPs - and the Pentagon's list of code names for them runs 300 pages. The intelligence community has hundreds more of its own, and those hundreds have thousands of sub-programs with their own limits on the number of people authorized to know anything about them. All this means that very few people have a complete sense of what's going on.
"There's only one entity in the entire universe that has visibility on all SAPs - that's God," said James R. Clapper, undersecretary of defense for intelligence and the Obama administration's nominee to be the next director of national intelligence.
Such secrecy can undermine the normal chain of command when senior officials use it to cut out rivals or when subordinates are ordered to keep secrets from their commanders.
One military officer involved in one such program said he was ordered to sign a document prohibiting him from disclosing it to his four-star commander, with whom he worked closely every day, because the commander was not authorized to know about it. Another senior defense official recalls the day he tried to find out about a program in his budget, only to be rebuffed by a peer. "What do you mean you can't tell me? I pay for the program," he recalled saying in a heated exchange.
Another senior intelligence official with wide access to many programs said that secrecy is sometimes used to protect ineffective projects. "I think the secretary of defense ought to direct a look at every single thing to see if it still has value," he said. "The DNI ought to do something similar."
The ODNI hasn't done that yet. The best it can do at the moment is maintain a database of the names of the most sensitive programs in the intelligence community. But the database does not include many important and relevant Pentagon projects.