Military Intelligence

The Winter 2009 Wilson Quarterly reminds us that even in peacetime, military intelligence is an oxymoron:

The most dramatic instance of a failure to override occurred in the Persian Gulf on July 3, 1988, during a patrol mission of the U.S.S. Vincennes. The ship had been nicknamed ­“Robo-­cruiser,” both because of the new Aegis radar system it was carrying and because its captain had a reputation for being overly aggressive. That day, the Vincennes’s radars spotted Iran Air Flight 655, an Airbus passenger jet. The jet was on a consistent course and speed and was broadcasting a radar and radio signal that showed it to be civilian. The automated Aegis system, though, had been designed for managing battles against attacking Soviet bombers in the open North Atlantic, not for dealing with skies crowded with civilian aircraft like those over the gulf. The computer system registered the plane with an icon on the screen that made it appear to be an Iranian F-14 fighter (a plane half the size), and hence an “assumed enemy.”

Though the hard data were telling the human crew that the plane wasn’t a fighter jet, they trusted the computer more. Aegis was in semi-­automatic mode, giving it the least amount of autonomy, but not one of the 18 sailors and officers in the command crew challenged the computer’s wisdom. They authorized it to fire. (That they even had the authority to do so without seeking permission from more senior officers in the fleet, as their counterparts on any other ship would have had to do, was itself a product of the fact that the Navy had greater confidence in Aegis than in a ­human-­crewed ship without it.) Only after the fact did the crew members realize that they had accidentally shot down an airliner, killing all 290 passengers and crew, including 66 ­children.

The tragedy of Flight 655 was no isolated incident. Indeed, much the same scenario was repeated a few years ago, when U.S. Patriot missile batteries accidentally shot down two allied planes during the Iraq invasion of 2003. The Patriot systems classified the craft as Iraqi rockets. There were only a few seconds to make a decision. So machine judgment trumped any human decisions. In both of these cases, the human power “in the loop” was actually only veto power, and even that was a power that military personnel were unwilling to use against the quicker (and what they viewed as superior) judgment of a ­computer.

Oh, the lack of common sense.

Update: another blogger discusses this issue here.