Monday, January 18, 2010

Underdetermination and 'Connecting the Dots' in a Terror Investigation

The failure of counterterrorism officials to 'connect the dots' before the attempted bombing of Northwest flight 253 (1) (2), highlights the very real epistemic difficulty posed by underdetermination.

Underdetermination is an epistemological problem stemming from how facts are related to theories and the conclusions that are entailed. For any fact pattern there may be many—perhaps infinitely many—valid explanatory theories that are incompatible with each other yet may be equally consistent with the facts (3).

[click figure for higher resolution]

In the case of the alleged airplane bomber, Umar Abdulmutallab, despite much evidence to support the conclusion that he posed a terrorist threat, officials did not stop him from boarding his plane and flying to the United States. Independent of any oversights, ineptitude, or bad luck that may have played a part in the investigation, the facts of the case underdetermined what conclusions were to be drawn regarding the threat; the thesis that he posed a terror threat was as equally compatible with the facts as was the thesis that he posed no threat.

To be sure, the facts that would tend to confirm the threat posed by Abdulmutallab were numerous: his father's warning to the US embassy about his son's radical views (4); his presence on the TIDE list, a database maintained by the National Counterterrorism Center on people with known or suspected links to terrorist organizations (5); US Custom's knowledge that he was on the plane (6); the purchasing of a one-way plane ticket in cash and the airline's knowledge that he did not check any bags (7).

However, there was also disconfirming evidence for the theory that he posed a threat, such as one official's belief that the father's warnings were “thin, with minimal information.” (8)

Depending on how this fact pattern is hung, it could plausibly support either theory regarding the threat posed by Abdulmutallab without logical inconsistency. The inference—or theory—that he was a terrorist may successfully correlate the disparate facts about the case but so would the contrary theory.

This is not to say that the theory that he posed a threat was not highly corroborated—indeed, it turned out to be true—nor that it ought not to have been believed or acted upon. Rather, it is to say that hindsight has a tendency to clarify what were previously shrouded relations between facts.

As the case of Abdulmuttalab illustrates, the underdetermination of a theory by the available evidence is simply part of the framework in any investigation.


(1) Superville, Darlene. “Obama Says 'Dots' Not Connected in Airline Attack.” US News & World Report, January 5, 2010.

(2) “FLIGHT 253: Danger Averted, But Why Were Dots Not Connected?” In Depth News, January 10, 2010.

(3) See, Quine, W.V.O. “Two Dogmas of Empiricism,” in Quine, From a Logical Point of View: Nine Logico-Philosophical Essays, 2nd Edition. Cambridge: Harvard UP, 2003. 42-43.

“But the total field is so underdetermined by its boundary conditions, experience, that there is much latitude of choice as to what statements to reevaluate in the light of any single contrary experience.”

(4) Cohen, Stefanie. “
Father of terror suspect reported Mutallab to US Embassy 6 months ago." New York Post, December 27, 2009.

(5) “Umar Farouk Abdul Mutallab: US Knew Suspect May Have Terrorist Ties, AP Reports." Huffington Post, December 26, 2009.

(6) “US officials awaited Nigerian plotter to land.” Agence France Presse, January 7, 2010.

(7) “Umar Farouk Abdulmutallab: one boy’s journey to jihad.” The Sunday Times, January 3, 2010.

(8) DeYoung, Karen and Michael Leahy. “Uninvestigated terrorism warning about Detroit suspect called not unusual.” The Washington Post, December 28, 2009.


im_michael_young said...

Maybe the problem isn't that all theories are undetermined, but that, relative to the relevant theories ("x is a bona fide terror threat"; "x is an innocent traveler"), the data here was not dispositive, and doesn't give us much of a reason to prefer one or the other theory (or much of a reason to act).

My worry is that we don't want to go too far and say (I don't think you do) that all data is compatible with all relevant theories-- if we said that, it would paralyze (rational) action, but, more importantly from the philosophical point of view, it goes too far in its own terms. (Given good enough data (specified however you like) that someone is dangerous, it feel weird to have to say: "Ah! But data is compatible with any number of theories!") As between relevant competing theories, there may often be data that legitimately acts as desiderata in theory choice. But, in this case, there wasn't. If there had been such data, then I think the accusation of "failure to connect to the dots" would be appropriate. The criticism there would just mean that someone (or some system) failed to recognize data bearing on a choice as between two relevant theories.

James said...

I don't think we disagree. Insofar as facts may be "not dispositive" toward, or "UNDERdetermine," a conclusion, then there isn't much reason to prefer one theory to the other.

Creative Commons License
Analytic Politics is licensed under a Creative Commons Attribution-Noncommercial 3.0 United States License.