Cognitive Bias in Intelligence Analysis
Testing the Analysis of Competing Hypotheses Method
- Reveals that a key element of current training provided to the UK and US intelligence communities (and likely all 5-EYES and several European agencies) does not have a proven ability to mitigate cognitive biases
- Demonstrates that judging the credibility of information from human sources means that intelligence analysis faces greater complexity and cognitive strain than non-intelligence analysis
- Explains the underlying causes cognitive biases, based on meta-analyses of existing research
- Shows that identifying the ideal conditions for intelligence analysis is a more effective way of reducing the risk of cognitive bias than the use of ACH
Recent high-profile intelligence failures – from 9/11 to the 2003 Iraq war – prove that cognitive bias in intelligence analysis can have catastrophic consequences. This book critiques the reliance of Western intelligence agencies on the use of a method for intelligence analysis developed by the CIA in the 1990s, the Analysis of Competing Hypotheses (ACH). The author puts ACH to the test in an experimental setting against two key cognitive biases with unique empirical research facilitated by UK’s Professional Heads of Intelligence Analysis unit at the Cabinet Office, and finds that the theoretical basis of the ACH method is significantly flawed. Combining the insight of a practitioner with over 11 years of experience in intelligence with both philosophical theory and experimental research, the author proposes an alternative approach to mitigating cognitive bias that focuses on creating the optimum environment for analysis, challenging current leading theories.
Introduction
Part I: Intelligence, Bias and Belief Acquisition: A Theoretical Perspective
1. Intelligence and Belief AcquisitionA Taxonomy of IntelligenceHow can Knowledge be Acquired?
2. The Efficacy of ACH in Establishing Epistemic Justification and Mitigating Cognitive BiasThe Analysis of Competing Hypotheses (ACH) Method Consistency of Information as an Indicator of TruthDiagnostic Value of Information as an Indicator of TruthSubjective Bayesianism as an Indicator of TruthCredibility of Information as an Indicator of TruthACH and the Mitigation of Cognitive BiasIntelligence, Bias and Belief Acquisition: A Theoretical Perspective
Part II: Intelligence, Bias and Belief Acquisition: An Empirical Perspective
3. The Efficacy of ACH in Mitigating Serial Position Effects and Confirmation Bias in an Intelligence Analysis ScenarioProcedure Results Discussion and Conclusions
4. Predicting Serial Position EffectsPredicting Serial Position Effects An Examination of Previous Serial Position Effects Prediction ModelsThe Impact of Analytical Conditions on Serial Position Effects
5. Predicting Confirmation BiasAn Examination of Previous Confirmation Bias Prediction Models The Impact of Analytical Conditions on Confirmation Bias Intelligence, Bias and Belief Acquisition: An Empirical Perspective
6. Reducing the Risk of Cognitive Bias in Intelligence Analysis Recommendations for Future Research and Best Practice for Intelligence Analysis
AppendicesReferencesBibliography
Whitesmith (Ministry of Defence, United Kingdom) argues against the key assumption that we have a "fix" for cognitive biases that creep into intel analyses, causing acceptance of findings that confirm erroneous beliefs. […] Summing Up: Recommended.
Whitesmith (Ministry of Defence, United Kingdom) argues against the key assumption that we have a "fix" for cognitive biases that creep into intel analyses, causing acceptance of findings that confirm erroneous beliefs. […] Summing Up: Recommended.
As we move further into the information revolution, the importance of robust, scientific methods of critical thinking becomes ever more pressing. In this context, Martha Whitesmith’s empirical analysis of the ACH technique is extremely timely and insightful, and will be of considerable significance to contemporary intelligence communities across the world.
Martha Whitesmith’s book insightfully demonstrates how to improve the effectiveness of structured analysis methods by first improving the intelligence verification process.
