Courts often rely on expert witnesses to better understand complex scientific or clinical issues that may be unfamiliar to judges, attorneys, or layperson jurors. Sometimes, courts appoint these experts directly, but more often, the adversarial parties select and retain the experts. In criminal trials, the prosecution or defense might select an expert to answer a question about scientific evidence, including mental health evidence on issues such as trial competence, sanity at the time of the offense, or risk of future violence. The opportunity for opposing sides to select experts raises obvious questions: Can experts who are retained by one side in adversarial legal proceedings offer the court genuinely objective findings and expert opinions? Or are these experts inevitably biased by the adversarial arrangements in which they work? Consistent with these concerns, recent research reveals strong evidence of adversarial allegiance, the tendency for some experts to drift from strictly objective findings toward findings that better support the party that retained them.
Legal scholars and other observers have long speculated that expert witnesses are probably biased toward the party that pays their fees. However, despite these long-standing concerns about allegiance and warnings against this type of bias, there has historically been no rigorous research on the topic. This article reviews recent evidence of allegiance among forensic experts, explores potential explanations for allegiance effects, and discusses the wide range of forensic settings across which evidence allegiance effects have been observed.
Forensic Assessment Instruments Allow Studies of Allegiance
One of the reasons there has been little systematic proof of adversarial allegiance is that isolating and quantifying the effect of allegiance across cases is quite difficult. Experts may answer different types of questions in different cases, use different methodologies in similar cases, and express their opinions in different terms, making it hard to compare the results across many cases. One important development from forensic psychology, however, has made it easier to study certain types of expert opinions. Specifically, forensic psychologists increasingly use forensic assessment instruments to measure constructs such as psychopathic personality, risk of violence, or risk of sexual re-offense. These instruments have become increasingly popular; thus, it is common for evaluators to use the same instrument across many similar cases, and for opposing evaluators to use the same instrument to assess the same offender. Many of these forensic assessment instruments are well designed and well researched, with ample evidence that the instruments can be highly reliable. In other words, different clinicians using the same instrument to score the same offender usually assign very similar scores, with no evidence of one type of rater assigning higher scores than another. However, if scores assigned in adversarial contexts reveal poorer agreement in the field and the direction of disagreement appears systematically related to the forensic expert’s side of retention, such a pattern would suggest that the poorer agreement is somehow attributable to adversarial arrangements.
Field Research Suggests Allegiance
Since 2008, studies have used exactly this strategy of examining instrument scores to explore the possibility of allegiance among forensic mental health experts. In particular, researchers have examined scores on the Hare Psychopathy Checklist-Revised (PCL-R), a widely used instrument designed to assess psychopathic personality features that has become common and influential in forensic assessments of risk of violence or sexual violence, and the Static-99R, the most widely used instrument to quantify the risk of sexual reoffending.
In the first of these instrument-focused field studies, researchers collected PCL-R scores assigned by prosecution-retained and defense-retained psychologists in trials addressing sexual offenders. In the majority of these cases, there was a significant difference between the two PCL-R scores, and in these instances, it was the prosecution-retained evaluator who had assigned a higher score. In other words, prosecution-retained experts assigned higher risk scores to offenders (consistent with the prosecution’s perspective of the case), and defense-retained experts assigned lower risk scores to the same offenders (consistent with the defense perspective). Similarly, researchers found continued evidence of bias in the scores experts assigned on instruments designed to predict future sexual offending.
Similar patterns of score discrepancies that suggest adversarial allegiance have emerged in case law reviews of U.S. and Canadian criminal cases. Evidence of allegiance effects has also emerged in surveys of score reporting and interpretation practices. Evaluators who tend to work for the prosecution favor reporting practices that suggest higher risk and less measurement error, while those who work for the defense favor reporting practices that suggest lower risk and more measurement error.
Thus, a growing body of recent research suggests that the scores clinicians assign and the way in which they interpret those scores are, at least to some extent, related to the side for which they are working.
Allegiance Effects and Selection Effects
Field studies strongly suggest adversarial allegiance, but there are other possible reasons that opposing experts may reach different opinions. One possibility involves selection effects. Because experts are not randomly assigned to sides, the field study findings demonstrating differences in expert opinions may be a product of how attorneys select experts for their cases (selection effects), as opposed to the experts’ decision-making after they begin to work on the cases (allegiance effects). Indeed, research demonstrates substantial individual differences among experts in their tendencies to reach particular forensic opinions, a pattern of variability that may be exploited by savvy attorneys. A savvy defense attorney, for example, would almost certainly select an expert with a greater history of reaching opinions that support insanity, incompetence, or low violence risk. A savvy prosecuting attorney would almost certainly select an expert with a greater history of reaching opinions that support sanity, competence, or high violence risk.
Experimental Research Demonstrates Adversarial Allegiance
Traditionally, observational field studies are the first step in scientific research, allowing researchers to observe phenomena and generate hypotheses. To conclusively demonstrate a phenomenon or to identify its cause, however, researchers must conduct strict experiments in which they carefully control the many factors that could influence or explain the phenomenon observed. To demonstrate the phenomenon of adversarial allegiance—as distinct from other forms of bias or selection effects— requires a situation in which attorneys are randomly assigned forensic experts, experts are randomly assigned to attorneys, experts have access to exactly the same materials, and researchers have access to data from all experts who conducted an evaluation. Only in such a scenario, discrepant scores from opposing evaluators would be clearly attributable to adversarial allegiance rather than attorney selection effects, expert selection effects, or genuine differences in the data available to opposing parties.
In the only experimental study of this sort, researchers recruited over 100 practicing, doctoral-level forensic psychologists and psychiatrists and deceived them to believe they were performing a formal, large-scale forensic consultation. Unbeknownst to them, these forensic experts were randomly assigned to either a prosecution- allegiance or defense-allegiance group in which they were paid by what they believed was either a public defender service or a special prosecution unit. These participants met with an attorney who posed as leading either the public defender service or the specialized prosecution unit and requested that they score particular risk instruments on the basis of extensive offender records (a type of consultation that is common in forensic practice). Participants were led to believe that, as a group, they were reviewing and scoring cases from a large cohort. In truth, each participant was scoring the same four case files of authentic case materials.
Overall, the risk measure scores assigned by prosecution and defense experts showed a clear pattern of adversarial allegiance. That is, experts who believed they were working for the prosecution tended to assign somewhat higher scores and experts who believed they were working for the defense tended to assign somewhat lower risk scores. Allegiance effects were stronger for the PCL-R, a measure that requires more subjective clinical judgment, than for the Static-99R, a more structured measure that permits less judgment.
Of course, not every expert demonstrated biased scoring, but when researchers compared all possible pairings of opposing defense and prosecution experts, many demonstrated large score differences in the direction of adversarial allegiance. This pattern could not be explained by chance or by other factors such as evaluator experience. Overall, findings from this rigorous experiment provided strong evidence that even scores on ostensibly objective forensic instruments can be compromised by adversarial allegiance, at least among some experts.
Potential Explanations for Adversarial Allegiance
Findings from field studies, surveys, and the experiment described earlier converge to show that performing an evaluation for one side in an adversarial setting can influence an expert’s decisions. This adversarial allegiance effect is distinct from, but can be compounded by, selection effects, evaluator differences, and other possible causes of differences between opposing evaluators.
In any adversarial case, there is a pool of possible evaluators that an attorney may retain. These evaluators differ to some extent in their attitudes, their thresholds for coming to certain conclusions, and the typical scores they assign on forensic assessment instruments. Attorneys who are familiar with these differences can take advantage of them by selecting an expert who appears more favorable toward their side of the case.
After an attorney retains an expert, however, the expert’s opinion (at least in some cases) begins to favor the retaining party to a greater extent than the case data warrant. None of the allegiance studies have empirically identified the precise mechanisms that cause these allegiance effects, but there seem to be three broad and overlapping theories. The first broad theory is relational: Adversarial allegiance results from social and psychological processes that encourage evaluators to feel as if they are on a side or team. The second broad theory is that allegiance results from common decision-making errors and cognitive biases. This viewpoint also holds that adversarial allegiance is unintentional but attributes allegiance effects primarily to well-known, common cognitive errors in human judgment, such as confirmation bias. Although the relational and cognitive-error explanations for adversarial allegiance effects are conceptually distinct, they probably interact. The third broad theory differs from the first two by assuming that adversarial allegiance results from more intentional processes and motives that can best be summarized as financial gain.
Each of these mechanisms could have contributed to the allegiance effects among evaluators in field and experimental studies, but of course, different clinicians are probably influenced to different degrees by different mechanisms.
Adversarial Allegiance Is Not Unique to Mental Health Experts
Although adversarial allegiance has been best studied in soft sciences such as psychology, there is no reason to believe that only experts in mental health disciplines feel a pull toward allegiance. Observers have documented anecdotal evidence of adversarial allegiance in a variety of fields, ranging from forensic medicine to accounting. Scrutiny of the forensic sciences underscores the breadth of biases similar to adversarial allegiance. In 2005, the U.S. Congress mandated the National Research Council (NRC) to review the state of forensic science. The NRC critically assessed a wide range of forensic science disciplines, such as analyses of DNA, hair, fibers, tool marks, bite marks, and ballistics. In a 2009 report, the NRC concluded that forensic scientists are prone to a variety of contextual biases, including some that emerge because they lack independence from those requesting their services. For example, forensic science labs tend to work closely with law enforcement officials and prosecutors. Thus, they may operate much as members of those teams, receiving information about suspects and case developments that is irrelevant to the analyses they perform. In other words, operating as a member of a side or team (a form of allegiance effect) may lead to errors in interpretation and decision-making.
Consistent with the NRC’s concerns, emerging research has clearly documented subjectivity and bias even in the forensic science procedures that courts have considered most reliable, such as analyses of DNA and fingerprints. This research revealing biases in forensic science procedures has proceeded in parallel with similar research addressing adversarial allegiance in forensic mental health evaluations. Although both research programs have revealed compelling evidence of bias, neither has yet developed to a point that specifically identifies, at least via controlled experiments, the precise mechanisms or processes underlying these biases.
All the forensic experts who work in adversarial contexts are asked to form opinions by parties that are motivated to achieve a certain outcome. Consequently, all forensic experts inevitably approach their task with certain assumptions, expectations, and even desires. These influences— which may be described as allegiance effects, expectancy effects, or context effects, depending on the details—may shape forensic experts’ perceptions and interpretations of the data they are asked to consider.
References:
- Kassin, S. M., Dror, I. E., & Kukucka J. (2013). The forensic confirmation bias: Problems, perspectives, and proposed solutions. Journal of Applied Research in Memory and Cognition, 2, 42–52.
- Murrie, D. C., & Boccaccini, M. T. (2015). Adversarial allegiance among forensic experts. Annual Review of Law and Social Science, 11, 37–55. doi:10.1146/ annurev-lawsocsci-120814-121714
- Murrie, D. C., Boccaccini, M. T., Guarnera, L. A., & Rufino, K. A. (2013). Are forensic experts biased by the side that retained them? Psychological Science. doi:10.1177/0956797613481812.
- Murrie, D. C., Boccaccini, M. T., Johnson, J. T., & Janke, C. (2008). Does interrater (dis)agreement on Psychopathy Checklist scores in sexually violent predator trials suggest partisan allegiance in forensic evaluations? Law and Human Behavior, 32(4), 352.
- Murrie, D. C., Boccaccini, M. T., Turner, D. B., Meeks, M., Woods, C., & Tussey, C. (2009). Rater (dis) agreement on risk assessment measures in sexually violent predator proceedings: Evidence of adversarial allegiance in forensic evaluation? Psychology, Public Policy, and Law, 15(1), 19–53.
- National Research Council, Committee on Identifying the Needs of the Forensic Science Community. (2009). Strengthening forensic science in the United States: A path forward. Washington, DC: The National Academies Press.