Hat tip to the Eyewitness Identification Reform Blog (10 July), which brings to our attention a paper which casts doubt on the much publicised Illinois Eyewitness Identification Field Study. As the Eyewitness Reform Blog explains:
The [Illinois] study, taking the form of a pilot project spearheaded by Chicago police across three counties, purported to reveal that current identification procedures protected against mistaken identifications better than reforms proposed by respected social scientists, based on extensive research on eyewitness memory. In other words, the Chicago police were happy to report that, notwithstanding the 19 wrongful convictions recorded in Illinois that resulted from faulty eyewitness evidence, everything was just fine and no pesky (scientific) reforms were needed.
Despite the fact that the “report” trumpeting the study was never subjected to peer review, and despite the fact that it was authored by a lawyer for by the very same Chicago police department that had fought reforms in Illinois for many years, these “findings” were trumpeted on the front page of the New York Times and have since served as serious impediments to reform in Legislatures around the country.
The new article, which is forthcoming in the journal Law and Human Behavior, was heralded by a 9 July press release from John Jay College of Criminal Justice:
A blue ribbon panel of social scientists… said the Illinois Eyewitness Identification Field Study, which challenged 30 years of academic research into eyewitness identification procedures, was itself crippled by a design flaw that made the study’s conclusions a dangerous basis for shaping public policy.
In the July 2007 issue of Law and Human Behavior, these social scientists, who included Nobel Laureate Daniel Kahneman of Princeton and Harvard Professor and author Daniel Schachter, contend that the Illinois program’s method of comparing new “sequential double-blind” line-up procedures to traditional non-blind procedures makes the report unreliable in determining effective eyewitness identification procedures. The researchers report that their reading of the Illinois materials forced them to conclude that the Illinois study’s fundamental design flaw, “has devastating consequences for assessing the real-world implications of this particular study.” In fact, the article also says, “…the design guaranteed that most outcomes would be difficult or impossible to interpret. The only way to sort this out is by conducting further studies…”
Kudos to Law and Human Behavior for getting the article through the peer review process so quickly (received: 19 April 2007, accepted: 24 April published online: 4 July), and also for making the full article freely available as a pdf download.
- Daniel L. Schacter, Robyn Dawes, Larry L. Jacoby, Daniel Kahneman, Richard Lempert, Henry L. Roediger (in press). Policy Forum: Studying Eyewitness Investigations in the Field [pdf]. Law and Human Behavior
Abstract: This article considers methodological issues arising from recent efforts to provide field tests of eyewitness identification procedures. We focus in particular on a field study (Mecklenburg 2006) that examined the “double blind, sequential” technique, and consider the implications of an acknowledged methodological confound in the study. We explain why the confound has severe consequences for assessing the real-world implications of this study.