During January, the Department of Justice (DOJ) released an unsigned statement entitled “United States Department of Justice Statement on the PCAST Report: Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods.” The statement criticizes a scientific report on the state of forensic science published by the President’s Council of Advisors on Science and Technology (PCAST). The 2016 PCAST report found many forensic techniques and methods are not well grounded in science and provided recommendations to employ scientific standards to ensure such techniques and methods are valid.
While it is completely reasonable for scientists to debate about the validity of scientific techniques or methods, the DOJ’s statement is not a scientific argument and does not appear to be informed by science. Therefore the DOJ statement is dangerous and susceptible to misuse raising scientific integrity concerns, especially in light of the Biden-Harris administration’s recent memorandum on restoring trust in government through scientific integrity and evidence-based policymaking. More importantly, the misuse of the DOJ’s statement in court could potentially lead an innocent person to be found guilty of a crime. Thousands of people’s lives have been upended by wrongful convictions – the majority affected being people of color.
As we have argued time and time again, people’s lives are significantly impacted when our government’s policies are not informed by the best available science.
The DOJ’s arguments are unscientific
The DOJ’s statement does not appear to be informed by science. Take, for example, one of the statement’s main criticisms of the 2016 PCAST report: that forensic feature comparison methods do not belong to the field of “metrology” – PCAST contends that such forensic methods do belong to the field of metrology. The DOJ statement claims that if forensic feature comparison methods do not constitute metrology, then the PCAST report’s recommended scientific standards are not acceptable. As a former member of a policy debate team, I would identify this as an argument of topicality – not of science.
We can certainly argue what larger branch of science forensic techniques or methods fall under. What is not being argued here is that such forensic science techniques and methods fall under the umbrella of “science” – that much is agreed on. And if forensic techniques and methods, such as feature comparison methods, are considered scientific, then they should be underpinned by empirical studies providing evidence that they yield an outcome that is correct. Regardless of what branch of science they fall under, as the 2016 PCAST report identified, feature comparison methods are not currently held to a set of scientific standards calling to question the validity of conclusions made via these methods. Furthermore, to the extent the distinction matters, there is a growing body of scientific literature aside from the PCAST report (see here, here, and here) arguing that forensic science fits squarely within the field of metrology.
Another argument made in the DOJ statement is that PCAST’s suggested criteria for validating forensic subjective methods are not acceptable as they are not supported by prior forensics studies that use or require those criteria. This criticism of a lack of citation for such criteria misses a critical problem identified by PCAST: that such criteria are missing, which is why they are needed.
This argument also misses the fact that the criteria set are generally accepted as good scientific practice. As an example of this, here are the last two of the six criteria suggested by PCAST:
(5) Data, software and results from validation studies should be available to allow other scientists to review the conclusions.
(6) To ensure that conclusions are reproducible and robust, there should be multiple studies by separate groups reaching similar conclusions.
I, and I imagine many other scientists, would argue that these criteria are not specific to any scientific discipline. These are general best scientific practice that any scientists would recognize as important to the body of scientific knowledge. Even the DOJ statement acknowledges that “…none of these criteria standing alone are novel or controversial.” As a scientist, every time that I publish results of a study, I do not need to cite an authority on best scientific practice as it is assumed that I followed such practices. And if another expert felt that I did not follow best scientific practices, such as using a large enough sample size, then this would be identified through the peer-review process. The DOJ’s citation criticism provides further evidence that this statement seems to have been written without an appropriate knowledge of the scientific method. And, once again, the DOJ statement misses the point made by PCAST that such criteria are missing from the field and should be established.
DOJ’s statement is influential
The DOJ’s statement is already being used by prosecutors and introduced as evidence in courts, potentially affecting many people’s lives (See, e.g., People v. Auimatagi, Case No. CR-2019-4995-1 (Yolo County, Cal. Super. Ct.) (Feb. 4,
2021). That is why we are submitting a letter to the DOJ today to ask them to swiftly retract this Trump-era anti-science statement. The letter covers more on the unscientific arguments made in DOJ’s statement. When science does not inform policies set forth by the federal government, public health and safety are at-risk. While the DOJ statement poses a risk to public health and safety, it will be most harmful for men and women of color. This is because the continued racial disparity in wrongful convictions, which is due to a host of factors including eyewitness misidentification, racial profiling, and even erroneous testimony from forensic experts. I fear that the misuse of this statement will perpetuate the wrongful conviction of people of color and racism that continues to be pervasive in the US legal system. The Biden-Harris administration has made commitments to racial equity and to strengthen scientific integrity – and here, the two are clearly intertwined. I hope this administration stays true to its promise and swiftly retracts this anti-science statement that is clearly dangerous to public health.