Angus, tell us a bit about yourself. What is your role, and how long have you been working in digital forensics?
Where to begin? I have a lot of different roles these days, but by day I’m a Lecturer in Cybersecurity – currently at the University of York, and also run my own digital forensic consultancy business. I drifted into the forensic world almost by accident back in 2001 when a server I managed was hacked. I presented a paper on the investigation of that incident at a forensic science conference and a few weeks later found myself asked to help investigate a missing person case that turned out to be a murder. There’s been a steady stream of casework ever since.
I’m registered as an expert adviser and most of my recent casework seems to deal with difficult to explain or analyse material. Alongside that, I’ve spent a lot of time (some might say too much) working on standards during my time on the Forensic Science Regulator’s working group on digital evidence and as a member of BSI’s IST/033 information security group and the UK’s digital evidence rep. on ISO/IEC JTC1 SC27 WG4, where I led the work to develop ISO/IEC 27041 and 27042, and contributed to the other investigative and eDiscovery standards.
You’ve recently published some research into verification and validation in digital forensics. What was the goal of the study?
It grew out of a proposition in ISO/IEC 27041 that tool verification (i.e. evidence that a tool conforms to its specification) can be used to support method validation (i.e. showing that a particular method can be made to work in a lab). The idea of the 27041 proposal is that if tool vendors can provide evidence from their own development processes and testing, the tool users shouldn’t need to repeat that. We wanted to explore the reality of that by looking at accredited lab processes and real tools. In practice, we found that it currently won’t work because the requirement definitions for the methods don’t seem to exist and the tool vendors either can’t or won’t disclose data about their internal quality assurance.
The effect of it is that it looks like there may be a gap in the accreditation process. Rather than having a selection of methods that are known to work correctly (as we see in calibration houses, metallurgical and chemical labs etc. – where the ISO 17025 standard originated) which can be chosen to meet a specific customer requirement, we have methods which satisfy much fuzzier customer requirements which are almost always non-technical in nature because the customers are CJS practitioners who simply don’t express things in a technical way.
We’re not saying that anyone is necessarily doing anything wrong, by the way, just that we think they’ll struggle to provide evidence that they’re doing the right things in the right way.
Where do we stand with standardisation in the UK at the moment?
Standardization is a tricky word. It can mean that we all do things the same way, but I think you’re asking about progress towards compliance with the regulations. In that respect, it looks like we’re on the way. It’s slower than the regulator would like. However, our research at York suggests that even the accreditations awarded so far may not be quite as good as they could be. They probably satisfy the letter of the regulator’s documents, but not the spirit of the underlying standard. The technical correctness evidence is missing.
ISO 17025 has faced a lot of controversy since it has been rolled out as the standard for digital forensics in the UK. Could you briefly outline the main reasons why?
Most of the controversy is around cost and complexity. With accreditation costing upwards of £10k for even a small lab, it makes big holes in budgets. For the private sector, where turnover for a small lab can be under £100k per annum, that’s a huge issue. The cost has to be passed on. Then there’s the time and disruption involved in producing the necessary documents, and then maintaining them and providing evidence that they’re being followed for each and every examination.
A lot of that criticism is justified, but adoption of any standard also creates an opportunity to take a step back and review what’s going on in the lab. It’s a chance to find a better way to do things and improve confidence in what you’re doing.
In your opinion, what is the biggest stumbling block either for ISO 17025 specifically, or for standardizing digital forensics in general?
Two things – as our research suggests, the lack of requirements makes the whole verification and validation process harder, and there’s the confusion about exactly what validation means. In ISO terms, it’s proof that you can make a process work for you and your customers. People still seem to think it’s about proving that tools are correct. Even a broken tool can be used in a valid process, if the process accounts for the errors the tool makes.
I guess I’ve had the benefit of seeing how standards are produced and learning how to use the ISO online browsing platform to find the definitions that apply. Standards writers are a lot like Humpty Dumpty. When we use a word it means exactly what we choose it to mean. Is there a way to properly standardise tools and methods in digital forensics?
It’s not just a UK problem – it’s global. There’s an opportunity for the industry to review the situation, now, and create its own set of standard requirements for methods. If these are used correctly, we can tell the tool makers what we need from them and enable proper objective testing to show that the tools are doing what we need them to. They’ll also allow us to devise proper tests for methods to show that they really are valid, and to learn where the boundaries of those methods are.
Your study also looked at some existing projects in the area: can you tell us about some of these? Do any of them present a potential solution?
NIST and SWGDE both have projects in this space, but specifically looking at tool testing. The guidance and methods look sound, but they have some limitations. Firstly, because they’re only testing tools, they don’t address some of the wider non-technical requirements that we need to satisfy in methods (things like legal considerations, specific local operational constraints etc.).
Secondly, the NIST project in particular lacks a bit of transparency about how they’re establishing requirements and choosing which functions to test. If the industry worked together we could provide some more guidance to help them deal with the most common or highest priority functions.
Both projects, however, could serve as a good foundation for further work and I’d love to see them participating in a community project around requirements definition, test development and sharing of validation information.
Is there anything else you’d like to share about the results?
We need to get away from thinking solely in terms of customer requirements and method scope. These concepts work in other disciplines because there’s a solid base of fundamental science behind the methods. Digital forensics relies on reverse-engineering and trying to understand the mind of a developer in order to work out how extract and interpret data. That means we have a potentially higher burden of proof for any method we develop. We also need to remember that we deal with a rate of change caused by human ingenuity and marketing, instead of evolution.
Things move pretty fast in DF, if we don’t stop and look at what we’re doing once in a while, we’ll miss something important.
Read Angus Marshall’s paper on requirements in digital forensics method definition here.
The hottest topic in digital forensics at the moment, standardisation is on the tip of everyone’s tongues. Following various think pieces on the subject and a plethora of meetings at conferences, I spoke to Angus Marshall about his latest paper and what he thinks the future holds for this area of the industry. You can […]
[Case Study] Computer Forensics: Data Recovery & Extraction From Platter Scratched Hard Drives. COMPUTER FORENSICS:
Editor’s note: As a forensic data recovery expert, Salvation DATA receives different data recovery cases every day. Our forensic customers usually turn to us for help when they run into a case they are not able to handle. And among all the data lost situations, platter scratch is one of the most difficult kinds of problem to deal with. So in this issue, let’s see what is the correct forensic process for a platter scratched hard drive.
What is platter scratch?
When platters are damaged it is usually in the form of scratching caused by debris and or the read/write heads when they come in contact during the reading-writing process.
This is also known commonly as a head crash, although that term is often mistakenly used by inexperienced individuals to relate to clicking drives or hard drives that need a read/write head replacement.
Once the platters are scratched to a certain degree this will, in turn, damaged the read/write heads and will render the drive unreadable. Oftentimes this results in a clicking, scratching, chirping, or screeching sounds. However, these sounds don’t automatically mean the platters are scratched.
When the platters are scratched in this manner the drive will not be able to be recovered, the files and data contained on the drive will be lost forever. This is known as a catastrophic head crash and most hard drive failure recovery cannot fix this.
How to work with a hard drive with platter scratch?
Is platter scratch truly unrecoverable? Actually sometimes if the scratches to the platter surface are not too severe, there is still the possibility to recover and extract the data as long as we strictly follow operating procedures.
Stop attempting to read data immediately to avoid further unrecoverable damage.
Open the hard drive in a dust-free environment and inspect for damage.
Remove the damaged read/write head, and replace it with a healthy head. Donor head must be selected by strict matching rule. For example, for Western Digital head replacement, donor drive must match the model number, batch ID, FW version and PN.
After repairing physical damages, we can continue to forensically recover and extract the data from this hard drive with SalvationDATA’s DRS (Data Recovery System).
What tools do you need for this process?
HPE Pro is a hard drive repair tool Head Platter Exchange it is the unarguable and the only equipment built to handle head stack and drive motor issues, in case the drive corruption is not caused by firmware but head stack or drive spin motor. With the pioneer platter exchanger, it can prevent the head from further damage or misalignment due to incorrect operations to maintain the user data intact.
DRS (Data Recovery System) is our next generation intelligent all-in-one forensic data recovery tool that can acquire and recover data from both good and damaged storage media like HDD simply and easily.
How do we know if the hard drive is fixed, and can continue to the next step? DRS’s disk diagnostics feature perfectly helps to solve the problem. DRS is able to scan the source disk in advance. With fairly new FastCheck technology, it allows rapid check within 5 seconds, avoiding the risk of second damage made to an important evidentiary storage device.
Insert the hard disk in DRS, and simply click the one-key Diagnose function to complete the process. DRS will tell you the detailed disk health status in no time!
After repairing the physical damages, the hard drive could still be fragile and easy to fail again. If not handled with care, we may permanently lose the opportunity to recover and extract the data. Therefore, it is crucial to first secure data stored on the hard drive. DRS also provides the solution. The forensic imaging function of DRS secures the evidentiary digital data by creating a physical level sector-by-sector duplication of the damaged hard drive. Once finished, a forensic image will be exactly the same as the source data and can be stored safely and analyzed at any time appropriate.
When dealing with a defective hard drive as in this case, it is recommended to use the Advanced Imaging mode in DRS to help bypass bad sectors and extract as much data as possible. Also, remember to set transmission mode as PIO (low speed) to safely extract the data from such damaged storage device.
Before imaging, we can also check the raw hexadecimal data view in DRS Sector View to make sure data on this damaged hard drive is accessible. Professional data recovery engineers can even acquire more information from this sector view.
SalvationDATA Computer Forensics Scratched Platters
Now with all the problems dealt with, we have one final step to make: recover and extract valuable evidentiary data. Use DRS’s File Recovery & File Carving function to locate and extract important digital files, and generate a forensic report at the end of the process. With DRS’s intelligent recovery technology, investigators can deal with deleted files, formatted partitions, corrupted file system and many other digital data lost situations without any professional skill requirements!
Platter scratch is the nightmare for data recovery engineers. However, it is not impossible to recover data from scratched platters. In this issue, we discussed the standard operating procedure to deal with a hard drive with platter scratch to maximize the possibility to recover and extract valuable evidentiary data. We hope the instructions we provide can help you with your work!
You can also visit our official YouTube channel for more videos: https://www.youtube.com/user/SalvationDataOfficia/featured
Editor’s note: As a forensic data recovery expert, SalvationDATA receives different data recovery cases every day. Our forensic customers usually turn to us for help when they run into a case they are not able to handle. And among all the data lost situations, platter scratch is one of the most difficult kinds of problem […]
In forensics, I get to choose the topics for all my speeches. This means that they’re very personal and matter a lot to me. Issues like racism, homophobia, and gun control are all topics I’ve done speeches on. Being able to have a space to express my opinions, and to hear other opinions has been really empowering for me.
I can express myself
Forensics isn’t just all formal speeches. Most of the events I do is under the interpretation genre, meaning that my speeches are more like acting than formal informative or persuasive speeches. Being able to express myself through my speeches has been a great stress reliever for me.
Being in interpretation events, I get to have a bigger creative licence with my speeches. This allows me to get creative with what I do with my movements and gestures. This creative side of forensics turns words into art.
Through this activity, I’ve developed the important life skill of public speaking. Many people who have not done forensics fear public speaking, but because I have grown such an affinity for it, speeches, and class presentations are much easier, and come quite natural to me.
I have met so many wonderful people through this activity. Most of my closest friends are on my team, and I’ve met so many beautiful people from other teams who are so inspiring, beautiful, and kind.
I stay informed
Most topics in Forensics are about either politics or current events. This means that I am not informed during tournaments, but motivated to read and watch more news, and stay current outside of tournaments. Staying informed on current events now has become important to my daily life.
Especially in college, forensics is a great way to travel to new places. The spring of 2018, my team traveled to Nashville, Tennessee for the Pi Kappa Delta national tournament. It was great to explore the city, and I can’t wait to have the opportunity to travel more.
Forensics has helped be able to accept constructive criticism, which is something a lot of people need to work on. It also has allowed me to feel good about the work I put into each and every one of my speeches. Forensics has given me a space where I feel confident about myself.
Of course, a proper team is not complete without the proper attire. Suits are the norm at tournaments, and if you dig around enough on Ebay or at Goodwill, you’re able to find some great pieces. Suits can be a great conversation starter with another competitor. A good suit is essential for forensics.
I get to talk about topics that matter to me In forensics, I get to choose the topics for all my speeches. This means that they’re very personal and matter a lot to me. Issues like racism, homophobia, and gun control are all topics I’ve done speeches on. Being able to have a space to […]
If you teach Anatomy & Physiology, you know the struggle of the first unit…. it’s HUGE!! … and jam-packed with things that are absolutely essential for students to know in order to be successful in the course. I usually struggle with finding activities to review the body cavities and directional terms. This year, someone suggested using the pickle autopsy and I’m so glad I did!
The lab I used was published in The Forensic Teacher and would be appropriate for either discipline (I teach both this year). Here is the link to the lab I used http://www.theforensicteacher.com/Labs_files/picklelabsheets.pdf A clever fellow teacher friend came up with the storyline that there was a gang war between the Claussens and the Vlasics in the fridge that resulted in no survivors. I loved it so I also used that storyline to frame my lab.
Set Up– The Basics
Now that I had my lab picked out and my story to tell, I had to figure the logistics of how to get everything set up.
First, the pickles….
I found the big jars of dills at Walmart for $5.97 each. The smaller pickles I got because I wanted some of my “victims” to be pregnant (or they could also be small children pickles lol). I had a hard time estimating how many pickles were in the big jars, but these 2 had a total of 33 pickles– more than enough for my classes. The picture below shows them separated by “male” and “female” victims (my “male” pickles are the ones with the stems lol).
Here are all the supplies I used for the lab:
How to make them look like victims….
I glued wiggly eyes onto thumbtacks for their eyes (so I can reuse them)
I also used pellets that go in pellet guns for bullet wounds (I smashed them a little with the hammer first and dipped them into gel food coloring before I stuck them in the “victims”)
I made their heads from an olive stuck on a toothpick– some I even squished so their “brains” fell out a little lol. I also gave all of them a “spine” (a toothpick on the dorsal side just under the skin). I also broke several of the toothpicks so this “injury” might be discovered and included in the story of their “victim”.
All the “victims” had a bead implanted in the vicinity of their heart. If the bead was red, they had a normal heart. If it was black or dark purple, it represented a heart attack. I found that if you make a slit on the side of the pickle (choose a wrinkle), it will often be completely unnoticeable and students will wonder how in the world you got those beads in there! I also slipped in a small green bead in the neck region of a few of the “victims” and told my students I heard that some of the gang members involved in the war were caught raiding the grapes from the fridge and several choked on them when their leader caught them.
I also told them that the gang members were not healthy and many had various diseases and disorders because they didn’t take care of themselves. Many had white beads implanted in various areas. These beads represented a tumor in the particular area. Knotted pieces of rubber bands in the abdominal region represented parasites. Many had broken toothpick “limbs”. I also had several who were pregnant.
This is the sheet of “Helpful Hints” I gave my students with their lab:
A Snapshot of My “Victims”
I separated my “victims” into 4 general types based on their cause of death:
Trauma or internal bleeding (Stabbed or gunshot, injected with red food coloring)
Poisoning/ Drug Overdose (I soaked them in baking soda but didn’t get a very good result)
Heart Attack (black bead instead of red bead in chest)
Drowning (blue food coloring injected in chest area)
My “victims” had multiple things that could have resulted in their deaths, but having 4 major things just helped me keep it organized. I also put them in separate dishes while I plotted their demise 🙂
I also kept them separate in labeled gallon ziplock bags to transport them to school.
The Lab Set Up
I set my lab up as a mini crime scene. I had some fake vampire blood from my forensics class that I also added to help set the scene. I also added in some extra plastic swords and pellets around the “victims”. (I let my students pick their own “victim” from the scene).
Students were in a lab group of 3 per “victim”. In my lab, every student in the group has a specific job and job description. It just helps my lab groups run more smoothly and tends to decrease the possibility that one student does the lion’s share of work. These are the jobs I gave my groups for this lab:
My Take on the Pickle Autopsy Lab
Would I use it again? Absolutely! My students became very proficient at actually using the directional terminology and identifying the body cavities that we talked about in class. I heard many meaningful conversations within the groups… “That’s a break in his arm that’s intermediate between the shoulder and the elbow” “I think this sword went through the abdominal cavity and not the thoracic cavity”…. This was so much better than hearing them try to memorize a diagram or a chart of the directional terms!
They loved getting into our “gang warfare” story. I had them fill out a Coroner’s Report detailing the abnormalities they found both in, and on their “victim”, as well as the location of these abnormalities. Then, they had to determine the cause of death for their victim, supporting their opinion with specific details from their autopsy. At all times within their report, they had to incorporate correct anatomical terminology. Finally, they had to create a narrative of what happened to their “victim” based on the findings from their autopsy. Several groups shared with the class. It was lots of fun!
A Pickle Autopsy? YES! If you teach Anatomy & Physiology, you know the struggle of the first unit…. it’s HUGE!! … and jam-packed with things that are absolutely essential for students to know in order to be successful in the course. I usually struggle with finding activities to review the body cavities and directional […]
Forensic psychology with an emphasis on prison-based rehabilitation is the focus of the Corrective Services 7th Annual Psychology Conference on 29-30 August. Keynote speaker Professor Jim Ogloff AM from @swinburne will discuss ways to reduce violence & serious sexual offending. pic.twitter.com/UmYrol3Yrj Forensic psychology with an emphasis on prison-based rehabilitation is the focus of the Corrective […]
In our Explainer series, Fair Punishment Project lawyers help unpack some of the most complicated issues in the criminal justice system. We break down the problems behind the headlines — like bail, civil asset forfeiture, or the Brady doctrine — so that everyone can understand them. Wherever possible, we try to utilize the stories of those affected by the criminal justice system to show how these laws and principles should work, and how they often fail. We will update our Explainers quarterly to keep them current.
In 1992, three homemade bombs exploded in seemingly random locations around Colorado. When police later learned that sometime after the bombs went off, Jimmy Genrich had requested a copy of The Anarchist Cookbook from a bookstore, he became their top suspect. In a search of his house, they found no gunpowder or bomb-making materials, just some common household tools — pliers and wire cutters. They then sent those tools to their lab to see if they made markings or toolmarks similar to those found on the bombs.
At trial, forensic examiner John O’Neil matched the tools to all three bombs and, incredibly, to an earlier bomb from 1989 that analysts believed the same person had made — a bomb Genrich could not have made because he had an ironclad alibi. No research existed showing that tools such as wire cutters or pliers could leave unique markings, nor did studies show that examiners such as O’Neil could accurately match markings left by a known tool to those found in crime scene evidence. And yet O’Neil told the jury it was no problem, and that the marks “matched … to the exclusion of any other tool” in the world. Based on little other evidence, the jury convicted Genrich.
Twenty-five years later, the Innocence Project is challenging Genrich’s conviction and the scientific basis of this type of toolmark testimony, calling it “indefensible.” [Meehan Crist and Tim Requarth / The Nation]
There are literally hundreds of cases like this, where faulty forensictestimony has led to a wrongful conviction. And yet as scientists have questioned the reliability and validity of “pattern-matching” evidence — such as fingerprints, bite marks, and hair — prosecutors are digging in their heels and continuing to rely on it. In this explainer, we explore the state of pattern-matching evidence in criminal trials.
What is pattern-matching evidence?
In a pattern-matching, or “feature-comparison,” field of study, an examiner evaluates characteristics visible on evidence found at the crime scene — e.g., a fingerprint, a marking on a fired bullet (“toolmark”), handwriting on a note — and compares those features to a sample collected from a suspect. If the characteristics, or patterns, look the same, the examiner declares a match. [Jennifer Friedman & Jessica Brand / Santa Clara Law Review]
Typical pattern-matching fields include the analysis of latent fingerprints, microscopic hair, shoe prints and footwear, bite marks, firearms, and handwriting. [“A Path Forward” / National Academy of Sciences”] Examiners in almost every pattern-matching field follow a method of analysis called “ACE-V” (Analyze a sample, Compare, Evaluate — Verify). [Jamie Walvisch / Phys.org]
Here are two common types of pattern-matching evidence:
Fingerprints: Fingerprint analysts try to match a print found at the crime scene (a “latent” print) to a suspect’s print. They look at features on the latent print — the way ridges start, stop, and flow, for example — and note those they believe are “significant.” Analysts then compare those features to ones identified on the suspect print and determine whether there is sufficient similarity between the two. (Notably, some analysts will deviate from this method and look at the latent print alongside the suspect’s print before deciding which characteristics are important.) [President’s Council of Advisors on Science and Technology]
Firearms: Firearm examiners try to determine if shell casings or bullets found at a crime scene are fired from a particular gun. They examine the collected bullets through a microscope, mark down characteristics, and compare these to characteristics on bullets test-fired from a known gun. If there is sufficient similarity, they declare a match. [“A Path Forward” / National Academy of Sciences”]
What’s wrong with pattern-matching evidence?
There are a number of reasons pattern-matching evidence is deeply flawed, experts have found. Here are just a few:
These conclusions are based on widely held, but unproven, assumptions.
The idea that handwriting, fingerprints, shoeprints, hair, or even markings left by a particular gun, are unique is fundamental to forensic science.The finding of a conclusive match, between two fingerprints for example, is known as “individualization.” [Kelly Servick / Science Mag]
However, despite this common assumption, examiners actually have no credible evidence or proof that hair, bullet markings, or things like partial fingerprints are unique — in any of these pattern matching fields.
In February 2018, The Nation conducted a comprehensive study of forensic pattern-matching analysis (referenced earlier in this explainer, in relation to Jimmy Genrich). The study revealed “a startling lack of scientific support for forensic pattern-matching techniques.” Disturbingly, the authors also described “a legal system that failed to separate nonsense from science in capital cases; and consensus among prosecutors all the way up to the attorney general that scientifically dubious forensic techniques should not only be protected, but expanded.” [Meehan Crist and Tim Requarth / The Nation]
Similarly, no studies show that one person’s bite mark is unique and therefore different from everyone else’s bite mark in the world.[Radley Balko / Washington Post] No studies show that all markings left on bullets by guns are unique. [Stephen Cooper / HuffPost] And no studies show that one person’s fingerprints — unless perhaps a completely perfect, fully rolled print — are completely different than everyone else’s fingerprints. It’s just assumed. [Sarah Knapton / The Telegraph]
Examiners often don’t actually know whether certain features they rely upon to declare a “match” are unique or even rare.
On any given Air Jordan sneaker, there are a certain number of shared characteristics: a swoosh mark, a tread put into the soles. That may also be true of handwriting. Many of us were taught to write cursive by tracing over letters, after all, so it stands to reason that some of us may write in similar ways. But examiners do not know how rare certain features are, like a high arch in a cursive “r” or crossing one’s sevens. They therefore can’t tell you how important, or discriminating, it is when they see shared characteristics between handwriting samples. The same may be true of characteristics on fingerprints, marks left by teeth, and the like. [Jonathan Jones / Frontline]
There are no objective standards to guide how examiners reach their conclusions.
How many characteristics must be shared before an examiner can definitively declare “a match”? It is entirely up to the discretion of the individual examiner, based on what the examiner usually chalks up to “training and experience.” Think Goldilocks. Once she determines the number that is “just right,” she can pick. “In some ways, the process is no more complicated than a child’s picture-matching game,” wrote the authors of one recent article. [Liliana Segura & Jordan Smith / The Intercept] This is true for every pattern-matching field — it’s almost entirely subjective. [“A Path Forward” / National Academy of Sciences”]
Unsurprisingly, this can lead to inconsistent and incompatible conclusions.
In Davenport, Iowa, police searching a murder crime scene found a fingerprint on a blood-soaked cigarette box. That print formed the evidence against 29-year-old Chad Enderle. At trial, prosecutors pointed to seven points of similarity between the crime scene print and Enderle’s print to declare a match. But was that enough? Several experts hired by the newspaper to cover the case said they could not draw any conclusions about whether it matched Enderle. But the defense lawyer didn’t call an expert and the jury convicted Enderle. [Susan Du, Stephanie Haines, Gideon Resnick & Tori Simkovic / The Quad-City Times]
Why faulty forensics persist
Despite countless errors like these, experts continue to use these flawed methods and prosecutors still rely on their results. Here’s why:
Experts are often overconfident in their abilities to declare a match.
These fields have not established an “error rate” — an estimate of how often examiners erroneously declare a “match,” or how often they find something inconclusive or a non-match when the items are from the same source. Even if your hair or fingerprints are “unique,” if experts can’t accurately declare a match, that matters. [Brandon L. Garrett / The Baffler]
Analysts nonetheless give very confident-sounding conclusions — and juries often believe them wholesale. “To a reasonable degree of scientific certainty” — that’s what analysts usually say when they declare a match, and it sounds good. But it actually has no real meaning. As John Oliver explained on his HBO show: “It’s one of those terms like basic or trill that has no commonly understood definition.” [John Oliver / Last Week Tonight]Yet, in trial after trial, jurors find these questionable conclusions extremely persuasive.[Radley Balko / Washington Post]
Why did jurors wrongfully convict Santae Tribble of murdering a Washington, D.C., taxi driver, despite his rock-solid alibi supported by witness testimony? “The main evidence was the hair in the stocking cap,” a juror told reporters. “That’s what the jury based everything on.” [Henry Gass / Christian Science Monitor]
But it was someone else’s hair. Twenty-eight years later, after Tribble had served his entire sentence, DNA evidence excluded him as the source of the hair. Incredibly, DNA analysis established that one of the crime scene hairs, initially identified by an examiner as a human hair, belonged to a dog. [Spencer S. Hsu / Washington Post]
Labs are not independent — and that can lead to biased decision-making.
Crime labs are often embedded in police departments, with the head of the lab reporting to the head of the police department. [“A Path Forward” / National Academy of Sciences] In some places, prosecutors write lab workers’ performance reviews. [Radley Balko / HuffPost] This gives lab workers an incentive to produce results favorable to the government. Research has also shown that lab technicians can be influenced by details of the case and what they expect to find, a phenomenon known as “cognitive bias.” [Sue Russell / Pacific Standard]
Lab workers may also have a financial motive. According to a 2013 study, many crime labs across the country received money for each conviction they helped obtain. At the time, statutes in Florida and North Carolina provided remuneration only “upon conviction”; Alabama, Arizona, California, Missouri, Wisconsin, Tennessee, New Mexico, Kentucky, New Jersey, and Virginia had similar fee-based systems. [Jordan Michael Smith / Business Insider]
In North Carolina, a state-run crime lab produced a training manual that instructed analysts to consider defendants and their attorneys as enemies and warned of “defense whores” — experts hired by defense attorneys. [Radley Balko / Reason]
Courts are complicit
Despite its flaws, judges regularly allow prosecutors to admit forensic evidence. In place of hearings, many take “judicial notice” of the field’s reliability, accepting as fact that the field is accurate without requiring the government to prove it. As Radley Balko from the Washington Post writes: “Judges continue to allow practitioners of these other fields to testify even afterthe scientific community has discredited them, and even after DNA testing has exonerated people who were convicted, because practitioners from those fields told jurors that the defendant and only the defendant could have committed the crime.” [Radley Balko / Washington Post]
In Blair County, Pennsylvania, in 2017, Judge Jolene G. Kopriva ruled that prosecutors could present bite mark testimony in a murder trial. Kopriva didn’t even hold an evidentiary hearing to examine whether it’s a reliable science, notwithstanding the mounting criticism of the field. Why? Because courts have always admitted it. [Kay Stephens / Altoona Mirror]
Getting it wrong
Not surprisingly, flawed evidence leads to flawed outcomes. According to the Innocence Project, faulty forensic testimony has contributed to 46 percent of all wrongful convictions in cases with subsequent DNA exonerations. [Innocence Project] Similarly, UVA Law Professor Brandon Garrett examined legal documents and trial transcripts for the first 250 DNA exonerees, and discovered that more than half had cases tainted by “invalid, unreliable, concealed, or erroneous forensic evidence.” [Beth Schwartzapfel / Newsweek]
In 2015, the FBI admitted that its own examiners presented flawed microscopic hair comparison testimony in over 95 percent of cases over a two-decade span. Thirty-three people had received the death penalty in those cases, and nine were executed. [Pema Levy / Mother Jones] Kirk Odom, for example, was wrongfully imprisoned for 22 years because of hair evidence. Convicted of a 1981 rape and robbery, he served his entire term in prison before DNA evidence exonerated him in 2012. [Spencer S. Hsu / Washington Post]
In 1985, in Springfield, Massachusetts, testimony from a hair matching “expert” put George Perrot in prison — where he stayed for 30 years — for a rape he did not commit. The 78-year-old victim said Perrot was not the assailant, because, unlike the rapist, he had a beard. Nonetheless, the prosecution moved forward on the basis of a single hair found at the scene that the examiner claimed could only match Perrot. Three decades later, a court reversed the conviction after finding no scientific basis for a claim that a specific person is the only possible source of a hair. Prosecutors have dropped the charges. [Danny McDonald / Boston Globe]
In 1982, police in Nampa, Idaho, charged Charles Fain with the rape and murder of a 9-year-old girl. The government claimed Fain’s hair matched hair discovered at the crime scene. A jury convicted him and sentenced him to death. DNA testing later exonerated him, and, in 2001, after he’d spent two decades in prison, a judge overturned his conviction. [Raymond Bonner / New York Times]
Bite mark analysis
In 1999, 26 members of the American Board of Forensic Odontologyparticipated in an informal proficiency test regarding their work on bite marks. They were given seven sets of dental molds and asked to match them to four bite marks from real cases. They reached erroneous results 63 percent of the time. [60 Minutes] One bite mark study has shown that forensic dentists can’t even determine if a bite mark is caused by human teeth. [Pema Levy / Mother Jones]
That didn’t keep bite mark “expert” Michael West from testifying in trial after trial. In 1994, West testified that the bite mark pattern found on an 84-year-old victim’s body matched Eddie Lee Howard’s teeth. Based largely on West’s testimony, the jury convicted Howard and sentenced him to death. Experts have since called bite mark testimony “scientifically unreliable.” And sure enough, 14 years later, DNA testing on the knife believed to be the murder weapon excluded Howard as a contributor. Yet the state continues to argue that Howard’s conviction should be upheld on the basis of West’s testimony. [Radley Balko / Washington Post]
West, who in 1994 was suspended from the American Board of Forensic Odontology and basically forced to resign in 2006, is at least partially responsible for several other wrongful convictions as well. [Radley Balko / Washington Post]
West himself has even discredited his own testimony, now stating that he “no longer believe[s] in bite mark analysis. I don’t think it should be used in court.” [Innocence Project]
The FBI has found thatfingerprint examiners could have an error rate, or false match call, as high as 1 in 306 cases, with another study indicating examiners get it wrong as often as 1 in every 18 cases. [Jordan Smith / The Intercept] A third study of 169 fingerprint examiners found a 7.5 percent false negative rate (where examiners erroneously found prints came from two different people), and a 0.1 percent false positive rate. [Kelly Servick / Science Mag]
In 2004, police accused American attorney Brandon Mayfield of the notorious Madrid train bombing after experts claimed his fingerprint matched one found on a bag of detonators. Eventually, four experts agreed with this finding. Police arrested him and detained him for two weeks until the police realized their mistake and were forced to release him. [Steve Pokin / Springfield News-Leader]
In Boston, Stephan Cowans was convicted, in part on fingerprint evidence, in the 1997 shooting of a police officer. But seven years later, DNA evidence exonerated him and an examiner stated that the match was faulty. [Innocence Project]
A 2012 review of the St. Paul, Minnesota, crime lab found that over 40 percent of fingerprint cases had “seriously deficient work.” And “[d]ue to the complete lack of annotation of actions taken during the original examination process, it is difficult to determine the examination processes, including what work was attempted or accomplished.” [Madeleine Baran / MPR News]
In 1993, a jury convicted Patrick Pursley of murder on the basis of firearms testimony. The experts declared that casings and bullets found on the scene matched a gun linked to Pursley “to the exclusion of all other firearms.” Years later, an expert for the state agreed that the examiner should never have made such a definitive statement. Instead, he should have stated that Pursley’s gun “couldn’t be eliminated.” In addition, the defense’s experts found that Pursley’s gun was not the source of the crime scene evidence. Digital imaging supported the defense. [Waiting for Justice / Northwestern Law Bluhm Legal Clinic] In 2017, a court granted Pursley a new trial. [Georgette Braun / Rockford Register Star]
Rethinking faulty forensics
Scientists from across the country are calling for the justice system to rethink its willingness to admit pattern-matching evidence.
In 2009, the National Research Council of the National Academy of Science released a groundbreaking report concluding that forensic science methods “typically lack mandatory and enforceable standards, founded on rigorous research and testing, certification requirements, and accreditation programs.” [Peter Neufeld / New York Times]
In 2016, the President’s Council of Advisors on Science and Technology (PCAST), a group of pre-eminent scientists, issued a scathing report on pattern-matching evidence. The report concluded that most of the field lacked “scientific validity” — i.e., research showing examiners could accurately and reliably do their jobs. [Jordan Smith / The Intercept] Until the field conducted better research proving its accuracy, the Council stated that forensic science had no place in the American courtroom. The study found that, regarding bite mark analysis, the error rate was so high that resources shouldn’t be wasted to attempt to show it can be used accurately. [Radley Balko / Washington Post]
After the PCAST report came out, then-Attorney General Loretta Lynch, citing no studies, stated emphatically that “when used properly, forensic science evidence helps juries identify the guilty and clear the innocent.” [Jordan Smith / The Intercept] “We appreciate [PCAST’s] contribution to the field of scientific inquiry,” Lynch said, “[but] the department will not be adopting the recommendations related to the admissibility of forensic science evidence.” [Radley Balko / Washington Post]
The National District Attorneys Association (NDAA) called the PCAST report “scientifically irresponsible.” [Jessica Pishko / The Nation] “Adopting any of their recommendations would have a devastating effect on the ability of law enforcement, prosecutors and the defense bar to fully investigate their cases, exclude innocent suspects, implicate the guilty, and achieve true justice at trial,” the association noted. [Rebecca McCray / Take Part]
The NDAA also wrote that PCAST “clearly and obviously disregard[ed] large bodies of scientific evidence … and rel[ied], at times, on unreliable and discredited research.” But when PCAST sent out a subsequent request for additional studies, neither the NDAA nor the Department of Justice identified any. [PCAST Addendum]
This problem is getting worse under the current administration. Attorney General Jeff Sessions has disbanded the National Commission on Forensic Science, formed to improve both the study and use of forensic science, and which had issued over 40 consensus recommendation documents to improve forensic science. [Suzanne Bell / Slate] He then developed a DOJ Task Force on Crime Reduction and Public Safety, tasked with “support[ing] law enforcement” and “restor[ing] public safety.” [Pema Levy / Mother Jones]
But there are also new attempts to rein in the use of disproven forensic methods. In Texas, the Forensic Science Commission has called for a ban on bite marks. “I think pretty much everybody agrees that there is no scientific basis for a statistical probability associated with a bite mark,” said Dr. Henry Kessler, chair of the subcommittee on bite mark analysis. [Meagan Flynn / Houston Press]
A bill before the Virginia General Assembly, now carried over until 2019, would provide individuals convicted on now-discredited forensic science a legal avenue to contest their convictions. The bill is modeled after similar legislation enacted in Texas and California. The Virginia Commonwealth’s Attorneys Association opposes the legislation, arguing: “It allows all sorts of opportunities to ‘game’ the system.” [Frank Green / Richmond Times-Dispatch]
Meanwhile, at least one judge has recognized the danger of forensic expert testimony. In a 2016 concurrence, Judge Catherine Easterly of the D.C. Court of Appeals lambasted expert testimony about toolmark matching: “As matters currently stand, a certainty statement regarding toolmark pattern matching has the same probative value as the vision of a psychic: it reflects nothing more than the individual’s foundationless faith in what he believes to be true. This is not evidence on which we can in good conscience rely, particularly in criminal cases … [T]he District of Columbia courts must bar the admission of these certainty statements, whether or not the government has a policy that prohibits their elicitation. We cannot be complicit in their use.” [Spencer S. Hsu / Washington Post]
Do you wonder how witchcraft and satanic children eating coven stories survive in this era of lies and misdemeanors and wrongful convictions? This article pushes back against what’s coming out of the US WH and DOJ (and some DAs) spiel about forensic reliability. https://injusticetoday.com/faulty-forensics-explained-fe4d41157452
“For over thirty years, FBI experts testified about comparative bullet lead analysis (CBLA), a technique that was first used in the investigation into President Kennedy’s assassination. CBLA compares trace chemicals found in bullets at crime scenes with ammunition found in the possession of a suspect. (…) Although the FBI eventually ceased using CBLA, the Bureau’s conduct in first employing the technique and then defending it after it was challenged provides an insight into how forensic science sometimes works.”
Paul C. Giannelli
“We cannot afford to be misleading to a jury. We plan to discourage prosecutors from using our previous results in future prosecutions.”
Letter from Dwight E. Adams — then FBI lab Director — to FBI Director Robert S. Mueller III
Since the 1960s, testimony by representatives of the Federal Bureau of Investigation in thousands of criminal cases has relied on evidence from Compositional Analysis of Bullet Lead (CABL), a forensic technique that compares the elemental composition of bullets found at a crime scene to the elemental composition of bullets found in a suspect’s possession. Different from ballistics techniques that compare striations on the barrel of a gun to those on a recovered bullet, CABL is used when no gun is recovered or when bullets are too small or mangled to observe striations. Follow us on Twitter: @Intel_Today
A True Story — In 1995, former Baltimore police Sgt. James A. Kulbicki was convicted of first-degree murder. The prosecutor convinced the jury that, in 1993, Kulbicki had killed his mistress — 22-year-old Gina Nueslein– with his off duty .38-caliber revolver.
The scientific evidence was “irrefutable”. The bullets recovered from the victim’s body and from the crime scene had been fired by his gun.
“I wonder what it felt like, Mr. Kulbicki, to have taken this gun, pressed it to the skull of that young woman and pulled the trigger, that cold steel,” the prosecutor asked rhetorically during closing arguments.
Forensic Science — In order to move along a stable straight trajectory, a bullet must spin on itself. To achieve such spin, spiralling “grooves” are machined in the inside of the weapon barrel.
The size of these “grooves” as well as the “lands”, the angle of the grooves, their number per length and the direction of rotation — clockwise or anticlockwise — generally permit to identify a type of weapon. For instance, Colt traditionally uses a left-hand twist while Smith & Wesson uses a right hand twist.
Moreover, specific imperfections of a barrel may allow in some case to match one bullet to a particular weapon. In the best-case scenario, two bullets fired by the same gun will not look alike but they are likely to show areas of resemblance.
When such test is not conclusive or not possible — because the bullets fragments are too small or because the gun is not recovered — it is still possible to analyze the lead content of the fragments and compare it to bullets known to belong to a suspect.
The Scientific Evidence Against Kulbicki
Maryland’s top firearms expert told the jury that the size of the bullet was compatible with Kulbicki’s gun and that he had cleaned the gun.
He added that he had not been able to identify the marks from the barrel.
Last, he testified that the lead content of the bullet that killed his mistress was identical to the content of bullets from a box belonging to Kulbicki.
“Out of the billions of bullets in the world, is this just a coincidence that this bullet ended up in the defendant’s off-duty weapon,” a prosecutor asked.
A prosecutor told the Jury that the evidence presented by the forensic experts was “a significant piece of evidence” and a “major link” to establish Kulbicki’s guilt.
The jurors agreed. Kulbicki was sentenced to life in prison without the possibility of parole.
Joseph Kopera, one of the forensic experts who testified at the trial, presented the formal reports to the defense.
But his working notes were not given to them either at the trial, or at the appeal, which Kulbicki lost.
These notes conflict with the report on all grounds.
Kopera testified that the fragments were consistent with a large-caliber, probably a .38.
His notes tell that the first fragment came from a medium caliber and that the origin of the second fragment could not be determined.
Kopera testified that the gun had been cleaned. His notes read, “Residue in barrel: Yes. Bore condition: Dirty.”
Kopera testified that he could not identify the grooves and lands on the fragments. His notes reveal that the fragment’s land width was 0.072 inches and its groove width was 0.083 inches.
Bullets fired from Kulbicki’s Smith & Wesson revolver had a land width of 0.100 inches and a groove width of 0.113 inches.
The difference is significant enough to state beyond doubts that Kulbicki’s gun did not fire the bullet that killed his mistress.
Kopera testified that he could not identify the twist. His notes indicate that he had detected a “slight left twist” while Kulbicki’s off-duty weapon makes right-twist markings.
Kopera testified that the lead content of the bullets were identical. It was not.
The amount of arsenic in the fragments significantly differed from the one contained in the bullets belonging to Kulbicki.
No Degree — At the trial, Kopera testified that he had an engineering degree from the Rochester Institute of Technology and a mechanical engineering degree from the University of Maryland. Neither institution has ever heard of him.
A Widely Used Technique
“Every critical part of Kopera’s testimony was false, misleading, based on improper assumptions or ignored exculpatory information,” Suzanne K. Drouet, a former Justice Department lawyer, told the judge in her recent motion seeking a new trial for Kulbicki.
“If this could happen to my client, who was a cop who worked within this justice system, what does it say about defendants who know far less about the process and may have far fewer resources to uncover evidence of their innocence that may have been withheld by the prosecution or their scientific experts?”
Following a 2004 National Academy of Sciences report that sharply criticized the FBI’s bullet-lead technique, the agency no longer relies on this method.
After retiring from the firearms section of the Maryland State Police, Kopera committed suicide.
For more than 30 years, his expertise has helped secure countless convictions.
Nationwide, it has been estimated that the method has been used in more than 2,000 cases over four decades.
Several former FBI employees believe that a review of all cases where the CBLA method was used in testimony should be urgently conducted.
“It troubles me that anyone would be in prison for any reason that wasn’t justified. And that’s why these reviews should be done in order to determine whether or not our testimony led to the conviction of a wrongly accused individual,” said Adams, the former FBI lab director.
The second in command agree.
“I don’t believe that we can testify about how many bullets may have come from the same melt and our estimate may be totally misleading,” declared deputy lab director Marc LeBeau in a May 12, 2005, e-mail.
So far, the FBI has rejected such reviews on the basis that it would be very expensive. A sum of US$70,000 was mentioned.
Since 2005, the nonpartisan Forensic Justice Project, run by former FBI lab whistle-blower Frederic Whitehurst, has tried to force the bureau to release a list of bullet-lead cases under the Freedom of Information Act.
In academic circles, some experts have not hidden their anger toward the program and what seems to be an attempt to cover-up decades of fraudulent forensic sciences.
Clifford Spiegelman is a statistician at Texas A&M University. He reviewed the FBI’s statistical methods for the science academy.
“They said the FBI agents who went after Al Capone were the untouchables, and I say the FBI experts who gave this bullet-lead testimony are the unbelievables.”
Several lessons can be gleaned from the CBLA experience. In the conclusion of his excellent paper on the subject, Paul Giannelli wrote:
First, the failure to publish the empirical data that supports scientific conclusions is unacceptable. Scientists “are generally expected to exchange research data as well as unique research materials that are essential to the replication or extension of reported findings.”
Second, defense attorneys were unable to successfully challenge the evidence until William Tobin, the retired FBI expert, became a defense witness. This is not surprising because no defendant, no matter how rich, can conduct extensive empirical studies. A defense expert in a particular case can critique the bases of a prosecution expert’s opinion but can rarely replicate the research upon which that opinion rests.
Forensic Science: Last Week Tonight with John Oliver (HBO)
Forensic science used in criminal trials can be surprisingly unscientific. Maybe a new television procedural could help change the public perception.
“For over thirty years, FBI experts testified about comparative bullet lead analysis (CBLA), a technique that was first used in the investigation into President Kennedy’s assassination. CBLA compares trace chemicals found in bullets at crime scenes with ammunition found in the possession of a suspect. (…) Although the FBI eventually ceased using CBLA, the Bureau’s […]