Police Issues

Thought-provoking essays on crime, justice and policing
 

     Home           About           Index           Links          For educators          Contact          Novel

  < More recent                Technology & Forensics 2016                Prior years >     

Posted 8/3/16

BETTER LATE THAN NEVER (PART II)

DOJ proposes rules for forensic testimony. Do they go far enough?

     For Police Issues by Julius (Jay) Wachtel. In Part I we reported that the Department of Justice was making an (alas, badly belated) effort to address the many wrongful convictions and other miscarriages of justice caused, in part, by forensic “experts” who reach conclusions unsupported by science. To help keep things on the straight and narrow DOJ has begun the process of issuing official regulations that will govern forensic testimony by Federal employees. (DOJ’s move doesn’t apply to state and local forensics experts, but one can imagine they will feel compelled to adjust their practices as well.)

     Why rules are needed was discussed earlier. We’ll start by commenting on those proposed for three forensic disciplines that came under fire in the National Academy of Science’s landmark 2009 report, “Strengthening Forensic Science in the United States: a Path Forward.” (Keep in mind that the regulations are in the proposal stage, with some still out for comments, so don’t expect anything final until the new Administration.) We begin with the granddaddy of all disciplines, fingerprint identification.

Click here for the complete collection of technology & forensics essays

     Latent prints. As best is known, fingerprints are unique. Comparing the individual characteristics of “questioned” (i.e., “latent”) prints recovered at crime scenes to “known” prints of individuals is a long-established practice that is seldom blamed for convicting the innocent. Still, the quality of latent prints varies greatly. What’s more, the process leaves it for examiners to select which “minutiae” (i.e., identifying characteristics) to compare and how many must match to conclude they came from the same source, thus introducing considerable subjectivity. Human nature also gets involved. It’s such things that undoubtedly led to the most celebrated FBI forensic goof ever, when its lab mistakenly identified Brandon Mayfield as the source of fingerprints found on evidence left behind by the perpetrators of the horrific 2004 Madrid train bombings.

     Proposed fix: Examiners could no longer testify that “two friction ridge prints [an impression taken from a person, another recovered at the scene] originated from the same source to the absolute exclusion of all other sources.” Instead, they would have to say that “two friction ridge prints originated from the same source [person] because there is sufficient quality and quantity of corresponding information such that the examiner would not expect to see that same arrangement of features repeated in another source.” Reducing conclusions to a probabilistic certainty is also forbidden.

    And the difference is…: Specialists might appreciate the distinction between the bad, old language (absolutely exclude all other sources) and the good new language (another source not expected.) But jurors are, after all, laypersons, and unless the different approaches are explicitly contrasted, the new, more modest method seems by itself unlikely to lead to a different decision.

     Footwear and tire impressions. In 1985 Derrick Jamison was convicted of robbery-murder and sentenced to death. The evidence against him consisted of a crime scene shoe print from the same brand of footwear that he wore, plus testimony of an alleged accomplice who testified in exchange for a ten-year term. Only thing is, Jameson was six-four in height, while several witnesses, whose testimony was kept from the defense, described the second man as about five-six. Jameson was released from death row and exonerated in 2005.

     Unlike fingerprints, which are unique, shoes and tires of the same brand and model share tread patterns - so called “class characteristics” - that create identical impressions when new. To determine whether an impression was made by a particular shoe or tire requires that it have been “individualized” through wear and tear. Just how many imperfections must correspond to call a match, though, is hard to say:

…there is no consensus regarding the number of individual characteristics needed to make a positive identification, and the committee is not aware of any data about the variability of class or individual characteristics or about the validity or reliability of the method. Without such population studies, it is impossible to assess the number of characteristics that must match in order to have any particular degree of confidence about the source of the impression.

     Proposed fix: As with fingerprints, DOJ’s proposal forbids examiners from excluding all other possibilities. Instead, they would evaluate shoe and tire impressions on a seven-point scale:

1. Identification: …shoe/tire is the source of the impression because there is sufficient quality and quantity of corresponding features such that the examiner would not expect to find that same combination of features repeated in another source…
    
2. Probably Made: …shoe/tire probably made the impression and it is unlikely that another shoe/tire is the source of the impression; however, there are limitations which prevent effecting an identification…
    
3. Could Have Made: …shoe/tire is a possible source of the impression, but other shoes/tires with the same class characteristics are also included in the population of possible sources…
4. Could Not Be Determined
    
5. Indications Did Not Make: …evidence indicates that the shoe/tire is not the source of the impression, but there are limitations which prevent eliminating the shoe/tire…
    
6. Elimination: …shoe/tire is not the source of the impression…
    
7. Unsuitable: …submitted evidence is unsuitable to conduct footwear/tire examinations…

     And the difference is…: Again, jurors aren’t specialists, so whether an analyst settles on #1 (identification) or #2 (probable) might make little difference. Actually, simply mentioning there is a scale, which seems inevitable, could lead jurors to exaggerate the probative value of items with extreme or near-extreme scores. As for #3, given that innumerable pairs of shoes and sets of tires have identical tread patterns when new, “could have made” seems a very risky option. Considering the scientific limitations, it would seem far better to restrict testimony about footwear and tire impressions to instances where examiners are positive about either a match (#1) or an elimination (#6).

     Hair examination. In the notorious 1978 case “Ford Heights Four” an Illinois state forensic analyst waxed astonishingly about the results of a hair comparison: “I couldn’t distinguish if I was looking almost at two hairs,” he testified. “They looked just like one.” Based in part on his account jurors convicted four defendants of rape/murder. Only problem is, all were innocent. It took eighteen years for DNA to clear them and convict the real evildoers.

     Improper hair analysis was cited by the Innocence Project as the second most frequently occurring forensic lapse in 300 DNA exonerations where improper or invalidated forensic techniques had been at least partly to blame. Indeed, the reputation of hair comparison is so grim and its scientific underpinnings so thin that the discipline received an unqualified thumbs-down from NAS: “The committee found no scientific support for the use of hair comparisons for individualization in the absence of nuclear DNA” (at page 161, paragraph 2.)

     Proposed fix: NIJ refuses to throw in the towel. To its credit, it openly acknowledges that “the comparison of hair characteristics does not constitute a basis for personal identification.” Accordingly, examiners may not “state or imply that a hair came from a particular source to the exclusion of all others.” On the other hand, the proposed rule would allow examiners to testify that “the questioned human hair is microscopically consistent with the known hair sample and accordingly, the source of the known hair sample can be included as a possible source of the questioned hair.

     And the difference is…: Irreconcilable. “No scientific support” seems pretty unequivocal. It will apparently remain for the defense to bring up the National Academy’s biting views during cross-examination.

     Everything else. DOJ also issued proposed rules for forensic anthropology, explosives chemistry, explosives and hazardous devices, forensic geology, forensic handwriting analysis, forensic metallurgy, Y chromosome and mitochondrial DNA typing, forensic paints and polymers, forensic toxicology, forensic examination of serology, forensic glass, forensic textile fiber, and general chemistry (click here and here.)

     Unfortunately, some key disciplines – forensic odontology (i.e., bite marks), causes of fire, and toolmarks and firearms – remain unaddressed. As mentioned in “State of the Art…Not!” and the other  posts referenced below, their application and misapplication have led to terrible injustices, and in the case of fire science, the execution of Cameron Todd Willingham, an innocent man. For now, NIJ’s regulations are also silent about bloodstain pattern analysis, or blood spatter, for short. Popularized in Phil Spector’s first murder trial, where its use by the defense helped hang the jury, the method’s inherent subjectivity led NAS to depict its uncertainties as “enormous” (report, p. 179).

Be sure to check out our homepage and sign up for our newsletter

      So where do things stand? DOJ is accepting comments on the proposed rules. (To review those received go to www.regulations.gov, enter “forensic” in the search box, then scroll through the hits.) For example, click here for the only comment we found on hair analysis. Filed by a professor of law and member of the National Commission on Forensic Science, it elaborates on the clash between the discipline’s fundamental uncertainty and DOJ’s proposed language, and suggests that its use would lead to “over-valuation of testimony.”

     That’s not to say that DOJ has a simple task, nor that it’s not trying. But at some point one must really, really stop splitting hairs or, in our favorite turn of phrase, making “distinctions without a difference”. Desperate efforts to keep forensic “sciences” like hair analysis alive virtually guarantee that innocent persons will keep getting convicted and imprisoned, and occasionally worse. It really is time to pull the plug on these derelict disciplines and move on.

     Tomorrow’s my birthday (President Obama’s, too!) Please, DOJ, give us a present!

UPDATES

9/18/19 At UC Santa Cruz, Dr. Ed Green, a paleogeneticist, reportedly developed a way to collect an individual’s full nuclear DNA sequence from a single strand of hair, an accomplishment that previously required having the root. A highly expensive process, it’s been used for more than a year to develop leads in cold-case murders by submitting profiles to DNA databanks.

Did you enjoy this post?  Be sure to explore the homepage and topical index!

Home   Top   Permalink     Print/Save     Feedback     

RELATED POSTS

Accidentally on Purpose     People do Forensics     Guilty Until Proven Innocent     Better Late (I)

State of the Art…Not!     Taking the Bite Out of Bite Marks     DOJ: Texas Executed an Innocent Man

NAS to CSI: Shape up!     Would You Bet Your Freedom on a Dog’s Nose?     House of Cards

Forensics Under the Gun     CSI They’re Not     The Ten Deadly Sins

BETTER LATE THAN NEVER (PART I)

A “hair-raising” forensic debacle forces DOJ’s hand

     For Police Issues by Julius (Jay) Wachtel. Since coming on scene in 2007 we’ve regularly warned about unproved forensic techniques, whose thoughtless use has led to numerous wrongful convictions and at least one execution. Most recently, in “State of the Art…Not!” we criticized the National Institute of Justice for doing little to counter the toxic effects of junk science.

     To be sure, we’re not the only (and certainly not the first) to complain. Seven years ago, after a detailed review of the sorry state of forensics, a truly influential body, the National Academy of Sciences, called for the creation of a standalone agency, independent of the Department of Justice, that would promulgate and enforce best practices in forensic science.

Click here for the complete collection of technology & forensics essays

     That didn’t happen. As we reported earlier, NAS issued a follow-up critique in 2010. NIJ responded with a glossy self-congratulatory piece (we’re doing great!) one year later. Reform would have probably foundered but for a providential 2012 exposé by the Washington Post, which revealed that quasi-scientific conclusions by FBI hair and fiber analysts had brought thousands of criminal prosecutions into question.

     It took three years, but in 2015 the FBI came clean with an unusually detailed press release entitled “FBI Testimony on Microscopic Hair Analysis Contained Errors in at Least 90 Percent of Cases in Ongoing Review.” For some “hair-raising” facts we return to the pages of the Washington Post:

The Justice Department and FBI have formally acknowledged that nearly every examiner in an elite FBI forensic unit gave flawed testimony in almost all trials in which they offered evidence against criminal defendants over more than a two-decade period before 2000….The cases include those of 32 defendants sentenced to death. Of those, 14 have been executed or died in prison, the groups said under an agreement with the government to release results after the review of the first 200 convictions.

     One of these “errors,” the wrongful conviction of Kirk Odom, was made possible by testimony from an FBI lab examiner that a single hair found on a rape victim resembled Odom’s hair (it turns out, mainly as to color,) and that such coincidences were exceedingly rare. Actually, Odom was in a way “lucky,” as there was DNA, and it ultimately fingered someone else. Alas, by the time that was discovered he had already served 21 years.

     When a renowned agency says “Houston, we’ve had a problem” denial is no longer an option. NIJ had to do something, and we’ll get to that in a moment. First, though, it’s important to emphasize that concerns went far beyond hair analysis by a handful of incompetent Feds (ed. note: your author, a retired Fed, was of the other kind.) Using prior posts and the website of the National Registry of Exonerations let’s take a quick, highly abridged trip down the junk science memory lane:

May 2005: Virginia’s crime lab comes under scrutiny after botched DNA tests nearly lead to the execution of a man who served 16 years after being wrongfully convicted of rape and murder.

June 2007: Deemed deficient “across the board,” Houston’s crime lab is blamed for at least three wrongful convictions, including two caused by faulty serology (the exonerees served 17 and 4 ½ years respectively).

September 2008: An audit of the Detroit crime lab’s ballistics work revealed that examiners erroneously declared a match, or overlooked a match, in nineteen out of a sample of 200 cases. Detroit PD shut down the entire crime lab and turned over all forensic analysis to the State. That same year, a Federal report rejected the notion that marks on bullets and cartridge casings can be conclusively linked to a single gun.

November 2008 - January 2009: Six defendants were exonerated by DNA after spending nearly two decades in prison for rape and murder. Their convictions relied in part on reports by serologists that blood found at the scene “could have” come from one of the accused, and that semen came from someone with a blood type “similar” to that of another defendant.

October 2010: A Texas man was exonerated after spending sixteen years on death row. How could this happen? A state’s witness testified that a knife similar to one that the defendant once owned fit the victim’s wounds “like a glove.”

April 2012: California Governor Jerry Brown commuted the sentence of a grandmother who allegedly shook her grandson to death, an act she vociferously denied. Pathologists had testified that the child’s brain evidenced “shaken baby syndrome,” a diagnosis that has since come under fire. In time, medical experts and an appeals court agreed that grandma was innocent. Unfortunately, by then she had already served ten years.

September 2013: Concluding that “the wrong person was prosecuted,” a judge exonerated a man who served 23 years for murder. His conviction was partly due to testimony by a dog handler who insisted that her dog only followed scents in the direction someone traveled. (Dog scent evidence has been heavily criticized. Click here and here.)

June 2014: A defendant who had served three years of a life sentence for murder was acquitted at his second retrial after his lawyers challenged a claim by prosecution experts that handwriting consisting of “55, 65, 9, 10,” “4 time stop,” and “left right left right” sufficed to positively identify its author.

December 2015: A state fire marshal’s testimony that a fire that killed six persons had two points of origin and was set using accelerants helped convict three men. One died in prison. His codefendants served thirty-one years before they were exonerated by testimony that “pour patterns” cited by the prosecution were actually produced by a natural effect called “flashover.” (For another case involving “accelerants” and “pour patterns” click here. Unfortunately, that defendant was executed.)

     Clearly, errors bedevil most forensic disciplines, not just hair analysis. In 2013 (one year after the Post blew the whistle,) to “promote scientific validity, reduce fragmentation, and improve federal coordination of forensic science,” DOJ and the Commerce Department’s National Institute of Standards and Technology formed the National Commission on Forensic Science.

     NCFS’s lofty goal is addressed through seven subcommittees: Accreditation and Proficiency Testing, Interim Solutions, Scientific Inquiry and Research, Medicolegal Death Investigation, Reporting and Testimony, Human Factors, and Training on Science and Law. Last year they started producing “views documents” and “recommendations” that provide limited forms of non-binding advice in each topical area. For example, on May 1, 2015 the NCFS interim solutions subcommittee issued “Views Document on Defining Forensic Science and Related Terms as Adopted by the Commission.” On March 22, 2016 its scientific inquiry panel cranked out “Recommendation to Fund Post-Doctoral Projects to Facilitate Translation of Research into Forensic Science Practice.” On August 11, 2015 members of the medicolegal team released “Views Document on Increasing the Supply of Forensic Pathologists as Adopted by the Commission.” And so on.

     Good enough. But what did the subcommittees think about hair comparison? Is it a good procedure or bad? We scoured the website but found no guidance on whether hairs can be accurately compared, and if so, what conclusions might be drawn. Ditto for analyzing marks on bullets and cartridge casings, dog scent evidence, handwriting and arson. NIST may be a useful exercise in building the discipline of forensics, but it seems to have little if any value as a guide for its actual practice.

Be sure to check out our homepage and sign up for our newsletter

     Mystery solved! It turns out that regulating the trenches of forensics is something that DOJ wishes to reserve for itself. Several days ago the agency released draft rules that specify how ATF, DEA and FBI forensic experts should report their findings, in writing and when testifying in court. These proposals cover toxicology, serology, latent prints, glass, footwear and tire impressions, textiles and fibers, and general chemistry (e.g., drugs and chemicals.) We’ll have more to say about this in Part II.

     Stay tuned!

UPDATES (scroll)

Did you enjoy this post?  Be sure to explore the homepage and topical index!

Home   Top   Permalink     Print/Save     Feedback     

RELATED POSTS

People do Forensics     Better Late Than Never II     State of the Art…Not!

Taking the Bite Out of Bite Marks     One Size Doesn’t Fit All     N.A.S. to C.S.I.: Shape Up!

Would You Bet Your Freedom...?     The Usual Suspects     Forensics Under the Gun     CSI They’re not

Labs Under the Gun

A DEAD MAN’S TALES

Apple extends posthumous protections to a dead terrorist’s cell phone

    For Police Issues by Julius (Jay) Wachtel. Here is something that never happened to your blogger during his ATF career:

Jay and his colleagues pull into Orange U-Store, a garage rental business. They have a Federal search warrant for the unit rented by Billy Badass, who, after a long investigation, was arrested for peddling guns on the street.

Jay approaches the main gate. He flashes his badge. “We’ve got a warrant for Badass’s unit.”

Employee leaves, returns with his boss, Mr. Crook.

Mr. Crook examines the warrant, snickers. “Sorry, boys. Can’t let you in.”

Jay is astounded.  He inspects his badge. It’s only slightly tarnished. “Whaddaya mean…?”

Mr. Crook sighs. “Look, letting you rummage through his stuff would break the bond between us and our clients, whose privacy we have pledged to protect, now and forever.”

Jay reddens. “But…we have a warrant! According to the Fourth Amendment…”

Mr. Crook smiles impishly. “Orange U-Store treasures its standing in the community. We have real good lawyers, too. Are you aware of our market capitalization?”

     Fast-forward to last week. That’s when Tim Cook, Apple’s COO (Chief Operating Officer, or His Majesty, for short) just said “no.” Mr. Cook was responding to the FBI’s request, backed by a court order, that Apple help unlock the iPhone used by the late Syed Farook. On December 2, 2015 Farook and his wife, Tashfeen Malik, murdered fourteen persons and wounded twenty at a workplace party in San Bernardino, California, then came out second in a vicious firefight with local cops.

Click here for the complete collection of technology & forensics essays

     Although the legal and technical aspects of the dispute between Apple and the Feds seem complex, the facts are disarmingly simple. After the shootout, the Feds recovered a cell phone used by Farook. Suspecting that other players might be involved, they want to scan the device for leads. Alas, they don’t have Farook’s password, and he’s in no position to help. To be sure, a supercomputer could feed the phone an endless stream of possible passwords. Apple’s new software, though, poses significant obstacles, as it creates delays between login attempts and wipes the unit’s memory clean after ten unsuccessful tries.

     Apple hasn’t always been so recalcitrant. But in 2013, after fielding thousands of requests for cell phone data over the years, it introduced encryption, then upped the ante one year later by making it supposedly impossible for anyone other than a phone’s owner to log in. Apple and its defenders scoffed at law enforcement claims that these measures would benefit terrorists, calling the concerns wildly exaggerated. After all, there are plenty other places where cops can get what they need.

     With Apple refusing to voluntarily cooperate, FBI agents turned to the “All Writs Act,” a Federal statute that can be used to compel private persons to lend a hand. A magistrate promptly ordered Apple to create software that would allow an unlimited number of passwords to be run through Farook’s phone without risk of purging its contents.

     Despite the horrifying context of the phone’s recovery, and the possibility that crucial leads rest in its memory, Apple demurred. According to its lawyers, the All Writs act is inapplicable. If the Government wants a law that forces technology companies to come to heel, let it pass one. What’s more, Apple insists that its position isn’t just about the law: it’s about principle. An open letter, signed by Mr. Cook, argues that prying into the dead man’s cell phone would “undermine decades of security advancements that protect our customers — including tens of millions of American citizens — from sophisticated hackers and cybercriminals.” In an eloquent doomsday scenario, Apple’s kingpin warns that helping the Feds would set a “dangerous precedent” with potentially catastrophic consequences:

The implications of the government’s demands are chilling. If the government can use the All Writs Act to make it easier to unlock your iPhone, it would have the power to reach into anyone’s device to capture their data. The government could extend this breach of privacy and demand that Apple build surveillance software to intercept your messages, access your health records or financial data, track your location, or even access your phone’s microphone or camera without your knowledge.

     According to the Government, it’s Apple’s concerns that are wildly exaggerated. After all, the Fourth Amendment remains very much in effect. Non-consensual searches still require a warrant based on probable cause, while compelling third parties to release information or cooperate calls for at least a subpoena. No one’s insisting that Apple redesign the phones or make its protective measures easier to defeat. Sure, a permanent back door would be nice, but the Fed’s bottom line is that Apple help unlock this phone, then keep helping on a case-by-case basis, just like in the good old days.

Be sure to check out our homepage and sign up for our newsletter

     But the iPhone no longer indisputably rules the roost. Android’s big splash made privacy a highly competitive commodity. That, according to the Justice Department, is what really explains Apple’s intransigence. It really is all about money. Meanwhile the rest of the tech industry remains mum but vigilant. On the one hand, no one wants to be branded as an enabler of crooks and terrorists. On the other, there is great uncertainty about the future. What will happen if Apple wins? If it loses?

Back in the ATF office, Jay and his colleagues finish cataloguing dozens of guns found in Billy Badass’s storage unit.

Jay turns to Tom. “Did you see Mr. Crook’s eyes bug out when you demonstrated our ‘key’?”

Tom fondles the group’s treasured sledgehammer. “Well, I wasn’t going to beg him to unlock the gate.”

Chuck walks in. He hands Jay a thick envelope. “We subpoenaed Billy Badass’s bank statement.”

Jay examines the contents, frowns. “It’s gibberish. Everything’s encrypted. Ever since Apple got away with it, everyone’s been demanding complete privacy about everything. Can you imagine working tax cases? Frauds?”

Tom returns the sledgehammer to the vault. “Good thing they can’t encrypt garages.”

UPDATES (scroll)

Did you enjoy this post?  Be sure to explore the homepage and topical index!

Home   Top   Permalink     Print/Save     Feedback     

RELATED POSTS

Technology’s Great - Until It’s Not     Loose Lips Enable Terrorists     Ban the Damned Things!

Massacre Control     After the Fact     Sometimes There is no Second Chance

  < More recent                                                                              Prior years >