U.S. Magistrate Judge Iain D. Johnston knows eDiscovery and has a fear of clowns. He showed both in the following footnote 2 in City of Rockford v. Mallinckrodt ARD Inc:
The Court pauses here for a moment to calm down litigators less familiar with ESI. (You know who you are.) In life, there are many things to be scared of, including, but not limited to, spiders, sharks, and clowns – definitely clowns, even Fizbo. ESI is not something to be scared of. The same is true for all the terms and jargon related to ESI. Discovery of ESI is still discovery, governed by the same Federal Rules of Civil Procedure as all other civil discovery. [Citation omitted]. So don’t freak out. City of Rockford v. Mallinckrodt ARD Inc., 326 F.R.D. 489, 492 n.2 (N.D. Ill. 2018)
The case was a massive lawsuit over medication pricing for infants with a rare seizure disorder and adults with multiple sclerosis. The claims included breach of contract with allegations of racketeering and antitrust violations. The projected discovery was going to be in the millions of documents produced by the Defendants. The good news was the parties agreed to the majority of an ESI Protocol. The bad news? Plaintiff wanted the Defendants to sample the null set from their TAR process, which the Defendants were not thrilled to do.
The Rockford eDiscovery Files
The parties both agreed on starting with keyword searches to identify relevant ESI. If the producing party was going to use technology-assisted review (TAR), that party would inform the requesting party. The protocol also included:
- Agreeing on the scope with search terms, dates, and custodians;
- Exchanging search terms;
- Providing search efficiency reports; and
- Procedures for evaluating whether a keyword was too broad.
All in all, the protocol was very thoughtful and earned praise from the judge.
The dispute was what to do after a discovery production. The Plaintiffs proposed having the Defendants provide a sample of the Null Set. While that sounds like technobabble from a sci-fi show, a Null Set is ESI identified as not responsive from document review. A review of the Null Set can identify technically relevant records. This is also summed up by the political quote, “Trust but verify.”
The Court agreed with the Plaintiff’s proposal because 1) Rule 26(g)(1) does not hold producing parties to a perfection standard, but one of reasonableness; 2) testing whether a production is complete with sampling was reasonable under Rule 26(g)(1); and 3) sampling the production was proportional to the need of the case.
The Court explained that validating and quality assurance of a production are “fundamental principles” under Rule 26(g). Rockford, 494. Moreover, parties have been sampling Null Sets of keyword searches long before the world of predictive coding.
The Court further explained that a random sample of a null set was not unreasonably expensive or burdensome. While the Court doubted there would be high costs, the Court left the door open for the Defendant to return if the costs were high. Rockford, 495. Moreover, the Court weighed the case’s claims were significant, with the potential for very high damages. Furthermore, the Defendants were the ones with the vast majority of the ESI in the case and had substantial resources. Based on these factors, the Court found that the burden and expense of sampling the Null Set did not outweigh its likely benefit of ensuring reasonable discovery productions. Id.
Judge Johnston is right that clowns are scary. However, a random sample of a Null Set to validate a search process is nothing that should scare lawyers.
