Statistics for eDiscovery: Judge Grimm And Others Recommend Using Sampling To Evaluate The Quality Of Our eDiscovery Process. How Do We Do That?
There are a number of sampling techniques that can be used for quality control. Here is one method that can be used to determine whether you have missed an unreasonable number of responsive documents. A highly accurate system will correctly identify most of the responsive documents. Therefore, there should be few documents that are incorrectly identified as nonresponsive. A quality measure can address the question: Of the documents that have been marked nonresponsive, what percentage of those documents is actually responsive? This measure is called “elusion,” reflecting the number of documents that have eluded identification.
Rather than estimate this value directly, elusion leads to a quality control measure with a very simple statistical test to determine whether the elusion rate exceeds a reasonable criterion. What percentage of documents could you leave behind and still find your process to be reasonable? In the Roitblat, et al. study (2010), using the original review as a standard, two new review teams showed elusion rates of around 6 – 7% (Team A: 6.6%, Team B: 6.3%). In other words, of the documents identified as nonresponsive by Team A, 6.6% of them had been identified as responsive by the original review.
To use the elusion test, we need to specify a level of confidence and a criterion percentage, called “prevalence.” We want to say that responsive documents were no more common in the set identified as nonresponsive than our specified prevalence rate. To do that, we draw a random sample of documents from those that have been identified as nonresponsive. The size of the sample depends on our prevalence rate and desired confidence level. If none of the documents in this sample is found to be responsive, then we can say with confidence that there were no more responsive documents left behind than the prevalence rate. With 95% confidence and a prevalence of 0.5%, you would have to review 598 documents. If none of the documents in the sample was responsive, then you could say with 95% confidence that no more than 0.5% of the documents we rejected were actually responsive.
Source: Statistics For eDiscovery (PDF) - Used by Permission of Herbert L. Roitblat, Ph.D.
Series: Statistics for eDiscovery – Blog Posts – Orange Legal Technologies
Orange Legal Technologies helps corporate legal departments and their outside counsel conduct the critical electronic discovery task of document review by providing advanced predictive coding technologies and expert reviewer assistance to accelerate the electronic discovery process. To learn more about our automated predictive review services, click here.
OrangeLT's Digital Evidence Services help law firms and corporations conduct the critical eDiscovery tasks of legal hold, data mapping and data collection for multiple sources from multiple locations without incurring the burdensome time, cost and equipment expenses.
To learn more about these services, click here.
The OneO® Discovery Platform is an integrated, web-accessible, forensically sound electronic discovery platform that enables online analytics, processing, and review of data from the security of a hosted centralized repository.
To learn more about OneO®, click here.
Information and Demonstrations from OrangeLT eDiscovery experts can help you quickly understand and begin to evaluate the benefit of our services.
To make an information request or schedule a personalized walk-though of the OneO eDiscovery Platform, click here.
The One Decision® Document Review Accelerator leverages advanced near-duplicate identification technology to enhance reviewer efficiency by grouping similar documents and allowing them to be considered together during legal document reviews as part of the eDiscovery process. This grouping and ability to propagate coding decisions throughout all near-duplicate documents results in more efficient and economical document reviews.
To learn more about One Decision®, click here.
OrangeLT's Complete Managed Review Service helps corporate legal departments and their outside counsel conduct document reviews by combining teams of expertly trained legal review attorneys with an integrated, forensically sound eDiscovery platform to allow for efficient, economical and defensible document reviews.
To learn more about managed review, click here.
Unfiltered Orange is a regular newsletter update providing legal professionals with quick reference to unfiltered electronic discovery news, views, and events.
To check out the latest update, click here.
To sign up for our regular updates, click here.
The Intermountain eDiscovery Conference (IEDC) is a one-day legal technology conference designed to provide electronic discovery practitioners with information on the latest tools, techniques, and practices of eDiscovery. Hosted Annually by OrangeLT and presented this year in conjunction with the Mountain West Chapter of the Association of Corporate Counsel, IEDC combines presentations, panels and interactive discussions led by nationally recognized experts to create an intimate and educational professional learning experience.
To learn more about IEDC 2013, click here.
