Rethinking Document Review: AI’s Role in Solving Long-standing Challenges

Rethinking Document Review: AI’s Role in Solving Long-standing Challenges  

Jim Sullivan thought he had closed the chapter on eDiscovery. After over a decade in the trenches working with predictive coding and analytics tools, he decided to explore new horizons beyond the relentless pace of the legal tech industry.  

“When you leave eDiscovery, the outside world is completely different—you don’t have to answer emails within five minutes, 24 hours a day, or wake up at 3 in the morning to run productions,” Sullivan shared on the Legal Tech StartUp Focus Podcast. 

Sullivan’s nearly four-year hiatus from eDiscovery was filled with building automation solutions for small businesses with his business partner. But like many of us in this industry who have left, those eDiscovery roots pulled him back in.  

A Tale as Old as Time: An eDiscovery Boomerang 

During his time outside our niche field, Sullivan and his team had to review social media posts to identify specific topics of interest, spending a staggering $10,000 a month on manual human reviews. They decided to experiment with AI and quickly saw its potential—it reduced costs to just $12 a month.  

“I realized that this is basically document review so that this same logic could work for eDiscovery,” said Sullivan on the podcast. “This was the biggest eye-opening experience for me; for 10 years, I listened to people’s problems with eDiscovery technology and now saw what AI could do to help solve them.” 

“I told my wife that I was thinking of going back into eDiscovery and would have to be back on my phone 24/7. Her only response was, ‘I don’t know what took you so long,’’ he joked on the podcast.  

From that moment, his phone was never in Airplane mode, and, on a more exciting note, eDiscovery AI was born. As the company’s founder, Sullivan worked closely with a diverse team of AI experts and legal industry veterans who shared his commitment to responsible innovation to build the company.  

TAR V. AI: Same, Same, But Different 

eDiscovery AI empowers legal teams with accessible, dependable, and revolutionary AI-driven tools to make document reviews faster and more accurate. Think that AI-powered document review is the same as TAR 1.0 or TAR 2.0? Think again; there are important differences between these innovations. While TAR 1.0 allows for the classification of massive volumes of data with minimal effort, TAR 2.0 efficiently prioritizes relevant documents. The core difference between TAR and AI-enabled review is the latter does not require time and expertise to train models. 

“In a typical TAR 1.0 case, I tell clients they should expect to train 10,000 documents to train the model (assuming no rolling uploads) sufficiently. However, in almost all instances, we get adequate results with around 5,000 documents, which includes a control set,” Sullivan wrote in his book, The Book on AI Doc Review. “Training 5,000 documents can take over 80 hours for a highly skilled subject matter expert.” 

For many legal teams, dedicating this time and effort is quite costly. 

“This is where AI changes the game,” wrote Sullivan.  

While AI review does benefit from training examples, these examples are not required like they are for TAR. 

“The model already understands natural human language. With AI, all you need to do is provide the instructions to the machine so it knows what to look for. That can be as easy as typing out simple instructions explaining what you want.” Jim explained.  

This shift from TAR to AI represents a fundamental change in how legal professionals can conduct eDiscovery. By eliminating the need for exhaustive training and allowing the machine to understand instructions directly, AI can achieve in hours what used to take weeks. 

When discussing AI-powered document review, Sullivan highlighted the benefits of using large language models (LLMs) to classify documents.  

“The most expensive and time-consuming part of document review is identifying the relevant documents. That’s where AI excels,” he said.

Ensuring Data Security When Working with AI 

As AI continues to reshape the eDiscovery landscape, data security remains a top priority for legal professionals. How AI vendors manage data is crucial, especially in an industry with highly sensitive and confidential information. Christopher Wall, HaystackID’s DPO and Special Counsel for Global Privacy and Forensics, shared that when working with AI vendors, it is crucial to know how the entity hosting or having access to your data is using your data and see if there is an agreement specifying that usage. 

“Make sure that any prompts that contain sensitive or personal information don’t inadvertently expose confidential data or personal data to third parties or compromise the security of individuals or your organization,” he advised during a recent HaystackID webcast. 

This concern isn’t unfounded. Most public AI tools are not built with the stringent data protection standards required in the legal field. As a result, there is a risk that data input into such tools could be exposed or used in ways incompatible with the expectations of confidentiality and privacy. Therefore, legal teams should conduct thorough due diligence on any AI vendor and ensure robust agreements that specify data handling, usage, and protection measures are in place. 

When working with an AI service provider, Sullivan echoed the following advisement.  

“You want to ensure all data is encrypted at rest and in transit, understand their data retention policies, and possibly do some investigation and penetration testing,” he said. “If a service is free, you should assume they are not protecting your data.”

An Effective and Defensible Approach to Document Review 

Recognizing the potential of AI, HaystackID launched HaystackID® Core Intelligence AI™ earlier this year, an advanced eDiscovery solution powered by eDiscovery AI and generative AI (GenAI) technology. This tool helps legal professionals streamline the discovery and review of electronically stored information (ESI), addressing challenges such as increasing data volumes, data security complexities, eDiscovery costs, and information management inefficiencies. Core Intelligence AI™ automates complex tasks like data classification and categorization, reshaping eDiscovery workflows to cut costs and maximize efficiency. It generates concise summaries and actionable insights from large datasets, enabling quicker and better-informed decision-making. 

“We’ve integrated generative AI capabilities into our platform to meet the unique needs of our clients, enabling them to tackle complex eDiscovery challenges with confidence,”said Andrea Wallack, President of HaystackID, in a press release. “Our priority is to provide practical and efficient solutions, ensuring high levels of efficiency, defensibility, and compliance.” 

Sullivan added, “With this technology, legal teams can swiftly, accurately, and securely identify crucial documents to manage their case strategy. By handling the heavy lifting, AI enables teams to work more efficiently, streamline eDiscovery workflows, and focus on making a significant impact on the bottom line.” 

Molding eDiscovery’s Future 

Sullivan’s career in eDiscovery has come full circle—from diving deep into predictive coding and exploring analytics to stepping away to build automation solutions, only to return with a renewed focus on innovation. For Sullivan, the future of eDiscovery is not just about keeping up with change; it’s about driving it.