[Webcast Transcript] Framing Construction Discovery’s Future with AI-Powered Document Review
Editor’s Note: The EDRM’s workshop, Framing Construction Discovery’s Future with AI-Powered Document Review (Tuesday, November 4, 2025), presented practical applications of generative AI for construction disputes. Panelists from HaystackID and outside counsel addressed high-variance data—Primavera schedules, BIM models, Microsoft Teams chats, and complex spreadsheets—and outlined defensible workflows centered on closed-system LLMs, governance, auditability, and human oversight. Key applications included Early Case Insight, chat summarization, spreadsheet risk mapping, image-based issue spotting, and scaled privilege-log drafting. The narrative overview below distills primary takeaways; the complete transcript follows for reference.
Expert Panelists
+ Esther Birnbaum
EVP, Legal Data Intelligence, HaystackID
+ Mark MacDonald, CEDS
VP, Enterprise and Strategic Accounts, HaystackID
+ Sam Morgan
Director of Global Legal Solutions, HaystackID
+ Aviva E.A. Zierler
eDiscovery Counsel, Goldberg Segalla
+ David Timm
Associate, Burr & Forman
+ Mary Mack
CEO, Chief Legal Technologist, EDRM
[Webcast Overview] Framing Construction Discovery’s Future with AI-Powered Document Review
By HaystackID Staff
On Tuesday, November 4, 2025, the EDRM workshop opened with a clear challenge: construction disputes generate data that resist routine review. Primavera schedules do not read like email. BIM models bury context inside layers of embedded detail. Microsoft Teams conversations fragment decisions across channels. Spreadsheets conceal logic behind links and formulas. The session set out to show how generative AI can organize this material quickly and defensibly—without losing sight of security, validation, or human judgment.
Mary Mack, EDRM’s CEO and Chief Legal Technologist, framed the objective: accelerate understanding while maintaining rigor. Moderator Mark MacDonald, Vice President of Enterprise and Strategic Accounts at HaystackID, then guided an expert group—Esther Birnbaum and Sam Morgan of HaystackID, Aviva E.A. Zierler of Goldberg Segalla, and David Timm of Burr & Forman—through a structured progression from policy to practice.
The first movement established guardrails. Timm outlined a practical baseline for legal teams: rely on approved closed-system LLMs, require training, obtain informed client consent, and keep human review in the loop. With those controls in place, Birnbaum and Morgan shifted the focus to outcomes. The discussion contrasted classic Early Case Assessment with Early Case Insight, where AI classifies, explains, and connects content before large-scale review. That change in sequence—insight first—supports stronger ESI negotiations, sharper witness preparation, and more efficient downstream workflows.
From there, the panel moved into construction-specific tasks. Fragmented Teams threads can be summarized to reveal decision trails and key participants. Spreadsheets can be mapped to expose linked formulas and potential risk. BIM and schedule materials can be grouped and connected to communications, helping case teams tie slippage to source evidence. Photo sets can be clustered by issue and time, with relevant images flagged for structural or safety indicators. In each example, AI concentrates attention on the right materials sooner; legal and subject-matter experts provide the final judgment.
Defensibility remained the throughline. Zierler emphasized auditability—prompt histories, rationale capture, and targeted sampling—to document method and decisions. Rather than prescribing technology in ESI protocols, the panel recommended retaining standard confidentiality and third-party provisions while prohibiting use of open systems that retrain on matter data. This approach protects flexibility for appropriate tools and preserves negotiated expectations around security.
The conversation closed on adoption steps that teams can execute now: begin with Early Case Insight to set strategy; apply AI to the heaviest data frictions (chats, spreadsheets, images, schedules/BIM); combine AI-assisted calls with active learning and expert review; and instrument validation from the outset. With that sequence, the workshop offered not just encouragement, but a usable path to faster, defensible results in construction discovery.
Watch the recording or read the transcript below to learn more.
Transcript
Mary Mack:
Thank you for joining today’s EDRM workshop call, Framing Construction Discovery’s Future with AI-Powered Document Review. It is sponsored by our trusted partner HaystackID. I’m Mary Mack, CEO and chief legal Technologist for EDRM, the Electronic Discovery Reference Model.
And we’re talking about construction today. Construction disputes generate data types that strain traditional review. We’ve got the Primavera schedules, the BIM models with embedded project data, dispersed Teams chats, expansive spreadsheets with linked formulas, you name it. Today’s discussion focuses on accelerating review for these kinds of sources with GenAI, while maintaining defensibility, security, and rigorous quality control.
Today’s expert panel is led and moderated by Mark MacDonald, who’s the VP of Enterprise and Strategic Accounts at HaystackID. And the panel includes Esther Birnbaum, she’s the Executive Vice President, Legal Data Intelligence for HaystackID. Sam Morgan, Director, Global Legal Solutions, also for HaystackID. Aviva Zierler, eDiscovery Council, Goldberg Segalla. And David Timm, an Associate at Burr & Forman.
We’re recording today’s webinar for your future on-demand access and slides will be available after the webinar, and this webcast will be available on the EDRM global webinar channel for the next quarter, to help support your learning and reference needs.
Sam, you are up. Please take it from here.
Sam Morgan:
Sure. Thanks, Mary. So I’m actually very excited to introduce the HaystackID members of the panel here. First, we have Esther Birnbaum. Esther’s a thought leader in generative AI, both in procurement and deployment, and execution of GenAI solutions. Prior to coming to HaystackID, Esther lived in the intersection between technology and litigation, and has vetted practically every GenAI platform on the market, and has done a great job here at HaystackID developing our offerings in generative AI.
Then we also have Mark MacDonald, who is a construction litigation pro. Mark has up to 20 years of experience working closely with construction corporations, as well as their outside counsel, in developing cost-effective solutions that streamline processes and get through data. Right now, we’re at a point where generative AI is really what we’re leveraging to help our clients get through large datasets. Mark is an expert on that.
And then you have me, I co-lead the construction litigation vertical at HaystackID with Mark. Also, I work with a number of other diverse corporations as well as law firm counsel. Always have an eye on getting through large datasets; now we have varying types of data. Just coming up with interactive solutions and progressive solutions that help folks save the bottom line.
With that, I’ll turn it over to Mark to introduce our other panelists.
Mark MacDonald:
Thank you, Sam. We’ll start with Aviva. Aviva is eDiscovery Counsel at Goldberg Segalla. She’s an experienced construction and design defect litigator, representing owners, contractors, insurance, and complex matters. Aviva’s also leading the charge at the firm with eDiscovery policy, what’s new, building their teams, and supporting their clients.
And David Timm, associate at Burr and Forman. David plays many roles at the firm, including being on the firm’s AI committee. He’s a leading construction litigator. He advises clients and his firm on AI policies, governance, and really the defensible use of these technologies.
So we are super excited to have these two with us, as well as Esther. As you might have guessed already, construction is our jam, and these are our people, and we are super excited for this presentation. So in the first half, we’re going to cover really what generative AI is for eDiscovery, why it matters, the transformation, we’re getting the challenges of data management, and then policy and ethics, which David will run that slide.
And in the second half we’re going to talk about generative AI in the discovery lifecycle, where we’re using it, where it fits, how to dip your toes in the water, and what we’re seeing in the market as the adoption, which has been lightning fast over the past 18 months, is affecting corporations, law firms and service providers like us. So let’s start.
Esther, this is really a group discussion on our first few slides here, but let’s talk about generative AI for eDiscovery and what it really means.
Esther Birnbaum:
So, GenAI for eDiscovery, we know that with tools like ChatGPT, we understand that really their best capability is to review and analyze documents, and that’s really what we do in eDiscovery at its core is review, analyze, and classify documents. So using LLMs, using large language models, and GenAI to assist us doing that is really the core of what we’re doing.
Aviva Zierler:
First of all, I just want to say thank you so much for having me. I’m delighted to be on this panel at a super important time, and I feel like the technology is really in flux all of a sudden. As Mark said, everybody is talking about GenAI, but a conversation I’ve had a lot is there are a lot of concerns with GenAI, but I want to just talk about… Maybe Esther can walk us through the different roles for GenAI and eDiscovery in construction litigation. It’s early case insight for me is where it is. We have so much data; construction litigation really is a rich opportunity for those tools. And later, I think we can talk about some of those barriers to implementation.
But Esther, if you don’t mind just talking about the different roles of GenAI through the life cycle of construction disputes, if that’s all right?
Esther Birnbaum:
Yeah, I think what we’re going to be addressing is how we fit GenAI into the different cycles. There’s been a lot of focus on specifically how GenAI can really change document review, but what we’ve really been seeing is that GenAI and eDiscovery have a much broader application than just for document review. So we’ll really talk about how it fits into each piece of the process, from starting with early case assessment, early case insight. So we’ll go through all of that.
Sam Morgan:
Yep, no, for sure. And David, do you want to just talk about some of the other practical applications, just from a large GenAI LLM model, and then we can distill down from there as well?
David Timm:
Yeah, so I like to talk about the limitations of the technology first and foremost, just because I had a presentation recently where I did a poll, and out of the 100 or so people, I think 50% had used only ChatGPT, 10% had never used a large language model. And so, one of the things that I focus on when I talk about these language models is just basically how they work, and I think that often gets swept to the side when we’re looking at demos and we’re talking about the capabilities and the efficiencies in abstract terms.
So it’s a helpful thing to understand that the technology basically started a good chunk of time ago, but in 2017, there was a breakthrough with transformers, which essentially enabled really parsing lots of contextualized token representations of data in the training models, and then it allowed it to predict, and that’s what most of these tools are. It’s like a contextualization of the data that you’re submitting and then a prediction about the next likeliest word based on all of that data, including the training data. And then also I think in our discovery situation, the context of the specific data that you are wanting to analyze. Those are two distinct aspects.
And so, when I talk about this I usually tell people, you remember when you were taking a Scantron test back when you were in school, and you weren’t penalized for guessing on the answers, because if you didn’t know the answer and you guessed C, which was my favorite letter, you had a one in four chance of being right. And I do think it’s important to remind ourselves that when we’re talking about the training of these systems, the AI, the language models, are incentivized to guess if they don’t know the answer, and they don’t have an internal mental model that allows them to be truth-seeking as opposed to likelihood-seeking.
And so, this is one of the key limitations that leads to things that we’ll talk about in more depth, which I think is generally called hallucinations, which are essentially fabricated or incorrect factual statements, but they often look very plausible. So I think that’s a pretty good intro for the limitation side.
Esther Birnbaum:
Well, I’m always going to take the counterpoint where our discovery tools are built with specific guardrails in place to ensure that hallucinations are not the result. We build those directly into prompts. We don’t force answers on something like ChatGPT. And in general, the discovery tools that I focus on, and that are generally offered in the market, really, the idea of training the model does not come into play, because tools are built on top of models, and it’s always going to go back to what your results are and if you can validate them.
And so, while we can talk about the complexities of and the shortcomings of LLMs, what’s really important is to focus on what the results are and how we make sure those results are defensible and we’re not dealing with things like hallucinations.
Mark MacDonald:
So for the people watching, as we led up to this presentation, we had several different prep conversations, and each one of them did this, where our panelists… We basically had four webinars that led up to this webinar. So I think those points that you both made are super important.
Adam, let’s move to the next slide, Why it Really Matters, understanding the importance of AI in eDiscovery. And again, this is a topic for all the panelists, but Esther, we’ll start off with you.
Esther Birnbaum:
So why it matters? I think it matters because we’re living in 2025, almost 2026. Clients expect it. And this time last year, I was still in-house counsel, and there was a mandate across the company to integrate GenAI into every single division, every single department, because from a client-facing perspective, you’re not going to be able to compete if you’re not integrating GenAI.
But then, if you bring it down to discovery, it’s the same way. It matters because it’s better in a lot of ways if done right. It’s more accurate, it’s more cost-efficient. And listen, we all go back to cost is king, right? Everyone wants to save money, and now it’s just part of our daily life. So a lot of what I talk about is that as lawyers, we have to get out of our own way in a lot of it, because we’re so used to overanalyzing and putting in rules, and we forget to take a step back and look at what we’re actually doing, what we’re actually using, and the efficiencies it’s creating. As long as you have that validation and defensibility aspect, it’s really almost a no-brainer at this point.
I think I would even take it a step further and say it’s our professional duty to be using the tools that exist and not fighting against that, but understanding that there is a risk, understanding what that risk actually is, to educate yourself, and then deploy them properly.
Mark MacDonald:
Yeah, thank you. Any input here from the other panelists?
Aviva Zierler:
Sure. I can jump in. I think… Oh, sorry, Tim. Yeah, I want us to go back again to the point I started to make, which is about when we would leverage something like GenAI in a review. So I think I would agree with Esther that the clients and attorneys are chomping at the bit, really excited to get their hands on… Clients are expecting that their attorneys are going to get their hands on these tools and use them.
I totally agree that clients would love to save time having humans review everything, especially first pass review, that sort of thing. I think that, again, sorry, getting back to the point where I started, I think I feel very comfortable at this point using GenAI to gain early insight into the data. A wise woman recently who said, “I think if you own the data, you have the advantage in the case.” That’s the second you can get from a huge volume of… Well, let’s call it even responsive data. It might be in a construction case, you’re collecting a ton of data, and you probably need to, but there’s the liability question, and then there’s the damages question.
So it will draw on a huge set of data, but on any particular issue, you may have tons of huge volumes of data that might be responsive in the case and required to be produced eventually, but may be irrelevant to the particular issue that you’re analyzing. So the question is, can you use GenAI to leverage, I guess, take the data and then figure out where you stand on a particular issue early in the case? And then there’s the question of are you going to use it actually to review? And then getting, we’ll talk about later, the defensibility. We’re going to assume, let’s say that the prompts that you use to get to some sort of answer response through a GenAI tool is let’s just assume that it’s work product.
When you get to something like the defensibility for a review question, I think it’s a really interesting conversation we can talk about a little bit later, but is that work product, are you going to end up negotiating prompts and things like that? So I would agree that it’s very exciting that clients are super excited about it, expect us to use it. We have a duty of competence to understand the technology that we’re using as well. But there are a lot of… Just from the defensibility perspective, you have a lot of questions of when it’s appropriate to use it. Just to flesh it out a little bit more, if it’s expensive, let’s just say anywhere at the moment. I think we’re in a moment of flux, but anywhere from five cents to 50 cents a document. When is the right time to leverage GenAI?
Sam Morgan:
No, what you’re saying is extremely important, and what we’re really talking about is the transformation of thought process and even capabilities around GenAI solutions. And then David, I know leading the AI committee at your firm, or co-leading that, and you come across these questions or challenges very often, so I know you have to take somewhat of a tepid approach from… You can maybe talk to us about where you were and where you guys are going now, and some of the things you might’ve found instructional along the way as well.
David Timm:
Yeah, thanks, Sam. I think that I just want to echo a point that Aviva was making earlier and say that a lot of this is about your risk tolerance. And you’re going to hear a lot from Esther, who’s extremely knowledgeable about, especially in particular, HaystackID’s capabilities, and I think it’s important to draw a distinction between its probably three different types of systems. One is large language models in general, and a lot of those are freely available, or there’s some sort of relatively low fee for you to subscribe, whether it’s $20 for ChatGPT, or you have the enterprise version, and it’s more expensive. Or if you have Copilot Enterprise. And then with some of these legal-specific large language models, like CoCounsel and Harvey, and then we’re also talking about discovery large language models like the capabilities that HaystackID is offering. And so, it’s really difficult to generalize about the capabilities when there are different layers of validation.
I think that’s one of the reasons I’m here to talk on this panel, because I think HaystackID is doing a good job leading the way in making sure that some of the fundamental risks and limitations of the underlying technology are mitigated to the extent that they can be by verification. So that’s really key.
And a lot of that comes down to your personal risk tolerance. Like Aviva said, if you’re comfortable with using GenAI in discovery on your initial case insights, compared to a relevance review that you’re going to have to defend to the court and to opposing counsel, I think those are very different situations. And so, you need to use your judgment about when it’s appropriate, and I think it’s very helpful, but just to understand the underlying structures of the technology so that you’re not surprised when there’s a problem and you didn’t think to catch it via a verification process.
Sam Morgan:
Yeah, no, absolutely.
Esther Birnbaum:
Apologies, my camera decided to not work.
Sam Morgan:
Oh, no worries.
Esther Birnbaum:
But I promise I’m good with technology. I think the key thing for me is we can talk about all the positives, all the negatives, all the benefits, all the risk, but when we really get down to talking about using GenAI for document review, I think that you have to do it to understand what we’re talking about. Because once you see the results that you get, and then once you validate the results that you get and you see that validation, and you can understand the process and see how it’s applied in a real scenario. For me, when I did the first GenAI document review, I immediately said, “I never want to do a human first-level review again, because these results are so fantastic.” And I was in-house counsel, and I was that end client.
And the more people who are using it, there’s more understanding of how… We’re talking about making it defensible, et cetera. But I’m at the point where I feel like the argument is that it’s more defensible, so it should be the standard. And I know that I’m ahead of where we are as a market, but I really think it’s important, and what we’re seeing now is such an integration in the market. We know Relativity made an announcement about integrating aiR with RelOne, and just like other platforms are so readily available that I really, as much as it’s really important to talk about all of this, I really, at the end of the day, encourage everyone to give it a try. I think that really will shift your perspective on everything that we’re talking about.
Sam Morgan:
Yeah, and one of the things… Go on, I’m sorry.
Aviva Zierler:
Sorry, go ahead, Sam.
Sam Morgan:
No, I was going to say one of the things, just taking almost a half step back, because we’re all talking around it, but the why, for instance. Why is GenAI so attractive, especially in a construction vertical? Is it that it’s all in our faces because of the huge data sets, and the cost surrounding those? How do we… folks like Mark and I, or even Esther, we’re really focused on getting people to their end goal in the most cost-effective way.
Dissecting or chopping it up and using GenAI is a very good and reliable way of doing so. So I think that’s why, if you guys want to talk about that for a sec.
Aviva Zierler:
Sure. I can talk about that. I think the why is easy, in a sense. I think you have something like errors and omissions claims, or a delay claim, and you’re looking closely at shop drawings plans, you’re looking at payment applications, schedules, all of these documents that look the same from the outside, to be frank about it. It’s really hard… If there was technology, and there has been, thank God over the years, to sift through this kind of… Attorneys are not subject matter experts, and they’re looking at tons and tons of very technical data and, frankly, a lot of them are form documents or look similar from the outside, so you really have to drill in on a very particular issue in construction litigation. And you have multiple, not copies, but versions of almost the same types of documents.
So the idea of using tools to cluster your documents and to make sense of the huge volume of data that’s coming in. And part of that is making sure that even your third-party subpoena responses come in with metadata and things like that, so you actually can filter. But the why is easy. You have a huge volume of data, and it would be amazing, it is amazing, when we have tools to help us sift through that. And it’s actually a great moment for me to say I always admired the name HaystackID, because that’s exactly it. It’s finding the needles in the HaystackID.
I got into this in 2022, 2023, when we didn’t have any real GenAI except for maybe TAR and predictive coding to take a complex construction dispute that had just a baseline, like 750,000 documents, and out of all those documents, maybe 300 really were meaningful in the case and belonged on a timeline, and that sort of thing. And the tools to help us sift through to what is actionable evidence are critical, and we’re excited about it.
I want to just talk about why I accept, and I pretty much accept Esther and the industry’s suggestion that if you can play with it, you can see that the technology really is there, and I believe it. But to Tim’s point, all of these defensibility concerns and risks, risk tolerance issues are so real. Initially, for me, it was things like confidentiality concerns. At this point, I accept after grilling vendor after vendor, that we’re not training a large language model outside of your matter. You’re not exposing client documents or documents to the larger world. They’re staying within the matter.
So I think at this point, I can accept that the confidentiality concerns are met. But then, so let’s say you accept that the technology is there. There are huge barriers to implementing these tools in a case. So the idea of just playing around with it, and we’ve all seen demos with Enron data, things like that, that is not enough to get below a surface level of, hey, this works, and trusting what the experts are telling us. I think that to be able to really play with the data in a sandbox environment, I mean, real data, is very difficult. Because your client’s case obviously is… The importance of their documents in the case cannot be understated, where you really have one chance to choose a tool and to apply a tool.
There’s cost, there’s time, there’s effort, there’s the difficulty of coordination and the cost of moving data. You never want to do that, so you want to make one choice, it’s going to be the right choice for your client for the issue that you’re talking about. And you don’t really have… It’s a shame that you, I guess, at least I haven’t figured out how to really play with the tools in a way that’s convincing to me, that makes me feel assured that my defensibility concerns are assuaged, that I’m using it at the right time in a case, all of this kind of stuff. I wish that I had more opportunity to play around with the technology in a way that I really felt assured that this is worth the risk and I can lead the client there and the company there, so Sam.
Mark MacDonald:
You know, Aviva, I think that there are probably a lot of people on this webinar or who will visit this webinar and listen to it at 2X at some point in the future who share your feelings. And our message to our clients is that this is vetted, this is ready, this is better, faster, and dare we say cheaper, than anything we’ve seen in the recent development of technology across our industry. And it’s a super exciting time for us.
And rest assured, you’re going to have lots of opportunities to play with the technology and see it. And I think it’s probably a good transition for us to go, Adam, to our next slide, number 12, to talk about really what the lead-up has been so far in this discussion. Where do we use it? Where does it fit in the life cycle? What different types of AI fit across the EDRM?
So if the panelists are good with it, let’s jump to slide 13 and really start to talk about where the rubber hits the road. Esther, we’ll start off again with you here, and then have David and Aviva chime in. But I think probably the audience would love to hear about where we are seeing it across our clients, specifically in construction, where they’re just massive data sizes, like Aviva noted. So it’s in your hands.
Esther Birnbaum:
Yeah, I’ll take it away. I like to… I’ve started to bifurcate our applications of GenAI into where we should be in an ideal world and where we are now. And I think that in that ideal world, we are going to be using GenAI for collection processes, for information governance, to make sure all corporate data is properly classified and stored correctly so that it’s searchable and auditable and we don’t have privacy issues, and everything is neatly packaged when it comes to something going wrong and there’s a discovery challenge. Because in an ideal world, discovery happens when something goes wrong, but it really shouldn’t be the point where a company steps back and says, “Hey, where’s my data? How is it stored? Is it searchable?” And especially in companies that have data retention and disposition requirements.
And GenAI, I think, in the future is really going to change the capabilities when it comes to information governance and the collection process. But that’s kind of aspirational thinking. Where we are right now, I think, and it’s been emphasized already, is in the early case assessment process. And a lot of the buzzwords you’re hearing now are early case insight. Because when I think ECA, I think of raw data, applied date filters, maybe some broad search terms, limit by custodians, promote it to review, and then figure it out from there. But when it comes to ECI, early case insights, the idea is that now you are able to take voluminous data sets, and really understand what’s in them in a way that we were never capable of doing before. Because we’re able to employ GenAI in a way that gives us classifications, explanations, summaries, timelines, identifies key people, and even the ability to query it with natural language, which we’ve never been able to do before.
So, ECA, we’re seeing really the biggest impact, because you’re just able to do and understand so much more at the beginning of a case that really helps you be a better lawyer. It helps you negotiate your ESI protocol, it helps you prep for witness interviews even before a document review, and it saves costs on downstream review. And on the same token… Oh, sorry, go ahead.
Mark MacDonald:
Sorry, Esther, I was going to jump in for just one second, if you don’t mind. For anybody who doesn’t fully comprehend what an early case assessment database is, this is an industry maneuver that we do when we have huge data sets. We take just the text, not the native files, so we’re not bloating the size of the database, and we’re just keyword searching and trying to figure out what’s in this big blob of data that we’ve now collected, filtered, and reduced to the point we can before we start to put the attorneys on it.
And this is one of those key places where now Esther is talking about early case assessment using AI, where we can in fact take this big unknown universe of documents, and before anybody puts any eyes on any native documents, emails, whatever those might be, being able to really classify it, categorize it, do timeline analysis, keyword analysis, it really does start the review team, the case team, in-house counsel, outside counsel at the 50-yard line instead of on the five-yard line.
And in my experience, this has been the hottest point of adoption so far across our client base, but especially in construction. So sorry, Esther, I just wanted to put a pin in that point for you.
Esther Birnbaum:
Yeah, and one of the key pieces is that what happens in ECA is often pre-disclosure, right? It’s when you’re at the really early stages of a case, it’s not like review, where you have to have decisions that are agreed upon by the courts, by opposing counsel, et cetera. So the same applications we’re seeing are for things like received production review, very often, especially in third-party productions, you get, again, enormous amounts of data that you have to sift through, and received productions is an area where no one has ever spent a lot of money, or nobody wants to spend any money to go through the other side’s data, and you don’t have to disclose anything when it comes to that, so using it on those data sets.
And also in the same token, internal investigations. One of the first GenAI projects I ever did was on a 2.2 million-document internal investigation, and it surfaced everything I needed within a couple of days. So the capability’s there, which are outside of the standard review and privilege review, relevance review that we talk about. But going into the privilege and relevance review, we’ve seen enormous value with GenAI on both sides. And whether it’s a relevance review, a relevance and issues review, a hybrid workflow with GenAI and active learning for relevance, and then priv log generation is really a no-brainer.
We did it on a 250,000-document matter for a second request. We did a generated priv log in a week, and the feedback we got was that the results far surpassed what a human could do for privileged descriptions. And we were able to customize it, build in all the guardrails, et cetera, and we were incredibly pleased with the results.
And then… Sorry, and I’ll quickly finish up. Also, just like the ability to use GenAI with audio review and image review, they’re really, really strong capabilities, as well as for discovery, like PII identification and auto-redaction, which is a huge use case that we’re seeing. So really across the discovery life cycle.
Mark MacDonald:
Yeah. Got a question for David from the audience. So are the prompts used in ECA, are those discoverable? So it’s a great question. I’ve heard this lots of times. So, David, let you answer that one, and maybe Esther and Aviva can fill in some gaps.
David Timm:
Yeah. In some of our prep calls, we discussed some of the issues related to privacy and confidentiality. I think I just want to take a quick step back before I answer the question.
One of the things we’re spending a lot of time talking about is HaystackID’s capabilities, which I think is most relevant for this discussion. I want to contrast what HaystackID has built and what other closed system large language models are, compared to others, where, let’s say, you use ChatGPT and you have it opt out of training, where you use the free model. And if you’re having a conversation with the large language model, that would be, in my view, textbook disclosure to a third party, which would breach attorney-client privilege. So it’s really important that you’re using a closed system large language model, and there are lots of them out there, and obviously, they serve a variety of different purposes.
When we’re talking about HaystackIDs and other discovery vendors, which I’m generally assuming are closed systems, like HaystackID’s, there isn’t the same concern about the discoverability of the prompts. Now I want to distinguish between different use cases. So, for instance, if you are using GenAI in the review for early case insights, and an attorney is involved, you are going to generally be covered by either attorney-client privilege or work product privilege in the analysis of your own documents. So you’ll be safe in that respect for sure with a closed system.
Now, I want to contrast that with where you are working on, for instance, a relevance production to the other side, it may very well be required that you come to an agreement with the other side about the sorts of prompts that you’re going to use, if they’re even going to accept use of prompts and GenAI on a relevance review. And so, they’re just very different things, and we’re talking about a lot of different capabilities here, and I think it’s really important to always burrow down as deeply as we can on exactly the point that we’re discussing.
One of the things that I’ve done in my firm is help draft our AI policy. And so, there are basically four main principles that we operate by. One is that the firm, you can only use firm-approved, closed system, large language models, so that’s critical for us. The second is that every attorney who uses any of those approved firm tools has to undergo mandatory training that’s refreshed on a regular basis when the capabilities evolve. And I think that’s also critical, because if you’ve only used ChatGPT and you don’t use it in a professional context, you may not understand the sort of risks that are attended to your use.
Third, we talk about the requirement of informed consent from clients. So to the extent that you are using a legal-specific or discovery-related large language model, it’s really important to make sure that your client understands that, that they know the risks, and that they’re willing to accept them via informed consent.
And then finally, the last principle that we have is just generally to have, I think lots of people call it a human in the loop, but what we say is typically that there needs to be some sort of review and approval of all language model output for these firm-approved tools. So I think that’s generally a good framework if you’re thinking about usage, that those four principles should be in there in some way, shape, or form.
Sam Morgan:
No, that’s a great point.
Mark MacDonald:
I think that’s excellent advice, David. Thank you for sharing that. And let me just state that this is not meant to be a HaystackID sales pitch. We’re really talking about technology that is widely available across the industry, and many vendors, just like us, who are global legal discovery and legal process outsourcing providers, are leveraging and tapping into. And I like to use the analogy that anybody can buy a Formula 1 Ferrari if you’ve got enough money, that does not make you Mario Andretti. So ask the right questions, just like David said, to your providers, to your counsels, and make sure we’re on the same page as you start to dip your toes into these waters.
But David, that was super valuable input from you. Aviva, anything to add, or Esther?
Aviva Zierler:
Well, I just want to say that first of all, I agree with everything David just said on the question of… I guess I also agree that prompts are work product, especially at the early case insight stage, early case assessment stage, which was the question. And I would also say that our firm’s evolving AI policy is also very similar to those and has essentially those four principles, so it’s nice to hear that all developed separately, they landed in a similar place, so that’s a good test for where people are thinking, I think.
No, I just wanted to, for David, maybe to also just bring up the… I thought we had a really interesting conversation just offline about ESI protocols, and this might be just an interesting point for those listening. I guess, as I think about ESI protocols, you think about, do you want to include GenAI? There’s been an understanding, there’s the question, do you want to include search terms? Do you not? Do you want to bind yourself? Do you want to bind the other side? There’s no one-size-fits-all. It depends on what position you’re taking, obviously.
And then the inclusion of TAR over the years is a lot of people would say, yes, include TAR, because if you’re called to task, saying, “How did you arrive at this set of documents that are responsive for production?” You would say, “Well, I was allowed to use TAR, right here.” But then the question is, do you talk about GenAI? So, I was coming to it thinking, well, I want to make sure that the other side is certainly not putting our documents into anything but minimally a closed system within an eDiscovery platform, or something like that.
But I think Esther had a great point, and David, you too. I landed after all this, maybe we just rely on the standard professional rules of our obligations to protect from inadvertent disclosure, confidentiality, all of our duties are… And let the professional duties guide us at this point before we really lock ourselves in. So, just, if you don’t mind, I wanted to throw that out either to David or to Esther to comment on that. I thought it was a really interesting conversation.
Esther Birnbaum:
Yeah, I think that, first of all, putting into an ESI protocol that your data can’t be used for purposes to train any models or for anything. The data protection should be in all of your ESI protocols regardless. And I think that’s a separate question, then, should we be including GenAI standards, requirements, or protocols for the review of data for production? And I think a lot of what we’re seeing in discussions about this is the idea that what we’ve done with including TAR protocols in ESI protocols has really hindered the use of TAR in the industry. There is really no reason that every single review that’s done in eDiscovery should not at least have some element of prioritization. But because we’ve put all of these rules and made it almost adversarial to use TAR and to use AI, it really held back adoption. And we should not do that with GenAI.
We should make sure that we’re allowing ourselves to use the best technology to respond to what we are required to by law, and not put limitations around that, other than the data security, et cetera, which should be inherent in everything. But not to back us into the corner that we’ve been backed into other ways. And always at the end of the day, it comes down to can you validate it. Can you show your validation and make it defensible? And you know what? At the end of the day, if you screw it up, you can be disbarred, or you can be sanctioned. And we have to start practicing law like adults. Use the best tool, use it safely, and if you don’t, you get in trouble.
So that’s my opinion on it. But again, data security is non-negotiable, always.
David Timm:
Yeah, I love that. Let’s practice law like adults; that’s a good one. You also made maybe an intentional or unintentional pun earlier.
Esther Birnbaum:
Definitely.
David Timm:
Okay. I’m done for the rest of the webinar, but… I think that Aviva alluded to what I agree with. In typical ESI agreements, there’s a confidentiality clause, or a disclosure to third parties clause, which basically says you can’t take our data that you obtain through discovery and publish it or send it to a third party. And I think this goes back to my point earlier about using a closed-loop LLM where the data that you’re uploading is not ultimately going back to another place to be trained, potentially to train another model or a newer model or to revise the model.
And in those cases, the thing that I land on in those ESI agreements is that the standard language is very likely to be protected, enough protection for you. But what you might consider is that because this technology didn’t exist in 2021, lots of people who are reading the agreement may not know that their data on their free use of ChatGPT, Claude, or DeepSeek is a disclosure to a third party in many cases.
And so, it can be useful to just be more specific about what constitutes disclosure to a third party. And I think to that end, I would suggest indicating that use of open systems where the training data is then used later on is not allowed, and that is considered disclosure to a third party. Now, I do think this begs the question, how will you find out? Well, that’s a very difficult one. If somebody abuses and uses an open system, I think it’d be really tough to determine whether they’ve done that. So that is an open question, but I think it again comes back to whether we have educated ourselves on the technology and the platforms that we’re using, and whether the other side understands that and those limitations as well.
Mark MacDonald:
Excellent points. So I think right now we’ve covered pretty much all of the topics. Esther or Adam, maybe we can sit on slide 15 for one minute and just have an overview of the process and validation. And then after that, maybe we can show off a couple of slides from a responsive review and really show some of the benefits of using AI for responsive review, with a couple of the images from recent construction cases.
So I’ll leave it to the group, hit these highlights. Again, remember that a lot of the people are listening to this, and those who will listen to it have the same feelings that our panel has, where we’re cautiously adopting these things and trying to keep the sharp objects away from each other, so that we’re doing this in a very sophisticated, defensible way.
So, Esther, I’ll turn it back to you.
Esther Birnbaum:
Yeah, you said that we’ve covered all topics, but I don’t know about the other panelists. I’m just getting started.
Mark MacDonald:
Well, look, each one of these slides could be its own webinar in itself, so yeah, noted for sure.
Esther Birnbaum:
Yeah. I think that the most important thing when it comes to talking about GenAI in eDiscovery is always bringing it back down to what we’re already doing. Because at the end of the day, David makes a lot of good points about the limitations of LLMs and hallucinations and things like that, but then brings it back to a human review. Because, listen, some people, when you’re hiring contract reviewers, you do an in-depth interview, et cetera, but that’s where the judgment of that person ends, and what really matters is the results.
And I saw on LinkedIn, and this was a while ago, somebody posting about a GenAI review that they did, and they were like, “Wow, it was 90% recall on precision.” Which basically means nine out of 10 of the documents were quoted correctly. Now, in a human review, we’re lucky if in a review you get a 70% accuracy rate, and those are the standards that have been accepted by the court. And on that post in LinkedIn, somebody commented, “Well, I’m really concerned about that one document of the 10 that AI got wrong.” And it kind of just blew my mind, because I was like, “But you’re not concerned about the three documents that the humans got wrong?”
And I understand that we have more concern about what GenAI is doing. And I think that when you think about it rationally, and then you really take the time to educate yourself about the tools, and you can really understand that there’s so much more transparency than there is with a standard review, because you get a classification, but you also get an explanation and an analysis of why coding is coding. And the thing that blew me away, and continues to blow me away, is that when you do document review with GenAI, you achieve a level of consistency that you just don’t when you have 50 humans on a review.
And I know that there’s a counterargument about the variability of LLM results. And to that, I would always say test it, because we tested inter-run variability, we ran the same prompts five or 10 times to see how it would affect the classification. And honestly, it came out negligible. And we have a case study on that, if you want to read that, reach out. But we’ve done all that testing. So I think that the more we educate ourselves, the less fear there will be and the more understanding of the real benefits.
Sam Morgan:
Great point. Aviva? David?
Aviva Zierler:
I wanted to speak about a few points here. I think maybe number two and number five, auditability, and transparent documentation of prompts and decisions. For me, that’s key. Being able to show, I guess, an audit trail for defensibility. If you have to show where you’ve been, how you filtered the data for a meet and confer, or any discussions with opposing counsel, or the need to defend your process. That’s huge, so to document every prompt you’ve used, all of the validation done, that’s key.
I wanted to also comment on three and seven, consistency with human review standards, and then human judgment and contextual interpretation. I totally appreciate how there’s human error and human reviewers’ eyes glaze over, and how you can really get just… I think we’re all convinced of how useful technology can be to give us really consistent, excellent data crunching results. But then, when it comes to something like an errors and omissions claim or something like that, where you don’t want to substitute the human judgment of your experts. So I think just being careful that you’re not relying too much on the comments… The rationale is helpful. I think, again, this is just about using GenAI as a tool; you hear the word augment a lot, a tool to augment the human role.
Obviously, you have experts, and then you also have the human… I guess the legal issues… Sorry about my dog in the background. The legal issues are the same, but the huge volume of data is… To get back to the legal issues is huge. So if we can use GenAI to hone in on the actual issues on the small amount of documentation that really pertains to a particular issue, and then bring in our outside human judgment, again, our experts, our internal subject matter experts at our clients, at the companies, and the case teams. Obviously, a hybrid approach when it comes to something like review of your own documents for production, or also the review of incoming productions is huge. So those are… David.
Mark MacDonald:
And because you said you were looking for a specific topic, we’re going to show an image of a crack in a foundation. Esther, talk a little bit about this one, if you don’t mind.
Esther Birnbaum:
Yeah. So this is a matter where I created a relevance review to detect a couple of issues that would be in a construction litigation. The first issue is simple: identify any images that have anything indicating a structural defect, and I gave some examples. And you can see in the results that it came back as it would be relevant, and it goes into a lot of detail about the crack itself, where it is, and how it would contribute to settling or a structural defect. Just a lot of nuance and detail in the photo analysis.
And the second issue I prompted for was for safety violations, on the next slide. The first one was not relevant to safety violations. And I gave a very similar, very basic prompt asking for safety violations again, and gave a couple of examples. And in this analysis of the relevance classification, the AI cited three specific OSHA rules that indicated three specific violations in the photo. And when we first saw this, we were like, “This is a hallucination. There’s no way this is true.” But spoiler alert, the exact citations are correct. We didn’t mention OSHA or anything like that, but then that opens up a whole possibility of what you can do if you’re citing, if you say, “Look through a corpus of pictures and find any OSHA violations, or find any construction safety violations.”
It just really demonstrates the power of what GenAI can do, and also just the additional benefit with the analysis that you wouldn’t get from a human reviewer.
Mark MacDonald:
I think it’s also helpful for people to understand that when we run AI for responsive doc reviews, we also have the ability to show what the AI output is and mirror that against a coding panel, so it doesn’t have to be a full AI review. We like to use TAR with a combination of AI to get us to a place where those two elements are coming together to help in doc reviews. It’s been really, really interesting to watch the evolution of this in such a compressed timeframe, and there’s more to come.
We’re at time right now. I want to say thank you so much to our panelists. Thank you to EDRM for hosting this. Sam and I are going to the Construction Super Conference down in Bonita Springs, Florida. I hope to see many of you there. This is the preeminent kind of construction conference, so I think about five or 600 attendees.
There are a bunch of comments and questions that are in the chat here that we didn’t get a chance to answer, but every panelist, every participant here, if you signed on, you’ll get a copy of the webinar, and we’ll send a document that has those questions with the answers as well.
So thank you again, everybody. Thank you, David. Thank you, Aviva, and again to the EDRM. We’ll see you next time.
Sam Morgan:
Yes. Thank you, everyone.
Esther Birnbaum:
Thank you so much.
Mark MacDonald:
Thank you.
Mary Mack:
Yes. And we would like to thank all of you for joining today’s EDRM workshop. Thank you to our panelists for a very lively discussion and to HaystackID for sponsorship and expertise.
Before closing, please mark your calendar for HaystackID’s next webcast, How Academic-Centric AI Projects are Driving Legal Tech R&D, on Wednesday, November 19th, 2025. Register at HaystackID.com, and we hope to see you there. Thanks very much.
Sam Morgan:
Thank you.
Mark MacDonald:
Bye-bye.
Expert Panelist Bios
+ Esther Birnbaum
EVP, Legal Data Intelligence, HaystackID
With a robust background in complex litigation and regulatory compliance, Esther brings a wealth of knowledge and practical experience to the table. She uses her unique expertise at the intersection of technology, data, and law to develop best practices and drive innovative workflows across many areas of the business. She enjoys sharing her insights with the wider eDiscovery community and frequently speaks at conferences, webinars, and podcasts on topics related to law and technology.
+ Mark MacDonald, CEDS
VP, Enterprise and Strategic Accounts, HaystackID
Mark is a seasoned eDiscovery professional with over 20 years of experience helping corporate legal departments design and implement effective and defensible ESI management models. He is a certified eDiscovery specialist (CEDS) and a thought leader in the industry, with expertise in legal hold, data breach, digital forensics, advanced analytics, managed review, and eDiscovery playbook development. Mark has managed thousands of matters and served as an expert witness across various industries and types of cases on several occasions. Since January 2023, Mark has been the Vice President of Enterprise and Strategic Accounts at HaystackID, responsible for consulting and supporting clients on best practices and workflows across the entire legal data lifecycle, leveraging HaystackID’s unrivaled blend of customer service, talented professionals, and innovative technologies. Mark’s mission is to ensure that his clients benefit from technology-driven, scalable, and cost-efficient solutions that address their unique ESI-related challenges and goals.
+ Sam Morgan
Director of Global Legal Solutions, HaystackID
Sam Morgan is a Director of Global Legal Solutions at HaystackID. Harnessing over two decades of industry experience, Sam drives immeasurable value for his law firm and enterprise clients. Having worked as a paralegal at leading AMLAW 100 firms, Sam knows firsthand the challenges that in-house teams and eDiscovery practitioners face, and provides clients with tailored strategies and solutions to overcome these obstacles, such as handling ballooning data volumes and a proliferation of data sources. Sam has a proven track record of success for some of HaystackID’s most prominent global corporate and law firm clients.
+ Aviva E.A. Zierler
eDiscovery Counsel, Goldberg Segalla
Aviva E.A. Zierler counsels and defends property and business owners, developers, general contractors, subcontractors, and engineers in a range of legal matters, including negotiating and drafting contracts, and defending claims involving construction defect, performance, delay, and property damage. Aviva has also litigated or resolved matters concerning commercial leases, enforcement of contracts and judgments, employment disputes, and breach of restrictive covenants and confidentiality provisions, effectively securing injunctive relief for her clients. She draws on her recent experience as liaison to outside counsel for a national infrastructure construction company in multi-million-dollar federal litigation arising from engineering and construction services contracts.
+ David Timm
Associate, Burr & Forman
David Timm is a member of the Burr & Forman Construction & Project Development practice group. He represents contractors and companies in complex disputes, claims, and bid protests involving federal, state, and local government contracts.
David’s practice includes matters before the Government Accountability Office (GAO), Boards of Contract Appeals, the U.S. Court of Federal Claims, the U.S. Court of Appeals for the Federal Circuit, various state courts, as well as in arbitration and mediation. He also advises clients on regulatory and compliance matters on the federal, state, and local levels, including those involving the Small Business Administration’s (SBA) rules, including the 8(a) program, HUBZone, Small Business Innovation Research (SBIR) and others.
In addition to procurement-related litigation, David represents clients in civil litigation, internal investigations, and related disputes. His extensive experience with e-discovery adds value for clients navigating complex discovery issues. David also maintains an active U.S. Top Secret security clearance.
+ Mary Mack
CEO, Chief Legal Technologist, EDRM
Mary Mack leads the EDRM, a project-based organization, and is the former Executive Director of a certification organization. Mack is known for her skills in relationship and community building as well as for the depth of her eDiscovery knowledge. Frequently sought out by media for comment on industry issues, and by conference organizers to participate, moderate a panel, lead a workshop or deliver a keynote. Mack is the author of A Process of Illumination: The Practical Guide to Electronic Discovery, considered by many to be the first popular book on eDiscovery. She is the co-editor of the Thomson Reuters West Treatise: eDiscovery for Corporate Counsel. Mack was also recently honored to be included in the book; 100 Fascinating Females Fighting Cyber Crime published by Cyber Ventures in May 2019. Mack has been certified in data forensics and telephony. Mack’s security certifications include the CISSP (Certified Information Systems Security Professional) and the CIAM (Certified Identity and Access Manager).
About EDRM
Empowering the global leaders of e-discovery, the Electronic Discovery Reference Model (EDRM) creates practical global resources to improve e-discovery, privacy, security, and information governance. Since 2005, EDRM has delivered leadership, standards, tools, guides, and test datasets to strengthen best practices throughout the world. EDRM has an international presence in 145 countries, spanning six continents. EDRM provides an innovative support infrastructure for individuals, law firms, corporations, and government organizations seeking to improve the practice and provision of data and legal discovery with 19 active projects.
About HaystackID®
HaystackID solves complex data challenges related to legal, compliance, regulatory, and cyber events. Core offerings include Global Advisory, Data Discovery Intelligence, HaystackID Core® Platform, and AI-enhanced Global Managed Review powered by its proprietary platform, ReviewRight®. Repeatedly recognized as one of the world’s most trusted legal industry providers by prestigious publishers such as Chambers, Gartner, IDC, and Legaltech News, HaystackID implements innovative cyber discovery, enterprise solutions, and legal and compliance offerings to leading companies and legal practices around the world. HaystackID offers highly curated and customized offerings while prioritizing security, privacy, and integrity. For more information about how HaystackID can help solve unique legal enterprise needs, please visit HaystackID.com.
Assisted by GAI and LLM technologies.
Source: HaystackID