[Webcast Transcript] The Accelerating Acceptance and Necessity for Remote Collections and Reviews
Editor’s Note: On April 22, 2020, HaystackID shared a comprehensive overview of remote collection and remote review considerations and capabilities as part of its monthly education webcast series. While the full recorded presentation is available for on-demand viewing via the HaystackID website, provided below is a transcript of the presentation as well as a PDF version of the accompanying slides for your review and use.
The Accelerating Acceptance and Necessity for Remote Collections and Reviews
Advances in remote eDiscovery technologies coupled with developments that are restricting the ability of organizations to support traditional onsite eDiscovery operations are accelerating the acceptance of remote collections and remote legal document reviews. However, not all remote-centric collections and document review technologies, processes, and providers are equal, and knowing how to compare different approaches may mean the difference between litigation or investigation success or failure. In this expert presentation, remote collection and review experts will share information, insight, and experiences designed to benefit leaders and organizations as they consider remote eDiscovery in our new remote world.
Webcast Highlights
+ Defining Remote eDiscovery: Definitions, Differences, and Decisions
+ Considering Remote Collections: Targets, Tasks, and Technologies
+ Reviewing Remotely: From Reviewer Selection to Secure Technologies
+ Remote eDiscovery Best Practices: Practical Considerations and Recommendations
Presenting Experts
Michael Sarlo
HaystackID – Partner and Senior EVP
John Wilson
HaystackID – CISO and President of Forensics
Seth Curt Schechtman
HaystackID – Senior Managing Director of Review Services
Presentation Transcript
Introduction
Hello, and I hope you’re having a great week and we wish you health and safety for you and your loved ones during this challenging time. My name is Rob Robinson and on behalf of the entire HaystackID team, I’d like to thank you for attending today’s webcast titled The Accelerating Acceptance and Necessity for Remote Collections and Reviews. Today’s webcast is part of HaystackID’s monthly series of educational presentations conducted on the BrightTALK network and designed to ensure listeners are proactively prepared to achieve their computer forensics, eDiscovery, and legal review objectives during an investigation and litigation. Our expert presenters for today’s webcast include three of the industry’s foremost subject matter experts and authorities on remote eDiscovery.
Our first presenter is Michael Sarlo. Michael is a Partner and Executive Vice President of eDiscovery and Digital Forensics for HaystackID and in this role, Michael facilitates all operations related to electronic discovery, digital forensics, and litigation strategy both in the U.S. and abroad for HaystackID.
Our next presenter is John Wilson, who is Chief Information Security Officer and President of Forensics for HaystackID. John is a certified forensic examiner, licensed private investigator, and information technology veteran with more than two decades of experience working with the U.S. government, and both public and private companies.
Our final expert today is Seth Schechtman. As Senior Managing Director of Review Services for HaystackID, Seth has 15 years of industry and 13 years of big law experience focused on legal reviews supporting multimillion-dollar review projects, including class actions, NDLs, and second requests.
Today’s presentation will be recorded for future viewing and a copy of the presentation materials will be available for all attendees. Also, all of our webcasts are made available on-demand viewing directly from both the HaystackID website and from the BrightTALK network after completion of our live presentations, and at this time, I’d like to turn the mic over to our expert presenters led by Michael Sarlo for their comments and considerations on the acceptance and new necessity for remote eDiscovery. Mike?
Michael Sarlo
Thanks so much, Rob. I appreciate that, for the introduction. This is Mike Sarlo speaking We have a very good presentation for you all today. Thank you for joining us again for our monthly webcast series. Today, we’re going to be breaking down into some key topics that we’ve been dealing with, all of us, conducting remote eDiscovery from your home, but also how we interact with our end clients, be it attorneys or their clients, or how we get evidence in the first place, so today’s agenda is going to focus on remote eDiscovery. We’re going to talk about some definitions, the feasibility of it, and different use cases. We’re going to about how we’re adapting to the remote eDiscovery world, and this is really focused on planning around execution, so we’re going to get into our ReviewRight offering, which HaystackID has been offering remote review for over the past six years with 400 plus projects, where other review shops are now trying to enter into the marketplace. We have a lot of institutional knowledge here, workflows and experience, we’re going to be really happy to share that stuff with you guys, and then we’ll just break down more of our security and give you guys a wrap up on the topics we discussed.
So, the first step is what is remote eDiscovery, and I think we’ve all seen this graphic before if you are a connoisseur of the eDiscovery, blogosphere world, or EDRM as well. It’s pretty standard stuff. This is the updated EDRM framework and certain elements of the EDRM are much easier to accomplish in a remote world than others. Some might even be more efficient is what we find and anywhere where we’re collecting data, physical devices, that can always be challenging, problematic. Movement of data is also an issue. Identifying data and identifying evidence when you’re dealing with physical evidence, knowing what you’re imaging, being able… something you can’t see, that’s important, and there’s a shift in burden there as we look at different types of matters, be it general litigations or more regulatory matters before government agencies. There’s a shift in risk, which is typical.
So, who is a candidate for remote eDiscovery? And just like all things, most organizations don’t have strong information governance. It’s just what we find. Highly developed corporations tend to have better information governance policies than smaller SMBs, although smaller SMBs are sometimes at an advantage depending on when they’ve started, just that they may be more heavily weighted towards the cloud. They don’t have as much legacy infrastructure. It’s up to eDiscovery practitioners to guide organizations, but now we’re really working even closer with IT on day-to-day tasks, and then custodians, a lot of project management there, to accomplish remote eDiscovery tasks while navigating the wave of COVID-19 disruption in a pandemic world. There are travel restrictions for the vendors, for the lawyers, for the end clients.
There are health constraints. We have to worry about are we sending out high-risk employees to do collections on the vendor side or vice versa? How are we getting people in contact with other people, if at all, physically, which we’ve been eschewing to date?
Just the overall logistics and supply chain disruption, this is a major factor that is causing problems with remote eDiscovery, and being able to plan ahead around those items are critical, especially when your shipping might take days longer to move something, or the procurement of equipment for large scale operations may be hampered.
Regional shutdowns and general quarantines, this is obviously… in the U.S., our freedom of movement has limited, as perhaps trying to do remote eDiscovery in the E.U., and if you need to travel, where we’re seeing a lot of border checks and lockdowns and border closures, and that’s something that we expect will continue and develop. Especially in the U.S., we would imagine that we’ll have more sophisticated models that will limit travel as tracking and tracing moves forward.
And then budget constraints and cash flow constraints, this is a really rough time for a lot of organizations, and you throw a litigation or government investigation into the mix, and the chains of debtors and creditors are highly disrupted right now, so there are certainly budget considerations and cashflow considerations that need to be considered as you start building out an eDiscovery plan for any matter.
John Wilson
Yes, and so when you’re speaking about the disruptions and the challenges, a great case example is, in the very early days of COVID, we had our forensic team out at a client site doing collections. They had a custodian who then was suspected of having COVID. We wound up having to quarantine our employees, sterilize equipment. They had to quarantine all the employees at their site, and then there were other third-party vendors that wound up having to get quarantined and those people were then pulled out of circulation or availability for work for two weeks, so those kind of disruptions are certainly challenging and things that you have to build into your timelines and planning when tackling these matters.
Sorry, Mike, just wanted to add that in there.
Michael Sarlo
No, go ahead. Definitely, and please jump in. And, guys, if there’s any questions or comments throughout the presentation, please feel free just to pipe in, we’ll do our best to answer them as they’re relevant live, and also at the end for more robust questions that we think would be best suited to be answered there.
Just getting back to it, so it’s much harder to conduct your remote eDiscovery when you’re dealing with individuals at their houses, these cases where it’s three or four custodians, where information governance, they don’t have these types of policies in their organization, they’re a small company, they’re an executive who is divergent from their general corporation. There’s a litany of reasons why you may need to interact with custodians at their home, and you might need to interact with custodians at their homes for large SMBs as well, but the chain of command in small organizations can oftentimes be a little bit disrupted around legal tasks and execution there, especially as focus for a lot of employees has shifted [not less focused], and obviously when I say interaction, I mostly mean remote interactions, in this case, and we’ll talk more about that as we get more into the presentation, but what we typically find is that organizations with well defined information governance policies and compliance systems are better suited to accomplish eDiscovery objectives in a pandemic world and this is due to the fact that they have a good sense of where the data producers and data sources are located, both in the physical geography standpoint and network and cloud location standpoint; asset tracking; and just having overall alignment between relevant stakeholders across the enterprise, which certainly pivots on functional policies that dictate outcomes that are positive, and a lot of these organizations are much… some organizations are much better suited for remote work. These are also going to be the organizations that are much better suited for remote eDiscovery.
So, how are we adapting to this? There’s been a lot of changes. We’ve made… HaystackID actually had a pandemic plan before the pandemic hit, so we were a little bit, I think, more prepared than some other organizations. Distance and go-to-market on data points are really two of the major considerations that you’re always trying to navigate. Some cases have less flexibility than others, and certainly what we’re seeing is in the pandemic world now there’s really a new proportionality test related to data preservation that has begun to develop, and I think this is something that lit support people are navigating, lawyers are navigating, the regulators are navigating this. We’re seeing different agencies change their policies and procedures for interacting with organizations who are responding to certain agency requests, be it investigations, the second request, various mergers, acquisitions, working with the FDA, everything’s changed there and timing seems to be the biggest… the biggest thing that we see as far as standardized changes is more time. That’s not necessarily always the case, though, and certainly, the big things that hinge on this, I think for an organization or for a litigator or lawyer and for a lit support person that they need to be aware of, is really going to be when you’re talking to your clients, and you’re planning a matter upfront, how are you going to keep data subjects safe. We need to be aware of legal practitioner safety. We have to really think about how feasible is it to even identify evidence in certain scenarios. You have offline storage, you might have devices that you need to go in and pick up for more of a complex antitrust investigation where you can’t rely on employees, perhaps, to hand over and pack up data, as they could be bad actors.
Really, the big thing here is how easily can you transfer the evidence that you collect, and that’s what it is, it’s evidence more than it’s data. The big thing that we are advising clients right now is that COVID-19 does not ease your burden to prevent spoliation. Oftentimes, we’re suggesting the two-phase workflow, be it, let’s say in the case of iPhones where we know that call logs in version 13.4.1 are no longer accessible from the iCloud. A lot of people don’t know this, this is a change that happened about three or four weeks ago, and you would have thought that in a COVID world, being able to collect mobile device backups from an iCloud would be a great solution. Now, we have this huge barrier with call logs that are missing, so partially what we’re advising clients there is go out and get the call logs, make your workflow transparent, and then start to supplement with secondary kits. In this way, you’re still being able to demonstrate rigor and a best effort as it relates to preserving that data, and from there, you’re also able to get closer to a more defensible [inaudible] collection, and having that documented is critical in this scenario, because we’re seeing different case teams that are maybe getting a little too lax, and I don’t think that regulatory agencies are going to be that forgiving once things pick back up, and we expect things to pick back up shortly, although this could be certainly the world that we’re living in for quite some time. We imagine it’s going to be more of a case of rolling shutdowns and certainly changes in the court systems.
John, I don’t know if you have anything you want to add here as well.
John Wilson
Yes, and you pretty much covered it. It’s things to be thinking about the new, as we call it, new proportionality tests. It’s not just the identification, collection, transfer. It’s worrying about the safety of all the individuals involved; if you have to do a physical, or when you’re doing remote, you still have to be worried about does that custodian perhaps have to meet with their IT person to get the system permissions changed, and so you’ve got to be thinking about all of the safety factors, all the people that may have to interact in order to get a task completed, and then also the timing of getting those tasks completed because the timing is certainly impacted by the additional safety measures, the additional bandwidth restrictions. So, somebody uploading or being connected remotely at their home may… they have residential internet and that mileage may vary. If you’re in New York City, you may have fiber and have a gig network in your house through your service provider, but if you’re out in a more rural area, you may be back in DSL, dial-up speeds. There are still areas that have five to 10-megabyte connections, so, you’ve got to really think through and plan around all of those impacting factors.
Michael Sarlo
I couldn’t agree more, and really what it comes down to what we face is just early on planning. It’s so important now where in the past, as much as eDiscovery and our industry, and the way we conduct electronic discovery has evolved. It’s not so much what we call the wild, wild west anymore, but one thing that does still remain is the fact that standard workflow for some corporations and litigators, even from a strategy standpoint, not just because they’re lazy, they don’t want to get into it, but oftentimes, strategy as far as budgeting spend is oftentimes piecemeal on collection and production efforts. Certainly, when we’re advising clients on data collection best practices, we’re always saying go in with the biggest shovel, so you don’t have to go back. Data collection is really usually no more than 15% of a general litigant’s budget. [Inaudible] is really 25% eDiscovery and the remainder is really motion practice and review and productions around eDiscovery, so we always recommend the bigger shovel with digital forensics, but you just need to know, really, the data mapping, which I say is in eDiscovery tales all this time, we always talk about it, but that’s all we do all day. John and I are constantly on these initial intake consulting calls where an organization has come to the head with an eDiscovery burden or a preservation burden, and we’re navigating a complex digital environment, but now also a host of other variables that you need to be aware of to make sure that you’re doing the best service to your client, not getting them in trouble, and certainly, just the safety of everybody. So, we’re sterilizing; in our remote kits, we’re sterilizing everything. We’re sterilizing them as they come back into our labs, and even things like maybe not putting a return label in the box because you want to be ahead of any type of local shelter-in-place orders, things like that that we found ourselves as a country with rolling into shelter-in-place, constantly having to mitigate that, and certainly dealing with appetites of certain custodians to do their part, to participate in the discovery process.
So, a lot of planning and it’s really going to be more towards planning for the worst-case scenario, which is really expanding your list of potential custodians, expanding your list of potential data lake repositories, and expanding your list of potential devices, and because of the time delay, heavily advising and documentary… really, heavily advising and documenting on just ways to avoid spoliation when there’s a time delay from knowing you have critical information that needs to be collected and then there’s more of a resource or another barrier that prevents data on my collection. You really need to document and need to prioritize around data that can easily die on the vine.
So, certainly, you need to be sensitive to social distancing norms. We actually have… in some regions, we have some mobile digital forensic labs that are basically a souped-up RVs with full digital forensics workstation and collection capabilities there for on-site collections, where required. We’re still traveling in certain situations. We also have really developed a good understanding of the geographical reach of each of our examiners and have onboarded and added new examiners regionally across the country and the world in order to accomplish data collection tasks, but most of this stuff can be accomplished remotely. It always has been able to be accomplished remotely, and it’s always something that we’ve pushed clients to do, so it’s not all that bad. We just need to plan a lot more upfront, and obviously, just following CDC guidelines regarding just the way that you’re developing discovery plans not to have egg on your face, or to get a custodian sick or anybody interacting with devices that are in scope for data collection.
John Wilson
Yes, and so a big part of that that we talk a lot about is 80% of the work can possibly have some remote solution, and then there are still circumstances where you do have to go in person, and then in those scenarios, we’re doing a lot of things with policy and procedures to ensure that we’re doing it at a social distance. So, OK, you’re going to leave your computer on the right side of the desk, and then you’ll exit the office, the examiner will then come into your office, grab the computer off the right side of your desk, and during that process, they’re going to sterilize the computer or any media that they have to handle, they’re going to wear gloves, they’re going to follow all of the CDC guidelines, safety guidelines, and so those are the kind of things to consider in the process. Again, those all add to the timing of the work. It slows things down as you’re adding additional steps or having to wait for the person to leave the room to then go into the room, and all those kinds of things, so just understanding that there’s a significant impact through even just the social distancing in the event, in those rare instances where you do actually have to do something in person.
Michael Sarlo
And certainly, when we think about remote when really bridging the gap between our remote capabilities and expertise, as we enter into the enterprise and become embedded, that’s the best way oftentimes to accomplish more large-scale discovery objectives. This is going to tie back a bit into just some of why certain organizations are better suited for this type of integration, or some matters necessitate it, and we know a lot of clients are going to try to breeze through here, because it’s definitely a dense presentation, and we want to make sure we cover everything but again, feel free to ask any questions.
So, really, strategy for setting up quickly. At the outset of the matter, even at the contracting stage, as soon as I get an email from somebody, or get a phone call, I am hedging and advising them very early on around any types of hiccups in timing, where we might need to stack additional resources assuming a person could drop, something could fail, and really just setting the stage that this is going to be a remote workflow as much as possible in the beginning, because oftentimes, with the way eDiscovery flows, it comes from a partner, it then goes to associates, it then goes to a litigation support like person, or maybe inside a large corporation, it goes through a chief legal officer to a GC to an AGC and then to an IT person, and you’re never getting that, oh, everybody on a call right away until you have two, three, four days of interactions until you get to that call, if you’re lucky, and trying to set the stage for that call now so that you have the right people is critical. You need to really be thinking about it as a vendor as well, being able to respond very quickly to security questionnaires. A perfect storm with COVID-19 was one of our major competitors, Epiq Systems, going down with a ransomware [attack], and that also created some extreme ripples in the eDiscovery industry at the time of COVID-19.
Corporate clients are extremely sensitive to use their eDiscovery vendor’s security and data policies, and I’d like to say HaystackID is a data security company, especially just considering really we’re forensics focused and that’s always a pillar, so we’ve never really had problems there based on our size and scale as well, but really having all that stuff ready to go, understanding that you’re going to be interacting too as you’re looking for remote access, and this is what this is about now. We’re not talking about imaging laptops or cell phones remotely, that stuff’s not that hard. Cell phones can be kind of difficult depending on Android, there are certain things that require more planning or heavier shipments. What we’re talking about now is really setting up virtual infrastructure inside of a corporation to target the new plethora of alternative data types that typically are popping up more and more in complex investigations and litigations, patent disputes, things like that.
Oftentimes, we’re finding that VPN alone is definitely not suitable for remote collection tasks, so really understanding how to speak with IT, or leveraging a vendor like HaystackID that can communicate efficiently and quickly with IT resources and legal to clearly articulate the type of compute and hardware storage resources they will require to complete a collection and/or discovery task. What we see more and more is regulators, plaintiffs, defendants, everybody’s gotten much smarter about eDiscovery. Before there was only so much time in a day leading up to a discovery deadline and as systems do become streamlined, capability to start to look at systems that maybe would have been forgotten many years ago where data wasn’t being pulled, or just like we used to not see cell phones come in all the time into general litigations. Really, over the past three years, it’s been an explosion exponentially. Almost every case I have has some type of mobile device now, but your preservation obligation certainly go more and more to email – I’m sorry. Outside of email network shares, and where this remote access is critical is when you’re interacting with those systems that may be web-based behind your client’s firewall, think of like an on-prem JIRA, any of that, it could be Confluence products, and just more custom-developed applications that organizations may be using internally, and/or being able to handle their data within their firewall. Some organizations might have complex security policies in place, or they’re very sensitive over intellectual property, they may want you to be able to do more of a data collection and calling exercise on-site, and then sometimes just the size and scale of the matter fully necessitates it, and coupled with geography issues.
John and I constantly leverage what we call our remote kit workflow with certain clients, really very easy to deploy on-site, installations of Nuix and/or Relativity where required, and we have a whole plethora of prepackaged tools that handle a lot of the web-based repositories that some vendors try doing it all in one, they get those out and ready perfectly for eDiscovery, and many of your clients are going to be in the cloud now, one way or the other. Cloud adoption is going up pretty significantly, and our ability to deploy globally at scale anywhere our clients are has increased. We’re able to turn on those switches. You’ve got RelativityOne pay-as-you-go models now. You can really be anywhere as you want with an Azure instance. You always want a couple of cloud instances with local data centers, where appropriate, and certainly boots on the ground, but we have a lot more capability to work remotely anywhere, as does every enterprise, and just being able to be agile and to understand these types of data sources, and just to understand the mechanisms for getting dial-in access and that type of remote access to facilitate, eDiscovery objectives is just an important piece of that. You should be mapping out your access plan early on and collaborating with clients there.
John, anything to add?
John Wilson
Yes, and so a big thing that we’re seeing an absolute explosion of it in the pandemic world as now everybody’s working remotely and you have all of these additional communication platforms. People are using the Zoom, they’re using GoToMeeting, WebEx, they’re using Microsoft Teams, they’re using other chat platforms; they’re using mobile device chat platforms, WhatsApp, and all of a sudden, where businesses would have normally said, yes, we don’t use any other chat platforms, now are finding themselves having five and sometimes even 10 different chat platforms that are being engaged and used throughout the organization, and so making sure that you’re thinking about the impacts of the scenario of the world that’s going on at that particular moment. For instance, in this COVID-19 situation where everybody has been put into shelter-in-place and working from home and using alternative methods of communicating with everybody. They’re no longer doing physical in-person meetings, so everything’s virtual, everything’s online. Are they recorded? Is there video, is there audio? Working through all of those additional details.
Michael Sarlo
So true, and anywhere where you can have video interactions with end custodians, so you can see evidence and document that creates such a better layer of defensibility in the remote process, and certainly, there are various ways to integrate and deploy software. We’ve got very large – we have a lot of network shared data that needs to be searched and indexed at once. It’s always been a go-to with Nuix, and usually, we’re not deploying review on-site unless it’s really a bizarre situation where data can’t leave for some reason, but there are different flavors of integration with your end clients, both on the… on the law firm side, [inaudible]. On this call, we have many law firms that have highly functional profit centers in-house for eDiscovery who are doing many of the same things that HaystackID is doing.
We’ve been doing a lot of business continuity planning for corporations and legal departments as it relates to their eDiscovery digital forensics capabilities, and we have actually a whole consulting team focused on that, so definitely feel free to reach out to us if that’s something you’re interested in getting more information on. We have a lot of prepackaged materials that are pretty nice and really have begun to flesh out this framework around the proportionality test with COVID-19 in a pandemic world as it relates to data accessibility, and we’ll be releasing more on that via our standard marketing and outreach channels.
So, we’re going to kick it off now a little bit to Seth Schechtman, review extraordinaire, a wealth of information who oversees our review ReviewRight Secure Remote document division service line and like I said earlier, we’ve been doing remote reviews for 6+ years with better results, oftentimes, than on-premise reviews. What that means is employees working from home or home offices and we’ve had over 500 successful projects in that domain with many of the [AM25s] and many of the largest corporations in the world, so Seth is going to talk about what makes this so special there. I’ll kick it off to him.
Seth Curt Schechtman
Thanks, Mike. We’re going to walk through the virtual environment first, talk about security there, our reviewer selection process, how special it is, our managed review piece, as well as one specific QC and QA technique, that we find really works really effectively to get reviewers on the same page right from the jump, communications and audits. In terms of the security piece, I’ll throw it over to John to talk about the virtual environment, the home environment, the privacy controls and then we’ll walk through some reviewer agreements, codifying protections and setting standards for them.
John Wilson
So what we want to talk about, in the secure virtual review environment, we have very extensive measures to ensure the security of that review, so we have access to the internet restrictions, where there are only whitelisted websites, so like if a client needs to access the law firm website, as the review attorney needs to access the client or the law firm website to do a lookup or use any specific types of tools, but it is very locked down, there’s not – you can’t upload/download things, you can’t transfer stuff, it’s a very locked-down environment we restrict access to email accounts, we restrict access to websites, to the Dropbox and the various common tools for copying files, and the virtual machines that they work through are, in fact, even locked down to where they can’t even save files out locally from within the environment. Everything is locked into a sandbox, which keeps things very secure, keeps the documents locked in, they can’t be copied out. We prevent the installation or upload or execution of any unapproved software. There are software lists, any non-approved software gets blocked, and even, like I said, writing to local drives, but to also even local media, so you couldn’t insert a USB drive and save something to there.
Then lastly, we restrict the application access on the system, so you can’t even go into whatever various applications you may use on your computer, such as Outlook or PowerPoint, other than the types of tools that are needed to do the review. Again, the files are not locally on the local machine, so they need access to it… the only thing they really need access to is the application browser, so they use that browser to connect to the instance, the relativity instance and do the review.
Michael Sarlo
Can I add in one thing, just so everybody is clear? Access into remote terminals is accessed via strong password and two-factor authentication, so you have a two-factor authentication application just to get into the terminal and credentials there. Then you have an IP lock going from the terminal into, in this case, Relativity, 99% of the time, and our manage review is vendor agnostic, so we oftentimes are happy to support other vendors and clients on document reviews as needed, and can easily bolt into other vendor security as it relates to access. Relativity is so robust in that domain. Everything is fully patched; we’re running up to date virus software. There’s full auditing of these. There’s full auditing down to the mouse click, and we’re going to talk about auditing later. What applications are running, what’s going on? This has always been secure and I know for certain organizations, they’re just very… there’s this feeling we’ve been battling for many years with some big law firms that remote review just isn’t something that works, and it’s so off the mark. The security element is the easy thing, as long as you’re being vigilant there.
John Wilson
Just to continue the cycle of what Michael was talking about, we have very specific requirements and the machines that are in use have to meet all those requirements, and that includes certain OS levels and certain security patches, and also a security patch cadence that has to be achieved. Then you have your password, your 2FA authentication that’s required to log into the system, as well as into the environment.
There’s also the antivirus, it gets checked to make sure that it’s up to date, there’s no lag time in the antivirus updates, the secure – connectivity is on a VPN or it’s a secured tunneled IP address that’s been whitelisted specifically or specific geolocation that’s specifically enabled control over firewalls to prevent any applications from… any browser activity from occurring, all those sorts of things.
It is quite an extensive set of security protocols that are utilized. On top of all of that, then we still have access within the review environment, and I’ll hand this over to Seth to talk about, but we have specific requirements for all of the reviewers they have to follow in order to be approved and actually do this work.
Seth Curt Schechtman
Thanks, John. So in terms of their workspace security and we codify this and require reviewers to sign an agreement per project that they’re working on, and we’ll turn to that in a little bit, but that their work is performed in a secure space, meaning non-shared space that is physically and digitally secure, so that’s been a lock on the door. If there’s a window in the room, it has to be locked and monitors have to be facing away from the window and blinds have to be drawn, also require that they do not have any voice-activated phone or phone devices, so Alexa or others in the room.
In terms of privacy controls, we require that the computer that they’re using is not left unattended. If they are getting up, they have to sign out of the virtual session, they have to lock the computer. For calls that they participate in for the project, they must use headphones, they can’t be using a speakerphone, and while on those calls, they can’t mention the clients or case names.
Also, no handwritten notes, we do not want anything physical or tangible. They can’t make any recordings, don’t take any pictures as well as they cannot print the review materials, as John discussed earlier, within the virtual environment, it’s locked down, so from their review platform, whether it’s Relativity or others, they can’t download from there, they can’t print to the computer, so no printing.
In terms of Reviewer Agreements, and this is per project, make them sign a confidentiality agreement and non-disclosure agreement. Also have work-from-home agreements that lay out all of these conditions, privacy controls, physical security, as well as within the virtual environment that they have to abide by. We also make them sign the company handbook to abide by all company policies and ethics attesting – and we’ll talk about this when we turn to the recruiting piece – attest that they don’t have any conflicts for the current matter. Of course, they are still bound – and we remind them of this – by their jurisdictional rules of professional responsibility.
So I’ll explain how reviewers come into our ecosystem, it’s a little bit different than other people, and then we’ll talk about each piece, being the qualifications, the identifying reviewers in terms of the background, what screens we do, ratings that were given for the reviewer, so on per task/per project, and then also the certification that’s required to move up within our ecosystem.
The first piece coming into our ecosystem is we require the reviewers to take a test. It’s 15 documents, four questions per document that touches on issue tags, relevance, responsiveness, and also privilege issue. We use that scoring to project speed and accuracy. As you can see there on the four-quadrant graph, what we’re looking for, people in that upper right quadrant who are performing very highly on both speed and accuracy to make sure that the reviewers that are placed on projects are the best of the best. This is unprecedented in the industry, nobody else does this, and it allows us to get faster better reviewers, which limits overturn percentages, gets you more accurate reviews, as well as saves money because they’re going through the documents at a quicker, more accurate pace.
Michael Sarlo
This is something we can demo on an individual basis. Really interesting. This is definitely a big piece of the secret sauce, both for remote reviews, and for on-prem reviews, and why HaystackID is such a quality deliverer when it comes to document reviewers, making sure that our main assets (the reviewers) are the best and well attuned to your matter.
Seth Curt Schechtman
Thanks, Mike. Yes, we’ll take you under the hood and show you how the system works if you were to reach out after this presentation.
So the next piece that we delve into is identifying the reviewer, so we’re looking on information on their backgrounds, what matters they’ve worked on in the past, what industries, what type of matter, the languages that they speak, the review platforms that they’ve used, the tasks and roles that they’ve performed, allowing us to really pinpoint the best reviewers for your individual matter so [inaudible].
Rare requests, and we’ll say rare requests as not the frequency of them, but in terms of the skills, whether it’s the industry, whether it’s the actual workflow or tasks that they are undertaking, the practice area and given our database of 18,000 reviewer-strong, we’re able to match those reviewers to the specifics of the project that you’re asking for.
We also do, I won’t say standard screens, they’re certainly not fully standard. We interview reviewers when we’re hiring remote reviewers or work-from-home, we really want to get at certain questions and find out certain information about those reviewers, about how they work, how they [rinse out] issues, if they’ve worked on any remote projects before. Reviewers have to have certain experience working on reviews before they’re allowed to work remotely. We do not want anyone green; we have a minimum of three projects and a certain amount of time that they spend on those projects. We’re really looking to get to know the reviewers to put them in places that they can succeed because if they succeed, we succeed, and all of our clients succeed.
In terms of the background checks, I’m not going to walk through this. Once the slide presentation is shared, you can delve into the checks that we’re running. We’re doing license checks, we’re running background checks, and screens to ensure that the reviewers do not have any flags in their background, as well as doing license checks and making sure that they’re in good standing.
The last piece of the reviewer selection process is ratings and certifications, so every project that a reviewer is on, no matter what level is that – the first level is QC, whether it’s assistant RM – they’re getting ranked and rated. So, we have a three-factor scale. The first factor is – I’ve got a five-point scale. The first factor is really coding speed. The second is coding accuracy. Those are purely objective. They’re numbers that are pulled and we’ll talk about it a little later in terms of our auditing that we do and tracking that we do. Then the third piece is intangibles.
We roll that up together, we do a weighted average and each reviewer gets a score, we keep track of those scores, and we keep track of their speed and accuracy from products they have worked on, so ensuring that the reviewers are meeting standards that are above par, and that rating has a huge factor as to whether they get future assignments from us.
In terms of the certifications to move in the ecosystem, we have tests that we’ve developed that we give the reviewers to make sure they know how to do all of the things necessary to move up to those levels, QC and assistant RM, so you’ve assured the reviewers at those levels have been certified by us in terms of minimum projects, minimum time, as well as passing those tests that we administer.
Then we’ll talk a little bit about the Managed Attorney Review process. I know Mike has mentioned the years that we’ve been doing this on the remote side. We’re the originator and leader in the industry for 6+ years and done hundreds upon hundreds of successful matters. You want to make sure that you’re using providers that have this experience, that they have tried and true processes and review managers who have managed remote reviews, they are different than onsite reviews. The communications, which we’ll talk about, and the auditing pieces are certainly different than traditional. Our review managers all have 7+ years in the industry with years and years and years of experience.
On the remote side, they will reduce review costs, optimize workflow, cut down on review sets, target key or interesting documents from the jump of the matter. We have processes that we put in place for search term testing. If you are using search terms, CAL and TAR consulting as well, as well as other pieces and workflows that we’ve put in place to really reduce review sets, whether it’s single instance of identification of non-responsive documents per se, custom deduping, which is really super interesting. As documents get collected from different sources, the exact same documents, which would normally be an MD5 duplicate at the family level, happen to get different MD5 codes, so we can identify those documents and not… we can either suppress them from review if that’s allowed or make sure that the coding is consistent across them. We do sorting and review, we typically like to do it by thread group, whether we’re suppressing non-inclusives or not, whether you have these documents that would be duplicates that haven’t been duped out, they’ll get different thread group identifiers, so if you separate them out to reviewing non-chronologically, you can have inconsistencies there, so we make sure that those are consistent as well as developing techniques for propagation of coding, as well as redactions across similar documents, typically individual MD5s.
One thing that we’ve developed as a quality control/quality assurance technique is to ensure reviewers are on the same page from the start of the review. So we talked a lot about the reviewer selection and testing and identification, matching them to the product, we screen them on the ratings and certifications, but each matter is unique, each matter is different and we want to make sure from the jump of the review that the reviewers are kept on the same page. They all go through training sessions with the client or with counsel to get the protocol. The first thing that we like to do is have them code the same set of documents, whether it’s 25 documents or 50 documents or 100 documents, whether they’re duplicated within the review platform or we have them fill out surveys or forms outside of the review platform in a secure system, secure chat system, we want to compare and contrast coding, we want to compare and contrast internally to what the team is doing against themselves to spot any reviewers are inconsistent. We also have counsel code those documents. We want to see that if our team is diverging from what counsel – and sometimes it happens, sometimes a review protocol is written that it isn’t foreseeing the type of information that is in the documents, or counsel just thinks they drafted the guidelines straightforward and our team is interpreting differently, so it allows us to both get individual reviewers who may not understand the protocol, as well as identify those protocol deficiencies, and right from the start, we can make sure that everyone is aligned.
In terms of communications, we like to say, “early and often”. The initial training session is mandatory, and we take verbal attendance on it. It is recorded and that’s all recorded with counsel’s and/or client’s approval, let’s say we’re adding to a team, you can play that recording for the reviewers, but the Q&A session is always done live. You want to make sure that the reviewers are getting their questions answered and not just rely on a recording. Those will have daily or weekly project calls. Certainly, daily, when the matter first starts. It can go less frequently than that. If counsel doesn’t want to be a part of those calls, fine, we certainly encourage it, but we have internal calls amongst the team. We also have calls amongst the QC-ers and the RMs to make sure they’re on the same page. It’s essential to have those calls to make sure people have an ability to ask questions over the phone.
In terms of instructions, we want all instructions to be written from counsel, so even when we do have calls, we designate two note takers and it’s not handwritten, obviously, on a pad, it’s electronic means, as we talked about before, we do not allow reviewers to take handwritten notes. Then the notes are circulated for counsel’s approval. Once they’re approved, they come back for wide distribution to the team to make sure that everyone is kept on the same page.
We also have a secure chatroom that the reviewers use to communicate with each other. Those rooms are organized by task, so if we have different members working on different tasks, let’s say ones that were sponsors of a portion of the review, ones a privileged portion of the review, we keep those, we setup separate rooms. Within each room, we can have separate threads for separate topics. We also setup a schedule whereby the RMs and the team leads have a schedule for monitoring to make sure we have full coverage to answer questions. We have separate rooms for the QC-ers and the team leads and the RMs so they can communicate amongst themselves, so if there’s some disagreement in terms of their coding, we can get it settled or bubble it up to counsel. We can also give counsel or clients access to the secure chat so they could monitor the team communications as well.
As standard in the industry, there’s an issue log, or a decision log (some people call it) where reviewer questions and feedback are logged chronologically and grouped by topic to make it easier. That’s distributed to counsel daily or sometimes even more frequently at the beginning of an initial stage in the matter, and it’s posted on a secure shared drive for the entire team to monitor. We have the ability to access the viewing history, so making sure that the reviewers are reading it daily before they start coding, or whether there’s a new iteration out, making sure that we’re checking that, and we’ll talk about that in the audit piece.
Also, notification of any technical issues, so if reviewers are working from home, they have either notify us, the RM on the matter or the recruiting department, if they have any platform issues, email issues, any issues with their virtual environment or secure chat, or anything else necessary to perform their duties, we want to make sure that we’re advised and can take care of the technical issues if they are present immediately.
Also, on the communications front, we have shared resources, so the reviewers are also pounding through the documents, and it’s a whole team of people spending hours and hours and hours per day, there are learning blocks, they are building that learning curve. We want to document those things to make coding easier, so whether it’s acronyms or initialisms, also shared lists of attorneys, we’re adding to the attorney list project codenames, which sometimes will appear, as well as chronologies and custodian crib sheets. All that stuff, that knowledge is built up by the review team and we wouldn’t want it to go to waste, we want to share it with counsel, share it with clients to be able to make that matter easier and maybe just other matters that they have in the future. It’s super important to give feedback with samples, both positive and constructive feedback. Those documents, they’re shared with the team, but saved searches within review platforms or folders and we can audit the history there to make sure the reviewers are actually looking at those sample documents. You want to make sure that the reviewers – any reviewer diverging from the standards, are kept in line.
In terms of auditing, our philosophy is trust, but verify. A few of the things I talked about already briefly, so mandatory participation on calls with verbal attendance, or they can be on a secure chat to say that they’re present. We also want to make sure that they are throughout the call, so taking it at the end, sometimes we’re even giving codenames and code words and make sure the reviewers give it back to us to make sure that they’re participating. Project emails, having read receipts to make sure they’re actually getting them, taking attendance by the team leads, the review managers in the secure chat. Reviewers are required to be in that secure chat while they’re reviewing documents. Viewing the audit history of docs and then reviewing the audit history for the issue log, the issue log to make sure that they’re viewing it every time anyone comes out or before they start coding each day.
We also do reviewer quizzes, so maybe throughout the review, maybe at the start of the review. Here’s just a little sample there, and the results of one question, we want to make sure that the reviewers are kept in line and they understand, it’s typically for procedural stuff and can be used for substance in terms of procedure. Are you fully coding family members of responsive documents, when you’re coding privilege types and the like?
We also audit individuals’ DPH (documents per hour) per task, as well we OT, meaning overtime tracking to make sure that people who are above or below multiple standard deviations are checked, so it means if they’re going super-fast or super slow, that we’re checking into those paces.
We talked about productivity audits, we’re checking platform usage, or billing time versus platform usage and active time, including plotting review rates over time. We’re looking for reviewers who are doing quick bursts of activity, followed by fallow period. Then we have separate accounting for non-review workspace time, it makes checking that time a little bit easier. Obviously, they’re spending some time outside the workspace reading issue logs, reviewing training materials, conference calls and taking reviewer quizzes as well.
The last thing we’ll talk about is coding consistency, so I mentioned MD5 duplicates a couple of times already. As long as coding is in the four corners of a document, we’re not propagating throughout based on family members. You can use that MD5 coding on those MD5s to see if the reviewers are outside of the norms for consistency and accuracy. You can also see – it helps elucidate portions of the protocol that may need further analysis or further information from counsel.
Michael Sarlo
Thanks, Seth. That was a really good breakdown and just so everybody knows, it’s a high-level overview of our QC procedures and playbook. We’ve spent years vetting, testing, evolving different types of matters and workflows.
We have some questions here. One that is a great one. “Who among you is willing to look into the future, post-pandemic shutdown and predict future of remote review? Will it fade away or become established?”
I’ll actually take a piece of that and anybody else who wants to jump in from my team feel free to as well. Like I said, we’ve been doing remote review for six years. These institutional players in the eDiscovery side of certain law firms that have been risk-averse to this, or general counsel’s office that has been risk-averse for a variety of reasons. Although the standard for experts, oftentimes foreign language experts has been remote for quite some time. Remote is definitely here to stay. All my clients who… when it was oftentimes, ‘oh, we really want to do it, we know it’s better, but a GC’s office, we just can’t get through there’. Everybody was forced to look at who does this and who doesn’t, and then who is trying to do it, and there are clear differences. Remote review is here to stay for sure.
The biggest thing is I get these cases and anybody who’s ever done a sector request in the Washington, D.C. market and you have a ton of deals going on at once and you start to get a depleted pool, there are no reviewers at all. Remote is the magic bullet to making sure that you’re not stuck with the bottom of the barrel as far as the bell curve as it relates to speed and accuracy.
Definitely here to stay and, overall, organizations are going to benefit from remote review and wider acceptance of remote review—
John Wilson
I was just going to say that remote everything is likely to be here to stay. A lot of businesses, industries are all getting a sudden force-feed use of remote work and remote operations and I suspect quite a bit of it will stick.
Seth Curt Schechtman
I couldn’t agree more, John. For all of you who have experience with onsite reviews in terms of a review center, then once you go remote, it’s really night and day. As Mike mentioned, the pool of reviewers opens up and you’re not limited by market condition. We have metrics to back all this stuff up. You get faster reviewers more accurate, and in terms of cost, comparing them to D.C. or New York or the high-cost places, it’s not even close.
All things being equal, even if they weren’t faster, even if they weren’t more accurate, you would be saving money, but you get the trifecta there; faster, more accurate, as well as lower cost.
Michael Sarlo
Well, I just want to thank everybody again for joining us. It’s been a pleasure as usual. Feel free to reach out to any of us. My contact information is, actually, on the questions slide so feel free to contact me directly, and happy to answer any questions. Always happy to provide consulting across the EDRM and for anybody who is working to evolve their practice in a pandemic world.
I want to kick it off to Rob Robinson to close out this session and look forward to seeing you guys on the next one.
Closing
Thank you so much, Michael, and thank everybody on the presenting team for the information and insight. We also want to thank everybody that took time out of their schedule to attend today. We know how valuable your time is and we appreciate you sharing it with us, and just want to reiterate for you that a PDF copy of the presentation is available from the HaystackID website, as well as an on-demand version of this presentation. Some additional information that might be of interest, I know there was a question on the future of secure remote review. Within the tabs on your presentation window, there’s an article written on secure remote review that may provide some industry information beyond just HaystackID commentary. Additionally, we do have a datasheet that is available for specifics on our ReviewRight Virtual and ReviewRight Match offerings that Seth, Michael, and John shared on during the presentation today.
Additionally, I wanted to invite everybody to attend our next webcast, which we conduct on a monthly basis. It’s scheduled for May 20 and it’s on the topic of eDiscovery Technology and Tools. In that presentation, we’ll have some expert eDiscovery technologists and authorities to talk about some of the current technologies for data analytics, artificial intelligence, process automation, and data enhancement. They’ll also offer some practical considerations for implementing these technologies in your eDiscovery operations.
Thank you so much, again, for attending and we wish you health and safety during this challenging time. Have a great week. Thank you from HaystackID.
CLICK HERE TO DOWNLOAD THE PRESENTATION SLIDES (PDF)
2020.04.22 – HAYSTACKID – eDiscovery in a Pandemic World PresentationCLICK HERE FOR THE ON-DEMAND PRESENTATION (BrightTALK)