The fascination we have with generative AI is rooted, of course, in its creativity. Going beyond more “traditional” types of artificial intelligence—the ones that compute, calculate, and categorize—generative AI applications present as companions as much as computers.
It’s a pretty wild world for everyday users. How nice to have a chat with a well-informed source on a fascinating new subject instead of simply googling and fending for ourselves.
It’s also a pretty wild space for legal practitioners. From conducting research to reviewing documents, generative AI is already well established with a large cohort of legal teams. But it’s also posing new questions around e-discovery that can’t be ignored.
At Relativity Fest Chicago 2024, an esteemed panel in a session entitled “Scaling the Cliffs of AI Insanity: Managing Data Created by Generative AI” helped answer some of these questions. Speakers included:
- Aron Ahmadia, Senior Director, Applied Science, Relativity
- E.J. Bastien, Sr. Director, Discovery Programs, Microsoft
- Todd Itami, Director of Artificial Intelligence and E-Discovery Solutions, Of Counsel, Covington & Burling
- Ben Sexton, Vice President of eDiscovery, JND
- Ashley Picker Dubin, Counsel, Day Pitney LLP
Last week, we shared the panelists’ insights on how generative AI data poses unique—and not so unique—questions for e-discovery teams. In this second part of our coverage of this session, read about their analysis of the complexities of AI-generated data discovery and their advice on how to get your arms around it.
Understanding Custodians’ Use of Generative AI and Its Implications
As with all new tools in an enterprise, the use of generative AI tools must be understood by Legal Data Intelligence practitioners in charge of downstream functions like compliance, information governance, e-discovery, and practice management. Because ultimately, they’re creating new content that is both an asset and a potential complication to organizations of all types.
Whether this content will consistently surface in litigation matters, specifically, is yet to be seen. But our panelists had some interesting thoughts on the question.
“Using generative AI for e-discovery and document review in particular: you have new content being generated, like your generative AI responsiveness field, considerations, rationale, citations, prompts, and their iterations,” Ben Sexton told the audience at Fest. “I’m not sure if we’re to the point where these are becoming part of disclosure, but the conversation is starting and what becomes discoverable is yet to be determined.”
To better prepare for the possibility of discoverability, legal teams should start at the source: how are custodians using generative AI, and why?
“Talking with custodians about where and how they work is important. Even in companies as monolithic as my own, people use the standard tools—but considering acquisitions and other activities, people use their own things, too,” EJ Bastien said.
In-house teams should have similar conversations as they’re determining retention policies for generative AI tools.
“I’d rather err on the side of preservation than have the government come back and say ‘we’ll fine you for failure to preserve.’ It’s not the fire swamp you imagine, but we’re in a place akin to the email revolution,” Ashley Picker Dubin said. “We haven’t seen this kind of mass change and adoption all at the same time in a long time. There was a time when people argued there was no need to preserve and produce email. That was a legitimate standpoint not that long ago.”
She continued: “We still need to consider defensible disposition. Should it be retained in ordinary course along with Teams data? Or your chat data generally? If you have a 2- or 5-year policy, can your prompt and response data fall under that policy? Where will it be—and are they subject to the same retention policies? Those are questions we’re already dealing with in collaboration tools, and it’s important to discuss.”
Ultimately, EJ said, “Let’s talk about what the policies should be and then examine how tech can enable that. If we know what should happen and find that it’s different than what can happen, that gap is real and needs to be discussed.”
Keeping a bird’s eye view of both the internal and external dynamics—including compliance, case law, and other protocols and guidelines—affecting retention and discovery of AI-generated data is also essential.
“There are rules already in place. You can be a good lawyer or a bad lawyer, checking or not checking work and citations. Generative AI doesn’t create a new bar; the rules already stand,” Ben said. He echoed the panel’s note that practitioners can use existing guidelines to make educated decisions on how to manage generative AI’s use and data in their organizations—while touching on some of the more novel questions we’ve yet to settle as an industry.
“On the tech side, we run a lot of collections. That involves, from both sides of a case, negotiations about what’s discoverable. You have a spectrum of availability: email is on every case, Teams gets pulled often, OneDrive often, desktops less so. Volatile memory is very rare; Google searches almost never come up in civil litigation,” Ben continued. “Where is that chat interaction going to fall on that spectrum? I’m curious. Will it be like Teams, which is becoming more prevalent, or like Google searches, where it only comes up when you really need to dig into something?”
The data itself, Ben noted, doesn’t align so easily to any established category in this regard.
“Querying a chatbot a similar user experience as Google, so I can see the producing side arguing it’s not as relevant. But from an availability standpoint, Microsoft is making this data very available,” he explained. “Google searches aren’t on the table because they’re very expensive to extract. But with these chats, it’s a simple export, so the brave new world might be TBD. We’ll have to see how firms are using the data.”
Getting Started with AI Data Discovery
Okay, now, all of this being said—where can legal teams begin? How can they proactively manage AI-generated data for discovery and other matters when so much remains to be seen?
Step one is: don’t let it scare you.
“In the end, what we have here is data of a not-so-unusual source. Yes, it’s new, it’s shiny, it’s generated. But as of now, it’s retainable, we can destroy it, it can be collected and produced,” Ashley said. “We can treat it as any other data source, but we have to know to ask. Ask clients what they’re using. Are they using anything they shouldn’t be?”
For EJ, educating oneself on the technology and its use in your industry—and getting familiar with it for yourself—are two big steps in the right direction.
“Understand the landscape, behavior, the tools. We tend to curse the problems, but there’s a lot of potential here. There are many things that can be done and gained from these solutions that will help us address real-world problems more efficiently,” EJ advised Fest attendees. “Look at what you can do now, and maybe six months from now—don’t invest a bunch of your energy into understanding just this one point in time. The pace of innovation is so fast. Keep an open mind and think about what you can do the next time the question arises.”
Todd leaned into the inevitability: innovation will continue, and its impacts will hit sooner or later.
“Before England entered the war Churchill said, ‘You were given the choice between dishonor and war, and you’ve chosen dishonor, but you’ll have war.’ This is happening no matter whether you want it to or care about it or not,” he noted. “But it’s so accessible. The sources that make you feel like you can’t do it, or it’s too hard—you don’t need that. It is accessible. People are here to help show you how this works, and take you along, and show you examples of how all this works.”
Members of the judiciary are coming along, too. The excitement around generative AI is broad and touches many functions in the legal world.
“Judges have reached out to us to ask for how to put some standards in place in their chambers. This is powerful stuff—a step function for the industry. It’s not just an ancillary feature you can pick and choose whether or not to use,” Ben emphasized. “I leave people with this: if you have access to Relativity aiR, run it on 10 docs. Just test it. Compare it to human review and get a feel for it. Use any tool, but whatever it is, get a feel for the new paradigm. It’s real.”
