When Withers was approached by software-as-a-service business Diligent, the Withers tech team knew that an AI solution could provide the answer to the firm’s problem, within the tight timeframe it had available.
Diligent was looking for a way to analyse certain commercial and legal elements of its entire suite of approximately 45,000 global customer contracts so that it had a clear picture of its contractual rights and obligations. In particular, Diligent was looking to extract 16 data points from the contracts and house them in a database that could be used to inform the plausibility of implementing new pricing structures for existing customers. If this could be successfully achieved, the firm was then hoping to take another 44 data points from the contracts for analysis that would give its in-house legal team a deeper understanding of the firm’s legal risks, allowing it to stress-test certain scenarios and make better-informed legal and business decisions.
“We knew that an AI system could provide the analytical power that Diligent was seeking, and it could be developed and deployed fast enough to meet their timescales. In fact, we were able to repurpose tools that we had been experimenting with prior to Diligent’s request to us, where we had been looking at the application of generative AI, ’embeddings’ technology and rules-based AI,” says Ben Williams, the Withers tech partner who led the project.
Ben and Alex Taylor (then, Senior Associate) assembled a team of associates and worked with a data scientist to implement a pilot phase on around 400 contracts. This allowed the team to de-risk the process for all parties and gave Diligent the chance to assess whether the proposed solution would be good enough. This also gave Withers the opportunity to identify any significant obstacles at an early stage and to assess the functionality of the technology in ‘real-life’ conditions and provided greater clarity on cost.
The outcome of the pilot phase showed that more work was required to overcome a number of technical and legal obstacles, but with the output delivered a week early and showing sufficient progress, Diligent authorised the next phase. This entailed the team working through a multi-stage process to pre-sort the data and configure the model by designing a comprehensive plan, methodology, and instructions for the AI. The tool was then ready to run through a number of test cycles where the configuration could be adjusted. Once sufficient accuracy had been consistently demonstrated, software was built to process all contracts at scale and all 16 data points were extracted within 48 hours at a composite accuracy of more than 96%.
The team were then able to move on to analysis of the further 44 data points, using the same methodology. This was completed at the same level of accuracy, which was improved further with manual analysis.
“Put simply, Withers succeeded where all others failed. We were let down tremendously by another provider who was unable to delivered anything close to what we needed. To put Withers into perspective, Alex and Ben were able to use AI to deliver exactly what was required on time and on budget. Having utilised generative AI ourselves, we were acutely aware of the scale and complexity of the challenge. However, Withers managed to push the boundaries and deliver an output that is, frankly, transformational for us. My in-house legal team now has the information and insights it needs to support the business in its decision making. Alex and Ben’s innovative use of large language models was extremely impressive,” comments Warren Allen, VP & Associate General Counsel at Diligent.
“The ‘lightbulb moment’ was the recognition that we could combine our legal experts with our emerging data science function and use generative AI to meet Diligent’s needs on a very short timescale and within the budget set for the project. Use of AI was the only feasible way to get this done right and, importantly, the solution was built entirely in-house without involving any ‘off-the-shelf’ legal technology provider,” adds Ben.
The successful project has been selected by the Financial Times as a shortlisted project for its Innovative Lawyer Awards Europe and by Legal Business as a shortlisted project for its Legal Technology Team of the Year.
