Wakeling has been particularly impressed with Harvey’s prowess at translation. It’s strong at mainstream law, but struggles on specific niches, where it’s more prone to hallucination. “We know the limits, and people have been extremely well informed on the risk of hallucination,” he says. “Within the firm, we’ve gone to great lengths with a big training program.”
Other lawyers who spoke to WIRED were cautiously optimistic about the use of AI in their practice.
“It is certainly very interesting and definitely indicative of some of the fantastic innovation that is taking place within the legal industry,” says Sian Ashton, client transformation partner at law firm TLT. “However, this is definitely a tool in its infancy and I wonder if it is really doing much more than provide precedent documents which are already available in the business or from subscription services.”
AI is likely to remain used for entry-level work, says Daniel Sereduick, a data protection lawyer based in Paris, France. “Legal document drafting can be a very labor-intensive task that AI seems to be able to grasp quite well. Contracts, policies, and other legal documents tend to be normative, so AI’s capabilities in gathering and synthesizing information can do a lot of heavy lifting.”
But, as Allen & Overy has found, the output from an AI platform is going to need careful review, he says. “Part of practicing law is about understanding your client’s particular circumstances, so the output will rarely be optimal.”
Sereduick says that while the outputs from legal AI will need careful monitoring, the inputs could be equally challenging to manage. “Data submitted into an AI may become part of the data model and/or training data, and this would very likely violate the confidentiality obligations to clients and individuals’ data protection and privacy rights,” he says.
This is particularly an issue in Europe, where the use of this kind of AI might breach the principles of the European Union’s General Data Protection Regulation (GDPR), which governs how much data about individuals can be collected and processed by companies.
“Can you lawfully use a piece of software built on that foundation [of mass data scraping]? In my opinion, this is an open question,” says data protection expert Robert Bateman.
Law firms would likely need a firm legal basis under the GDPR to feed any personal data about clients they control into a generative AI tool like Harvey, and contracts in place covering the processing of that data by third parties operating the AI tools, Bateman says.
Wakeling says that Allen & Overy is not using personal data for its deployment of Harvey, and wouldn’t do so unless it could be convinced that any data would be ring-fenced and protected from any other use. Deciding on when that requirement was met would be a case for the company’s information security department. “We are being extremely careful about client data,” Wakeling says. “At the moment we’re using it as a non-personal data, non-client data system to save time on research or drafting, or preparing a plan for slides—that kind of stuff.”
International law is already toughening up when it comes to feeding generative AI tools with personal data. Across Europe, the EU’s AI Act is looking to more stringently regulate the use of artificial intelligence. In early February, Italy’s Data Protection Agency stepped in to prevent generative AI chatbot Replika from using the personal data of its users.
But Wakeling believes that Allen & Overy can make use of AI while keeping client data safe and secure—all the while improving the way the company works. “It’s going to make some real material difference to productivity and efficiency,” he says. Small tasks that would otherwise take valuable minutes out of a lawyer’s day can now be outsourced to AI. “If you aggregate that over the 3,500 lawyers who have got access to it now, that’s a lot,” he says. “Even if it’s not complete disruption, it’s impressive.”
