🔢 Is AI Bad with Numbers?

The Truth About Clause Numbers in Legal Documents

I often hear people say, “AI is bad with numbers.” But is it really? The answer is more nuanced than you might think!

📝 Just Tell Me the Clause Number!

Let’s dive into one specific scenario: clause numbers in legal documents. Can AI identify them accurately?

AI doesn't see the clause number

Well… not always. But is that the AI’s fault? 🤔

Turns out, it’s often not an AI issue — it’s a problem with document conversion! It is important to understand that AI products (e.g., ChatGPT, Claude, Wordsmith) don’t read Word documents directly. First, the document gets converted to plain text, and this conversion can lose some critical information — like clause numbers. So when you ask AI about the “Termination for Cause” clause, it may not see the clause number simply because it wasn’t included in the converted text!

And it is very easy to see this - if you pass the same document as PDF, AI will have no problem identifying the clause number!

🔧 What Can You Do About It?

Here are a few ways to ensure better results when working with AI and legal documents:

• Copy & Paste the text of the document directly into the chat. It’s a bit manual and inconvenient, but you control exactly what AI sees.

• Convert to PDF: PDF format often preserves clause numbers better. However, be mindful that you might face some other issues here. For example, multi-column document layout can present a challenge.

• Use Specialized Legal AI Tools: Legal-focused AI solutions are designed to handle these challenges and validated on a large number of legal documents. Some of them also provide additional useful features like direct references to the original document, making it easier to verify answers.

There are many more nuances and tricks about AI and numbers, which I’ll talk about in subsequent posts, so stay tuned and feel free to subscribe to my substack if you don’t want to miss it.