Motivation

Is there a better way to express the law?

The law is currently expressed in regular human language.
- This leaves much open to interpretation (possibly by design?)
- This is not simple to understand
- It requires experts to reliably tell what is/isn’t legal, and even then there is regular dispute
The superior expression of ideas is foundational to human history. A breakthrough in the expression of ideas leads to a boom in productivity.
- Writing
- Printing press
- Code
You do not always know the exact use cases before you implement
- Writing arose from tokens designed to keep track of grain & livestock
  - Precursors to writing were used purely for counting & accounting for 5 thousand years (~8,000 BC to ~3,000 BC)
- Computers were made to crack the enigma code
Common law was an efficient strategy to crowd-source good decisions around specific cases - cases which were too specific to be written into law
- Precedent = data

If there is a better way to express the law, LLMs are the key to implementing that expression across relevant laws and proving immediate benefit.

Suggested Use-cases

Computation: Converting law into data will permit novel computation on the law (e.g. rules engine)
Search: Distilling the law into a simpler form will make building search on top of it simpler
Visualisation: Identify connections between statutes

Experiment

Convert a law into a simple data structure, mapping to prohibitions with exceptions and penalties: https://www.revisor.mn.gov/statutes/cite/481.02

I used Palantir's AIP as a prototyping tool for this problem

Prompt

You are a lawyer, logician, information theorist, and data scientist. You believe it is possible to express laws in a superior way using data and logic. Your goal is to distill laws down into their fundamental and simplest form without losing any detail. You are aiming to convert the law into a consistent format that generalises across all laws. For example, laws contain prohibitions, exceptions, and penalties. You hope to be able to run computations on the data. Please extract the following law into the requested data structure in its simplest form without losing detail.

Requested data structure

export interface IProhibition {
  prohibition: string

  exceptions: string[]

  penalties: string[]
}

Result

const test4 = [
  {
    prohibition: 'Unlawful for non-Minnesota bar members to practice law or hold out as qualified to give legal advice or prepare legal documents.',
    exceptions: [
      'Members of the Minnesota bar.',
      "Drawing documents without charge for oneself or one's employer, except wills.",
      'Drawing a will in an emergency if death is imminent.',
      'Insurance companies defending insureds through selected lawyers.',
      'Licensed attorneys acting for common-carrier corporations.',
      'Labor organizations giving legal advice to members.',
      'Cooperating with a licensed attorney in preparing legal documents.',
      'Licensed attorneys drawing documents for corporations they are employed by.',
      'Drawing certain non-testamentary documents like leases or mortgages.',
      "Attorneys rendering services to corporations at stockholders' expense.",
      'Collection businesses engaging attorneys for legal actions.',
      'Publishing legal Q&A by licensed attorneys in periodicals.',
      'Management agents of rental properties in certain legal actions.',
      'Nonprofit corporations in certain rental property actions.',
      'Specialized legal assistants with a pre-1995 license.',
      'Sole shareholders appearing for their corporation in court.',
      'Representatives of associations in conciliation court.',
      'Real estate professionals drawing papers related to property transactions.',
    ],
    penalties: [
      'Misdemeanor charge for violations.',
      'Prosecution by county attorneys.',
      'Injunction suits by county attorney or attorney general.',
      'Public and private penalties as per section 8.31.',
    ],
  },
  {
    prohibition: 'Corporations for profit cannot practice law or hold out as providing legal services.',
    exceptions: ["Attorney's professional firms organized under chapter 319B."],
    penalties: [
      'Misdemeanor charge for violations.',
      'Prosecution by county attorneys.',
      'Injunction suits by county attorney or attorney general.',
      'Public and private penalties as per section 8.31.',
    ],
  },
  {
    prohibition: "Unlawful to charge attorney's fees for mortgage foreclosure unless conducted by a licensed attorney.",
    exceptions: ['Fee division with another licensed attorney who assisted in services.'],
    penalties: [
      'Misdemeanor charge for violations.',
      'Prosecution by county attorneys.',
      'Injunction suits by county attorney or attorney general.',
      'Public and private penalties as per section 8.31.',
    ],
  },
  {
    prohibition: 'Corporations acting as fiduciary agents cannot perform legal work except through a licensed attorney.',
    exceptions: ['Fee division with another licensed attorney who assisted in services.'],
    penalties: [
      'Misdemeanor charge for violations.',
      'Prosecution by county attorneys.',
      'Injunction suits by county attorney or attorney general.',
      'Public and private penalties as per section 8.31.',
    ],
  },
  {
    prohibition: 'Non-Minnesota attorneys cannot practice law in Minnesota without court permission.',
    exceptions: ['Reciprocal permission for Minnesota attorneys in other states.'],
    penalties: [
      'Misdemeanor charge for violations.',
      'Prosecution by county attorneys.',
      'Injunction suits by county attorney or attorney general.',
      'Public and private penalties as per section 8.31.',
    ],
  },
]

Analysis

The LLM seemed to do a passably decent job of identifying different prohibitions as well as their exceptions and penalties. Qualitatively, I found the LLM's prohibitions & exceptions easier to follow than the law itself.

As a non-lawyer, I would have a very challenging time affirming the exact accuracy or completeness of the result. That said, it is very likely that parts of the law are omitted. e.g. "in accordance with the terms of the policies" is left out of the exception "Insurance companies defending insureds through selected lawyers."

Any serious solution would require prompt engineering, validation, and a better data model.

Direction

Identifying a better structure/expression

Should express logic (AND/OR)
Should express variables (e.g. link laws that reference each other)
Investigate/Assess Blawx data structure.
Requirement: Ideally should be agnostic of the tooling built upon it (i.e. a somewhat universal expression)

Data pipeline

Scrape laws en masse (e.g. all MN statutes)
Classify which laws are “relevant” to the problem
Use an LLM to map the relevant laws into the data structure
Consider using another LLM to test/review the output of the first, and rerun incorrectly mapped laws
Requirement: It must be easy to change the data structure we map to and rerun the pipeline
Requirement: It must be easy to update the LLM mapper and rerun the pipeline

pipeline

Tooling

Rules engine: Blawx?
Search engine

Experiment/Refine

How do open source LLMs perform?
Is there a way of evaluating performance?
What if we train (i.e. fine-tune) an open source LLM on our laws set - does it perform better?

Part 2

To see how selected a data structure to dial this in to something more specific, see part 2

Law Expression