Most firms have one category of work they would automate tomorrow if they could, and never will: the documents they are not allowed to send anywhere.
Client intake. Case files. Deal memos. Medical records. Anything covered by a confidentiality agreement someone actually read. The rule is reasonable, so the work stays manual, and the firm watches every other process get faster while that one does not move.
The constraint was never really about AI. It was about where the file goes. Change that, and the rule stops being a wall.
The rule is not the problem
When a firm says it cannot use AI on its most repetitive work, it is rarely a philosophical objection. It is a contract. A client agreement, a regulatory obligation, or a policy written by someone who was right to write it.
Sending a client file to a third-party API means it lands on hardware the firm does not control, under a retention policy the firm did not write, in a jurisdiction it may not have chosen. The vendor may promise it is not used for training. The promise is not the point. The firm cannot verify it, and if it is wrong, the firm is the one holding the liability.
So the work stays on someone's desk. The intake form gets read by a person, the fields get retyped into the practice management system, and the follow-up gets written from scratch for the four hundredth time.
What actually changed
The reason this was a dead end for so long is that running a capable model on your own hardware used to mean a server rack, a specialist to run it, and a capital expense that dwarfed the problem.
That is no longer the shape of it. A single desktop machine with 48GB of unified memory, bought used, runs a 30-billion parameter model at usable speed. It costs less than most firms spend on software in a quarter and it fits under a desk.
Two things happened at once. Models got dramatically better at the small end, and consumer hardware got enough memory to hold them. A model that would have been research-grade three years ago now runs on a machine you can buy on eBay.
Being honest about the tradeoff
Local models do not beat frontier models. They lose, and anyone telling you otherwise is selling something. On open-ended reasoning, long context, and anything requiring broad world knowledge, the gap is real and it is not close.
That gap matters enormously for some jobs and not at all for others.
Pulling eight known fields off a standard intake form is not open-ended reasoning. Neither is drafting a follow-up email that follows the same shape every time. These are narrow, repetitive, well-defined tasks with a correct answer that a person can check in seconds. For work like that, the question is not "which model is best" but "is this model good enough, and where does the file end up."
| Cloud API | On your own hardware | |
|---|---|---|
| Raw capability | Higher | Lower, often sufficient |
| Where the file goes | A vendor's servers | Stays on your network |
| Cost shape | Per token, forever | One-time hardware |
| Verifiable | By policy | By architecture |
The last row is the one that matters. A cloud vendor's data handling is a promise you accept. A machine on your own network is a fact you can point at.
What a pilot actually looks like
We are running this setup now, on fabricated sample documents, because that is what a pilot is for. The shape:
A document arrives. A smaller model pulls the structured fields out of it. A larger model drafts the follow-up. Then the workflow stops and waits for a person to approve before anything goes out.
That checkpoint is not a limitation of the local setup. It is the point. The same rule applies whether the model runs in a data centre or under a desk: the system handles the repetitive part, and a person still signs off on the part that carries consequences.
The whole thing runs on one machine, reachable over a private encrypted connection, with no port open to the internet. The execution history is the audit trail. If someone asks what the system did with a document six months ago, there is a record, and the record is yours.
What to take away
- Check whether your blocked work is blocked by capability or by data movement. These have different solutions, and most firms have not separated them.
- Size the task honestly. Narrow extraction and templated drafting run fine locally. Open-ended analysis still wants a frontier model.
- Keep the approval checkpoint regardless of where the model runs. Local hardware changes where the file goes, not whether a person should be the one deciding.
The AI does the busywork. You still make the calls.