Adding AI to legacy software without a rewrite

The system that runs your business is fifteen years old, nobody enjoys working on it, and the quote to replace it is a year and a number with several zeros. Then somebody asks whether you can add AI to it, and the answer that comes back is that you would have to rewrite it first. That answer is usually wrong, and it is expensive in a specific way: it converts a small improvement into a project that competes with everything else you wanted to do this year.

A rewrite is the right call when the old system cannot express what the business now does — when the data model itself is the constraint. It is the wrong call when the system works and is merely unpleasant to extend. The second case is the one we are usually asked into, and three patterns put modern behaviour beside an old system without touching its core. They compose, and we rarely reach for one on its own.

Pattern 1 — the API layer

Put a documented boundary in front of the old system and make everything new talk to that. It is a small service exposing the handful of operations the new work actually needs — read a customer, list open orders, create a note — implemented against whatever the old system offers: a stored procedure, a nightly export, a screen you have to drive, a real endpoint if you are lucky.

The rule that makes this pattern safe is that the AI never touches the database. Every read and every write goes through the boundary, which means validation, authorisation, rate limits and an audit trail all live in one place written this year, rather than scattered through code you inherited. When the agent asks for something it should not have, the layer refuses, and the refusal is logged.

The layer also buys the option you actually want later. Once the new work depends on the boundary rather than on the schema, the old system can be replaced behind it in pieces, one operation at a time, with the callers unchanged. That is the same escape route a rewrite promises, except you collect value along the way instead of at the end.

Pattern 2 — retrieval over data you already have

Most of the answers people want are already in your systems: past tickets, order histories, product notes, the internal wiki, the files on a shared drive. They are simply not retrievable at the speed of a question. Retrieval puts an index beside the old system — never inside it — and lets a model answer from passages it can cite.

Two things make this work rather than merely demo well. The first is permissions: the index has to know who may see each document, and retrieval has to filter on that before the model sees anything, not after. An index that ignores the old system's access rules is a data leak with a chat window in front of it. The second is freshness: decide what recent enough means for each source — minutes for orders, a day for documentation — and re-index on that schedule rather than once, at launch, forever.

Retrieval is also the cheapest way to find out whether your content is any good. When the answers come back thin, the problem is usually not the model; it is that the documentation was written for people who already knew the answer. Fixing that helps your staff whether or not the agent survives the pilot.

Pattern 3 — the sidecar agent

The sidecar is a process that runs beside the old system rather than inside it. It watches for an event — a new document, a queue item, a status change — does the multi-step work, and writes the result back through the API layer, with a person approving anything that matters.

It is deliberately boring in the way it integrates: no code inside the old application, no new tables in its database, no shared deployment. Switch the sidecar off and the old system keeps working exactly as it did, with the process reverting to the manual path staff still remember. That property is what makes a sidecar approvable by the people who are accountable for the old system, and their sign-off is usually the real obstacle.

The human gate belongs in the sidecar, not in the old screens. Put the approval queue where the context is: the source document, what the agent proposed, why it proposed it, and one action to accept or correct. The corrections are the training data you will want three months from now, so capture them as data rather than as a decision somebody made and closed.

Choosing between them

The decision is short. If the new work has to read or change records in the old system, you need the API layer first and regardless — the other two patterns write through it. If people are asking questions your systems can already answer, add retrieval. If the work is multi-step and starts from an event rather than a request, add the sidecar.

Sequence them in that order too. The API layer without retrieval is still useful; retrieval without the layer is a read-only demo; a sidecar without either is a script nobody is supervising. When budget is tight, build the layer for exactly the operations the first use case needs and resist the urge to model the whole domain — the boundary earns its keep by being narrow enough to finish.

What to instrument before you start

None of this is provable later if you do not measure it now, and everyone agreeing it feels faster loses to a finance review every time. Before the first line of the integration, capture four things.

1. How long the process takes today, measured on real cases rather than remembered.

2. How often it is wrong today, and what a wrong one costs to put right.

3. Volume, per day and at the peak, because the peak is what people are actually complaining about.

4. Who touches it, and which step they would drop first if they were allowed to.

Then instrument the new path the same way and keep the two comparable: the same cases, the same definition of wrong, the same clock. Log every call across the API layer with a request identifier, so a disputed result can be reconstructed end to end instead of debated. The instrumentation costs a day or two, and it is the difference between a second phase that gets approved and one that gets argued about.

Legacy is rarely the obstacle people assume it is. The obstacle is usually the absence of a boundary: no documented way in, so every idea turns into a change to a system nobody wants to change. Build the boundary, index what you already hold, and put the new behaviour beside the old system rather than inside it. The rewrite, if it is ever genuinely needed, gets easier for exactly the same reasons.