2 min read

The model is not the hard part

AI EngineeringProductionEnterprise

The model is not the hard part

I came into AI from industrial engineering, a four-year generalist degree that spends most of its time teaching you to look at a whole system before you change any part of it, and to notice where value leaks out of a process. That habit is the thing I actually kept, and it is why I tend to treat the model as one component among many rather than the centre of a project.

In practice the model is rarely the component that decides whether the work succeeds. A few things matter more.

The first is the handoff. Wherever the model's output goes next, someone is holding it, and if it is wrong they need a way to tell. An answer that is mostly right and lands on a person with no means of checking the rest is often worse than no answer at all.

The second is the data. Most of the effort in a deployment goes here, and none of it is visible in a demo. If retrieval is pulling the wrong documents, or the schema the agent reasons over is subtly incorrect, the strongest available model will produce confident and wrong results.

The third is trust. Whether the person receiving an answer believes it enough to act on it depends on how the system explains itself and how easily it can be corrected, not on which model generated the text.

Most of my work has been forward-deployed, which means sitting with the client and owning the build from the first prototype through to production. The pattern repeats across projects. The interesting questions are almost never about which model to use. They are about what the system replaces, what someone does with its output next, and how anyone knows when it is wrong. Get those right and a mid-tier model is usually enough. Get them wrong and no model recovers the situation.