What is AI software development, really?
Published: Oct 18, 2026 · Last updated: Oct 18, 2026
AI software development is building a product where a model, usually a large language model, makes some of the decisions that ordinary code used to make. The code around it looks familiar, but the model's answers vary from run to run, so you test it, budget for it and contract for it differently.
Maybe you want a support assistant that answers from your help docs, a tool that reads supplier invoices and fills your ERP, or a feature that drafts weekly reports for your managers. A demo worked in an afternoon. Now you are trying to work out what it takes to turn that into something customers and staff rely on every day.
That gap is the whole subject of this article.
Most pages about AI software development list tools or sell 'AI-powered' services. This one covers what changes in the build itself: how quality gets measured, where the effort and the running cost come from, and which risks you should write into the agreement before work starts.
Why does software with a model inside behave differently?
Normal code is deterministic: the same input gives the same output, every time. A model is probabilistic. It predicts likely next words, and small changes in the input, the instructions or the model version can change the answer.
Three things cause most of the variation:
Sampling. Models pick from several likely words with some randomness. You can turn that down, but even at the lowest setting the same request can come back worded differently.
Context. The answer depends on everything sent along with the question: the system prompt, the retrieved documents, the chat history. Change one paragraph of instructions and behaviour shifts in places you did not touch.
Model updates. Providers update and retire models on their own schedule. A newer version can be better on average and worse on the exact cases your business depends on.
This is why a feature that looked perfect in the demo can fail in its first week. The demo used five friendly inputs. Production sends thousands of messy ones: half-filled forms, blurry scans, a customer writing in two languages, someone deliberately trying to trick it.
The model also has no sense of when it is wrong. It will state a made-up refund policy with the same confidence as the real one. That is hallucination, and it comes from how these models work, not from a bug someone forgot to fix. The engineering job is to reduce it, detect it and limit the damage when it slips through.
What does it cost you to build it like normal software?
If an AI feature is built and tested like a normal feature, the failures show up after launch, in front of your users. You pay in rework, in lost trust and sometimes in exposure you never agreed to carry.
Here is how it usually plays out:
The team ships after a handful of manual checks. Users find wrong answers the team never saw.
Every prompt fix quietly breaks something that used to work, because nobody kept the old cases to re-check.
Running cost was never modelled. Usage grows, a long set of instructions travels with every request, and the monthly model bill becomes a board conversation.
Nobody owns quality after launch. The provider updates the model, answers drift and you hear about it from a complaint.
The hardest cost to see is trust. A user who catches the assistant inventing a policy does not file a bug report. They stop using the feature, and often start doubting the rest of your product too.
There is a security side as well. If the model can read customer data or take actions in other systems, a cleverly worded message or document can turn it against you. That needs a design answer and a contract answer, not a patch after the incident.
None of this means AI is a bad bet. It means the build has to be planned around variation from the first sprint, and that is a planning problem you can solve before any code is written.
Do you need a development partner for this at all?
Not always. If the AI part is a convenience your own team uses, with a person reading every output before it matters, you can often do it yourself with an off-the-shelf tool.
You probably do not need outside help if all of these are true:
A person reviews the output before anyone acts on it.
No customer data or confidential documents go to the model.
A wrong answer costs a few minutes, not a refund, a compliance problem or a lost customer.
A ready-made product already does most of the job: a chat assistant with your documents uploaded, a writing tool, or an automation platform with an AI step.
In that case, try the off-the-shelf route first and watch what your people actually ask it to do. That learning makes any later custom build smaller and sharper.
You need engineering help when the output reaches customers directly, touches money, health or legal data, triggers actions in other systems, or has to run thousands of times a day at a cost you can predict.
How do you test something that gives a different answer each time?
You replace pass/fail tests with an evaluation set: a fixed list of real inputs, each paired with what a good answer looks like, scored automatically every time the prompt, the data or the model changes. Quality becomes a score you track over time, not a box you tick once.
A useful evaluation set has four kinds of cases, each there for a reason:
Common cases. The 50 to 200 questions or documents that make up most real traffic. Pull them from support tickets, inboxes and logs, because invented examples are always easier than real ones.
Hard cases. Ambiguous questions, poor scans, mixed languages, very long documents. These are where users lose patience first.
Must-refuse cases. Requests the system should decline or pass to a person, such as legal advice or anything outside your policies. These protect you from confident answers you never wanted given.
Attack cases. Inputs that try to override the instructions or pull out data the user should not see. These prove the guardrails work, not just that they exist.
Scoring depends on the task. Extraction can be checked exactly: did it pull the right invoice number and total? Classification gets an accuracy figure per category. Free text is scored against a written rubric, either by a person or by a second model acting as a grader, which you check against human judgement on a sample before trusting it.
Ordinary tests still matter. The code around the model, its permissions, its API calls and its data handling, gets unit and integration tests like any other software. The evaluation set sits on top and runs in the release pipeline, so a prompt edit that drops the score cannot ship quietly.
One question tells you a lot about any vendor: show me the evaluation set and the score from the last release. If there is no answer, there is no quality process.
What drives the effort and the running cost of an AI product?
Three things drive most of it: model calls, data preparation and monitoring. The screens and business logic matter too, but those behave like any software project. These three are where AI projects run over.
Model calls. Providers charge per token, the small chunks of text sent in and returned. Cost grows with the length of your instructions, how many documents you attach to each request, how many steps an agent takes before answering and which model tier you choose. Design choices move this a lot: caching instructions that repeat, sending easy requests to a smaller model and trimming the context to what the question needs.
Data preparation. This is usually the largest effort and the most underestimated. If the model answers from your documents, someone has to gather them, remove outdated versions, split them sensibly, tag who may see what and keep everything in sync as it changes. A retrieval pipeline is only as good as the content it searches.
Monitoring. After launch you need a log of inputs and outputs, a sample reviewed on a schedule, alerts when scores or costs move, and a simple way for users to flag bad answers. This is ongoing work that never really finishes. Our guide to monitoring AI agents in production goes into the setup.
Two quick rules. If your documents are messy or scattered across tools, expect data work to take longer than the model work. If volume will be high, ask for a cost-per-request estimate before the build and measure the real figure during the pilot.
Which AI risks belong in your contract?
Hallucination, prompt injection, data handling, vendor lock-in and ownership should all be named in the agreement, with a clear owner for each. Standard software contracts were written for code that behaves the same way every time, and they say nothing about most of this.
What to ask for, and why:
Hallucination. An agreed evaluation set and a minimum score as the acceptance test, plus defined behaviour when the system is unsure: a fallback answer, a citation to the source or a handoff to a person. Without a measured standard, 'it works' is just an opinion.
Prompt injection. A written description of what the model can read and do, limits on the actions it can take, and attack cases in the evaluation set. Any text the model reads, including a customer email or a web page, can carry hidden instructions. Our piece on AI agent prompt injection shows how this works.
Data handling. Which provider receives your data, in which region, whether it may be used for training and how long it is kept. Your privacy promises to your own customers depend on these answers.
Vendor lock-in. Prompts, evaluation sets and model calls kept in your codebase behind a thin layer, so the provider can be swapped. Prices, models and terms change, and you want the option to move without a rebuild.
Ownership. You own the code, the prompts, the evaluation data and any fine-tuned model. These are the real assets of an AI product, so they should never sit in a vendor's account.
With Geminate Solutions, clients own 100% of the code and IP, and we sign an NDA before we talk about your product.
The build order that keeps an AI product reliable
Start narrow, measure early and widen the scope only when the numbers support it. This order works for most products:
1. Write down the job. One task, who uses it, what a good result looks like and what a wrong one costs.
2. Collect real examples. Build the first evaluation set before any code. If you cannot find 50 real cases, the use case may not be ready yet.
3. Prototype against the set. Try two or three models and prompt approaches, then choose on score, speed and cost per request.
4. Build the guardrails. Permissions, fallbacks, human handoff and logging.
5. Pilot with real users. A limited group, every output logged, a review every week.
6. Launch with monitoring on. Alerts on score, cost and complaints, and a re-run of the evaluation set whenever the provider changes the model.
Two decision rules help along the way. If the pilot score is good but cost per request is too high, try a smaller model or a shorter context before rebuilding anything. If the score is stuck after several prompt changes, the problem is usually the data or the task definition, not the wording of the prompt.
Accept one trade-off early: a narrow first version that works reliably beats a broad one that only impresses in demos. Our article on moving an AI pilot to production covers the step where most projects stall.
If you have an AI feature planned, prototyped or half built, send it to us. With 50+ products shipped, Geminate Solutions will review your idea or app and reply within 24 hours with where the quality, cost and contract risks sit and which evaluation cases to write first. You can also see how we handle AI integration for existing products.
