Start with the workload
Business buyers often begin with model brands when they should begin with a workload. A customer-support summariser, contract reviewer, coding assistant and visual quality inspector have different requirements. Define the input type, acceptable latency, expected output, error cost, data sensitivity and daily volume. Then create a test set of at least 50 representative examples, including ambiguous and adversarial cases. Score correctness, completeness, formatting and citation quality separately.
Frontier proprietary models from providers such as OpenAI, Anthropic and Google usually offer strong general reasoning, multimodal input and convenient managed APIs. Open-weight families from Meta, Mistral and others offer greater deployment control and can be economical at sustained high volume. Specialist models—for embeddings, reranking, speech, vision or document extraction—often outperform general models on narrow tasks at a lower unit cost.
How the main options differ
| Model route | Best for | Advantages | Trade-offs |
|---|---|---|---|
| Frontier API | Complex reasoning and fast pilots | High capability, low setup effort | Variable cost and vendor dependency |
| Smaller managed model | Classification, extraction and chat | Lower latency and price | Weaker on long, ambiguous tasks |
| Open-weight deployment | Private or high-volume workloads | Control, portability, customisation | Infrastructure and MLOps burden |
| Specialist model | Search, audio, vision or reranking | Focused performance | Requires multi-model architecture |
Price the complete transaction
API pricing is usually quoted per million input and output tokens, with output commonly costing more. Headline rates change frequently, so confirm current figures on the provider’s official pricing page before procurement. Do not compare token prices alone. Include retrieval queries, retries, tool calls, safety checks, observability, caching and human review. A cheap model that needs three attempts can cost more than a stronger model that succeeds once.
For a realistic estimate, record average input and output tokens during a pilot, multiply by monthly volume and add a failure allowance. Prompt caching can materially reduce repeated-context costs. Batch APIs may offer discounts when immediate responses are unnecessary. Self-hosting replaces token charges with GPU rental, engineering time, monitoring and capacity risk; it tends to make sense only when utilisation is predictable or data-control requirements justify it.
Context windows are not memory
A large context window lets a model receive more material in one request, but it does not guarantee accurate use of every detail. Long prompts can dilute instructions and increase cost. Retrieval-augmented generation is usually better: search an approved knowledge base, insert only relevant passages and require citations. Test retrieval recall independently from generation quality, because a model cannot faithfully answer from a document it never received.
Persistent customer or employee facts should live in governed databases, not an ever-growing conversation transcript. Store the minimum necessary information, apply access controls and give users a way to correct or remove it. For regulated work, log the model version, prompt template, retrieved sources, tool calls and final decision.
Privacy, security and deployment
Enterprise API terms may provide no-training commitments, regional processing, shorter retention and administrative controls, but these features vary by plan. Verify them contractually. Examine subprocessors, breach notification, encryption, single sign-on, audit logs and deletion procedures. Never assume a consumer chat product has the same controls as its enterprise API.
Open-weight models can run in a private cloud or on premises, reducing external data transfer. That does not automatically make them secure. Teams remain responsible for patching, model provenance, endpoint protection, abuse controls and output monitoring. Fine-tuning can improve style or repeated task behaviour, but it is not a dependable way to inject changing facts. Use retrieval for knowledge and fine-tuning for behaviour.
A durable selection process
- Define success: set quality, latency, cost and safety thresholds before testing.
- Build an evaluation set: use real, de-identified work with edge cases.
- Test multiple sizes: route simple tasks to smaller models and escalate difficult ones.
- Run a shadow pilot: compare AI outputs with existing decisions before automation.
- Monitor drift: rerun evaluations when providers update models or prompts change.
The winning architecture is often a portfolio rather than one universal model. A small classifier can route work, a specialist system can retrieve evidence and a frontier model can handle exceptional reasoning. This approach improves cost control and reduces dependence on a single provider. Keep model access behind an internal gateway so models can be replaced without rewriting every application.
Buy measured performance, not perceived intelligence. A controlled evaluation on your own work is worth more than a hundred generic benchmark points.