Cost is driven by volume times length, not by cleverness
The intuition that a smarter answer costs more is mostly wrong. What actually drives AI chat
spend is the amount of text processed: how many conversations happen, how many turns each one
runs to, and how much context is sent along with every single turn.
That last item is the one teams underestimate. Each turn in a conversation typically re-sends
the history plus whatever reference material has been supplied, so a long conversation is not
linearly more expensive than a short one — the cost per turn grows as the conversation
grows. A twenty-turn chat can cost several times a five-turn chat rather than four times.
The choices that quietly multiply the bill
- Sending your entire knowledge base with every message. The most common and most expensive mistake. Retrieving a few relevant passages costs a fraction of attaching everything.
- Very long system instructions. These are sent on every turn, so a thousand words of policy is paid for once per message, not once per conversation.
- Letting conversations run indefinitely. Without a turn limit or a handoff trigger, a confused visitor and a confused assistant can loop expensively.
- Answering traffic that was never going to convert or resolve. Bots, scrapers and idle curiosity all cost the same per message as a real customer.
- Verbose answers by default. Output usually costs more per word than input. An assistant told to be concise is cheaper and, in chat, generally better.
None of these are quality decisions dressed as cost decisions. Shorter context, tighter scope
and a firm handoff usually improve the experience as well as the bill.
Model choice is a real lever, used carefully
Capable models cost more per word than small ones, and the temptation is to use the cheapest
that appears to work. The honest position is that this is a trade-off rather than a free saving:
a weaker model handles the easy questions perfectly well and degrades on exactly the awkward
edge cases where a wrong answer is most expensive to you.
A practical middle path is to keep the capable model for customer-facing answers and use
cheaper processing for the surrounding machinery — classification, routing, summarising a
transcript at the end. The customer-facing text is the part where quality is worth paying for,
and it is usually a minority of total processing.
Make the spend predictable
Unbounded usage-based pricing is uncomfortable for most businesses, and the discomfort is
about variance rather than the absolute number. A few controls make it manageable:
- Set a budget cap and decide in advance what happens when it is reached — fall back to a human queue, or to an offline form. Decide this deliberately rather than discovering it during a traffic spike.
- Cap turns per conversation, with a handoff at the limit rather than an abrupt stop.
- Scope where AI answers. Running it on your documentation and pre-sales pages while leaving account-specific questions to humans is both cheaper and safer.
- Watch for abuse. A public widget attracts automated traffic, and rate limiting protects the budget as well as the service.
- Review weekly at first. Cost per conversation stabilises quickly, but the first weeks after a change are when a bad assumption is cheapest to catch.
Compare against the real alternative
AI chat is worth judging against what it actually replaces, which is rarely nothing. The
comparison is against agent time for the questions it genuinely handles, plus the value of
conversations that would not have happened at all outside staffed hours.
Be honest in both directions. Deflected questions that a person would otherwise have answered
are a real saving. Conversations that only exist because an assistant was available at midnight
are new value, but they are not a cost reduction and should not be counted as one. And a wrong
answer given confidently has a cost that never appears on an invoice — the follow-up
contact, the lost sale, occasionally a complaint — which is the main argument for scoping
carefully rather than cheaply.
How MyLiveChat fits
MyLiveChat meters AI usage per site so the spend is visible rather than inferred, and a budget
cap can be applied to a site and takes effect on conversations that are already running, not
only on new ones. You can also supply your own provider key instead of using the managed
allowance, which suits teams that already have a provider relationship and want the usage on
their own account. Because handoff to a human is built into the same widget, hitting a limit
degrades into a staffed conversation rather than a dead end.
What to measure
Cost per conversation is the headline number, but it is most useful broken down by
conversation type — a pre-sales question and a troubleshooting session have very different
economics. Track average turns per conversation too, since it is usually the fastest-moving
driver and the easiest to influence.
Then hold cost against outcomes rather than in isolation: resolution rate, handoff rate, and
satisfaction on AI-handled chats. A falling cost per conversation alongside a rising handoff rate
is not a saving, it is a quality problem that has moved the work to your agents. The pairing is
what makes either number meaningful.
Model choice is the lever this guide treats most carefully, and the screen where you exercise it is easy to misread. Why no model is marked active on your AI models page explains what the active badge is testing, and why the usage table underneath it is the better evidence of what you are actually paying for.
The budget cap in that list is not hypothetical here: your account carries three of them, and they count different things over different windows. What your AI spend caps do when you reach them covers what each one measures and what reaching one currently does.
On the managed side specifically, top-ups are bought as fixed packs rather than metered — what happens after you buy an AI reply pack explains how one is credited and how long it lasts.