A WhatsApp sales bot with no RAG: the whole knowledge base rides in a cached prompt

Context

Divo is an Indian car dashcam brand that sells through its own store↗: five camera models in three series, plus accessories such as a hardwire kit, memory cards, mounts and cables. Customers ask pre-sales and support questions on WhatsApp. The owner asked me for a bot to handle them, so that he would not have to answer each customer himself. It is a freelance project and I worked with him directly.

The bot has been live on Divo's main WhatsApp number since 30 September 2026. Divo moved that number off the WhatsApp Business app and onto Meta's Cloud API for the bot, and its human support moved to a new line. Getting there was mostly Meta's business setup rather than code, and I wrote that part up separately in Putting your bot on a real WhatsApp number. I first wrote this page on 17 September, before launch, and updated it on 1 October, the day after.

Real conversations are coming in, but very few. Langfuse counts four different people since 26 September, and I can't yet tell which of them were customers rather than tests. So there is still no conversion figure and no hours saved. There is no "before" to compare against either: the owner was not working through enquiries by hand at a volume anyone recorded, which is the reason he wanted a bot. What I can report is engineering, and now some measured costs: what is cached, what a reply costs, how large the prompt is, and how the bot holds up when I try to break it.

The job changed twice. It began as a support bot. On 15 September the client's rules document turned it into a named sales and support advisor: answer the question, recommend a camera, close. A second document of 20 rules on conversational tone followed. Then on 1 October, the first full day live, I reworked how it talks, which has its own section below.

The constraints were these. Cost per message was the stated driver, so the model is the cheapest tier of its family, gpt-5.6-luna, and prompt caching is the main lever. Replies must not drift from the client's approved facts. No price, discount or stock status may appear in the knowledge base or in a reply, because those change while the knowledge base is static; the bot links the product page instead. Replies are in English, even when the customer writes Hindi or Hinglish. And it is one service on one instance, because sessions live in process memory.

I built it on my own over September 2026, working with an AI coding assistant (Claude Code). The knowledge base is 12 Markdown files, about 42,500 characters in all. I wrote it from the brand's four product manuals and a hand-picked subset of its website.

Approach

It is one Express service, and most of it exists to avoid or police a model call. Figure 1 is the path a message takes.

The webhook verifies Meta's HMAC signature over the raw body and returns 200 straight away; each message is then processed on its own. In production the app refuses to boot without the app secret. A set of message ids with a 10-minute lifetime absorbs Meta's redeliveries. Text messages from one customer are held for 1.5 seconds and merged into a single model call, including messages that arrive while an earlier answer is still being generated, and the merged text is cut at 2,000 characters. Work for each customer runs through its own queue, so replies cannot overtake each other.

Then come the exits. A photo, voice note or sticker gets a canned reply. A whole message of 40 characters or fewer that exactly matches one of 46 greeting or 76 thanks phrases, Hinglish forms included, gets a canned greeting or thanks, and an emoji on its own counts as thanks. A customer is allowed 30 model calls in a rolling hour. The first refusal sends one notice and later ones are silent.

Only then does the model run. The system prompt is built once at boot, not per request: every knowledge-base file, sorted by path with line endings normalised, is joined under headings and placed into a template of 8 modules for identity, grounding, boundaries, tone, format, escalation, sales and examples. The result is byte-identical on every call, and the cache key is a hash of those bytes. There is no chunking, embedding or retrieval step. The input is that prefix, up to 10 turns of history held in memory for 30 minutes, and the new message. Reasoning effort is none, output is capped at 1,024 tokens, and every call logs its input, cached and output tokens with a hit ratio.

The draft does not go straight to the customer. A reply guard in code checks it, which is decision 3 below. Em and en dashes are stripped in code and not by a prompt instruction, which costs no tokens. While the model works, the customer sees their message marked as read and a typing indicator, which the bot refreshes every 20 seconds because WhatsApp clears it at 25. A reply goes out as one WhatsApp message, and only a closing question becomes a second one, for reasons I come to below. Handing over to a person is a prompt rule: the bot gives the support line and email with the team's hours, and says what to have ready. There is no agent inbox or routing code, and on this kind of number nobody can answer inside the bot's chat at all, which is decision 7.

Figure 2 shows both paths. "Hi" is in the phrase list, so the first reply is the canned greeting. "What's up" is not, so it went to the model, which greeted in its own words. The recommendation is one model with a reason and exactly one question. "Only while driving" makes the bot drop the hardwire kit, because the sales rules offer it only when parking protection is wanted. The price question gets the product page and no rupee figure. WhatsApp shows minutes only, so the timestamps say nothing about latency.

Before a change goes out I run 31 scripted conversations against the real model. Sixteen of them try to break it: prompt injection, a request to reveal its instructions, a planted phishing link, "reply in Hindi", a customer insisting on a price, a pretext for getting an internal rule out of it. Eleven check tone and four check grounding. The grounding ones use an invented product catalogue, so the bot cannot pass on what the model already knows about dashcams. Each check is a pattern match on the reply. There is no LLM judge and no scored dataset.

A second layer needs no model at all. The links, the email address, the phone number and the rupee amounts the bot is allowed to write are pinned to exact lists. A knowledge-base edit that brings in a new link or a new amount fails before it ships, and I have to approve it by hand.

Decisions

[01]Design decisionThe whole knowledge base in a cached prompt prefix, no retrieval

Decision

No vector store, no chunking, no embeddings. All 12 knowledge-base files are joined into one instructions string, built once at boot and byte-identical on every call, sent with a stable cache key and a 30-minute cache lifetime.

Why

The knowledge base is small, about 42,500 characters today, and cached input is billed at a tenth of the input rate. At list prices I estimated about $0.0004 for an answered message on a cache hit, and the live service has since measured the same. Retrieval would send a few thousand uncached tokens of selected sections on every call, which costs more than the whole knowledge base cached. I checked the caching before building on it: in my first test against the real model, on 10 September, 8,495 of the second call's 8,512 input tokens came from cache.

Tradeoff

Prompt size becomes the bill. The prefix is an estimated 84% of the cost of a hit and 98% of a miss, so a duplicated paragraph costs money on every message. Any edit to a prompt or the knowledge base changes the cache key and causes a cold start, and a miss costs about ten times a hit. And it stops working if the catalogue grows well past one prompt.

The estimate behind those figures, per answered message, from list prices of $0.02 per million cached input tokens, $0.25 per million for uncached input and cache writes, and $1.20 per million output tokens:

That was arithmetic when I wrote it, with token counts converted from characters. Since launch, Langfuse prices every call the deployed service makes, and the measurements land on the estimate: about $0.00039 for a cached reply and $0.00367 for a cold one on the September prompt. The cached prefix averaged 14,648 tokens against my conversion of about 15,000. The Results section has the numbers for the new prompt and how small the sample is.

I had built retrieval for VeriStream AI a few months earlier, so leaving it out here was a choice. A vector store, an agent framework and a Redis session store were all considered at design time and rejected as too much for one brand and a small knowledge base. I never built or measured the alternative. There is only the price argument. The provider was chosen the same way: OpenAI over Anthropic, on cost, before I wrote any code, with no side-by-side test. Streaming was rejected for a different reason, which is that a WhatsApp message cannot be edited once it is sent.

[02]Design decisionPrompt size is a budget, and a failing test enforces it

Decision

A unit test caps the compiled prompt at an exact character count, and I lower the cap each time content is trimmed. A new rule has to be paid for with a cut somewhere else. The knowledge base follows one rule: one home per fact, and the manuals win when they disagree with the website.

Why

The trigger was money. The OpenAI bill rose after I scraped the brand's website into the knowledge base on 11 September. I have no bill figures to show, only the cause: after the scrape, the knowledge base stated the warranty four times and carried one paragraph eight times. The first consolidation cut it from 113k to 67k characters while keeping every customer-facing answer, and the audit also turned up a wrong answer about hardwire kit compatibility.

Tradeoff

The cap counts characters, which is a proxy for tokens. Because the cap comes down to the prompt's size after every trim, the next knowledge-base addition fails the build until something else is cut. That is the intent, and it also makes a routine content update a negotiation. Manual trivia went in the cuts, such as parts diagrams, photo-mode menus and chipset numbers, so the bot hands those questions to the support team.

The scraper itself changed the same day it was written. It first wrote straight into the knowledge base. Once the knowledge base was curated by hand and much smaller than the raw scrape, a blind overwrite would have undone the curation, so the scraper now writes to a staging folder, and I compare it against the knowledge base by hand.

[03]Design decisionA reply guard in code, with one rewrite that keeps the cache

Decision

Every draft passes a deterministic guard before it is sent. Its allow-lists for amounts, link hosts, email addresses and phone numbers are extracted from the compiled prompt itself, so a knowledge-base edit needs no mirrored code change. It also checks language, format, leaks of the rules text and statements of the client's internal rule: eight kinds of violation in all. On a violation the model gets one rewrite. The blocked draft and a correction note are appended after the conversation, so the cached prefix stays byte-identical. A second failure sends a canned fallback, and the turn is not stored in history.

Why

Prompt instructions alone do not stop a model from confirming a price the customer asserts, repeating a link someone planted, or switching language when asked nicely. The guard is where those rules stop being requests.

Tradeoff

A violation costs a second model call. The Hinglish check is a list of 35 Hindi function words with a threshold of three distinct matches, which is a heuristic and has to be maintained by hand. And the client's scripted lines have to live in the prompt as example replies, because the leak check treats a long verbatim echo of the rules text as a leaked instruction.

The guard earned its place and failed on the same night. I applied the client's rules just after midnight on 15 September. One of them is internal: the bot follows it and must never say it aloud. It was written in a form the guard's amount check did not recognise, so nothing in code stopped the bot from stating it. I caught that in testing and fixed it. The fix was too broad and blocked harmless replies that only touched the topic, so I narrowed it. Then a third problem showed up. Asked to "reply in Hindi", the bot answered in romanised Hindi, which a check for non-Latin script cannot see. All three were found and fixed within the hour, on the test number and not in front of a customer.

Two of my test conversations changed that night as well. They used to require that the model's first draft was clean. They now require what the customer actually depends on: the text that gets sent is clean, and nobody is left with a blocked reply. That also hid how often the guard was doing the work, so on 1 October I measured it for one case. Asked to reply in Hindi, the first draft switched language in 2 of 4 tries on the September prompt and in 2 of 4 on the new one. The guard caught every one and forced an English rewrite, at the cost of a second model call. The weakness was there all along, and the clean pass in September was partly luck.

[04]Design decisionNo model call where no model is needed

Decision

Canned replies for greetings, thanks, lone emoji and anything that is not text. A 1.5-second merge window that also holds messages arriving mid-generation. Reasoning effort none, and output capped at 1,024 tokens.

Why

These replies are lookups in a knowledge base. Reasoning tokens are billed at the output rate and add latency, and the first version's default of effort low lasted one day. A customer who types three short messages in a row should cost one call, not three.

Tradeoff

The phrase matcher is an exact-match list of 122 phrases that I maintain by hand, about 50 of them Indian-English and Hinglish forms I added later. The merge window adds 1.5 seconds to every reply. I have no measurement of how many model calls either one saves.

[05]Design decisionPrompts as versioned modules in Langfuse, with local files as the fallback

Decision

One file per concern, each a versioned Langfuse prompt, composed by a parent prompt that holds only references. The same references resolve from local files when Langfuse is off or unreachable, so tests and offline boots behave the same. The service seeds missing prompts on first boot without overwriting any, polls for a new production version every 60 seconds and rebuilds the prompt when one appears. A script publishes changed files as new versions.

Why

A prompt change ships without a redeploy, and every generation in a trace is tied to the prompt version that produced it.

Tradeoff

Two sources of truth that can drift apart. The publish script is meant to skip prompts that someone edited in the Langfuse UI, and for four days that check was wrong: in my normal order of work every changed module looked UI-edited, so it was skipped while the script reported nothing to publish. Every published change also resets the OpenAI cache key, so a prompt tweak is a cold start.

Tracing had its own bill. An answered message was about 12 Langfuse units by my count during a cost check on 15 September: 9 spans, plus about 3 traced status webhooks that Meta sends for every customer message. Most of those spans carried counts, not timing worth a span of its own. I cut the tree to 3 units: the message trace, the answer span and the generation, with the bookkeeping moved into metadata. The webhook request itself is no longer traced. By that arithmetic the free tier's 50,000 units a month covers roughly 15,000 customer messages. That is a capacity calculation, not traffic.

The first day live

On 1 October I read the bot's replies the way a customer would, and I didn't like them. The facts were right, but every reply was a block of text with no human feel: too technical, no selling, nothing that connected with the person asking. I also asked myself whether about $0.0004 a reply was a lot. It isn't. That is about ₹0.035, and 10,000 replies a month would cost about $4. The only lever left is the cold call, and there was too little traffic to tell whether keeping the cache warm would pay. So cost became a rule for the rework rather than its goal: the new voice had to leave the prompt smaller, not bigger.

I rebuilt the prompts on Google's conversation design guidance↗. Its first step is a persona, so a list of prohibitions became a character: a friendly expert at a good car-accessories shop, warm, confident and straight-talking. Of three levels of selling I considered, I chose a confident advisor. It recommends the complete setup, gives the customer something useful before offering the cheaper model, and closes with the product page and a reason it's an easy buy. One exclamation mark is now allowed; replies stay in English, with no emoji. A second pass went through Google's pages on acknowledgements, informational statements, discourse markers and questions, and turned them into habits. The customer's car and situation come first, the benefit comes before the feature, specs are given in plain words and never as a list, linking words replace bullets, acknowledgements vary, and nothing an earlier message already said gets repeated.

The same questions, before and after, from two of my own test conversations on 1 October. I've left out the support line's number and address.

Says hello

Before. "Hi, I'm Shyam from Divö. Tell me which car you drive or what's going on with your dashcam and I'll help you sort it out here."

After. "Hi, I'm Shyam from Divö. Are you looking for a dashcam for your car, or need a hand with one you already have?"

Drives a Creta

Before. "For your Hyundai Creta, I'd recommend the D30R. Most Creta owners we speak to choose it with 24-hour parking mode, and it gives you 4K front plus rear coverage with GPS. Would you like it to record only while driving, or also monitor the car while parked?"

After. "Got it. For a Creta, most owners we speak to go for the D30R. It records in 4K ahead and HD behind, with GPS, so you've got clearer evidence in traffic or if someone scrapes the car." Then, as a second message: "Should it also keep watch while the Creta is parked?"

Only while driving

Before. "Got it. For driving-only use, the D30R is a good fit for your Creta, and you won't need the Hardwire Kit. You'll need a high-endurance memory card too, ideally 128GB, or 256GB if you drive heavily."

After. "Makes sense. For driving only, I'd recommend the D30R with a 256GB high-endurance memory card, so you get front and rear coverage without needing the Hardwire Kit. It's especially useful if someone hits you from behind or disputes what happened on the road." Then: "You can check today's price and order here: https://divoindia.com/products/d30r↗. Want help with the fitting?"

Wants a real person

Before. "Of course, the Divö team reads this same chat and can take over. They're available Monday to Saturday, 9am to 6pm; I'll flag this for the team here."

After. "Of course, our team can help you directly. Call [support line] or email [support address], Monday to Saturday, 9am to 6pm. Keep your order number handy if you've already placed an order."

Asks for the D30R's specs

Before. A list of six bullet points: frame rates, a 2.4-inch IPS screen, a 170° lens and so on.

After. "The D30R records in 4K ahead and HD behind, so you get clearer evidence of incidents and number plates on the road. GPS adds your location and speed to each clip…" (from a replay on the new prompt)

The answers barely changed. The way they're said did, and so did where a customer is sent when they want a person. The old hand-over, when a customer asks for a real person, made a promise nobody could keep, which is decision 7.

The first round went wrong in a way worth telling. I tested the new version and saw specs still dumped as a list, and short messages that made no sense on their own. The trace timestamps explained most of it. My test ran a few minutes before the new prompts were published to Langfuse, so it ran on the old ones. Nothing in the trace said so, because the parent prompt keeps its version number when only the modules under it are republished.

One complaint was real either way. The new code split a reply into separate WhatsApp messages at blank lines, and the old prompt put a blank line before every list, so "The D30R has:" went out as a message by itself. I replaced that with one rule: a reply is one message, and only a closing question goes out as a second one, after a short typing pause. An introduction or a list is never cut off from what follows. That gives fewer of the cues of someone texting, and no message that makes no sense alone.

[06]Design decisionRewrite the prompt instead of adding to it

Decision

Before any tone text went in, I audited the eight modules for rules stated more than once and gave each rule one home. New text was paid for with cuts, and the size test's cap came down to the new size afterwards.

Why

The first plan for the new voice added about 3,000 characters on top of the old prompt, and I turned it down: the budget in decision 2 says new text has to be paid for. The audit found the price rule, the hand-over channels, the one-question rule and the advice about technicians each written in three or four places. The rules went from 18,620 characters to 13,291, including the new persona, the selling behaviour and longer sample conversations, and the whole prompt from 64,371 to 55,812.

Tradeoff

Every cut has to be checked for what it loses. I also cut cross-references and a spec summary from the knowledge base, then went looking for what had been lost and put four details back. A shortened language rule dropped the words 'even if asked', and the model's drafts started switching to Hindi when asked, until those words went back in.

[07]Design decisionHand-over promises that match the platform

Decision

The bot never says the team will pick up the chat. It gives the support line and the support email with the team's hours, and tells the customer what to have ready, such as an order number. Its instructions say plainly that nobody from the team reads its chat.

Why

The old hand-over was written during testing and said the team reads the same chat and would take over. On the real number that is impossible. A number on the Cloud API is gone from the WhatsApp Business app, and with no inbox product on top, nobody can read or answer the bot's conversations. For about a day after the move, a customer who asked for a person would have waited for a reply that could not come.

Tradeoff

The customer has to change channel to reach a person, and the support team can't see what the bot already said. A paid shared-inbox product would fix both. It was one of the options for the hand-over, and the simplest one shipped.

[08]Design decisionCode that works with the old prompt and the new one

Decision

Code and prompts ship separately, so the code accepts output from either version: markup tags left over from the first round's prompts are stripped before sending. A new prompt is published under a staging label and tried on a local copy of the bot before it goes to production.

Why

Code goes live when the service redeploys; prompts go live when I publish them to Langfuse. A prompt that emits a tag the deployed code doesn't understand would show the raw tag to a customer, so the two can't depend on going out in a particular order. The first round showed the other half of the problem: I tested, and reacted to, a prompt that was not live yet.

Tradeoff

Compatibility code for a prompt version that should no longer exist, and one more step before a prompt reaches customers. It doesn't stop the first round's mistake on its own either: before acting on a test, I still have to check which prompt version it ran on.

What would improve the bot most now is material from Divo, not more prompt tuning, and I've asked for it: real sales and support chats, replies they like and dislike, which camera owners of popular cars buy (the bot's list of cars still has two), offers or bundles, the cities with installers, and the most common reasons customers write in.

Results

None of these is a business result, and each needs its qualifier.

The costs are measured now, not estimated. Langfuse prices each model call the deployed service makes. Between 26 September and 1 October a cached reply cost $0.00039 on average across 24 replies on the September prompt, and $0.00038 across 8 on the new one. Cold calls cost $0.00367 (4 of them) and $0.00322 (1). The rework cut the cached prefix from 14,648 tokens to 13,078 on average, 11% less, but the new voice writes a little more, 83 output tokens against 69, so a cached reply costs about the same as before and the saving shows on a cold call. The cache served 24 of 28 calls on the old prompt and 8 of 9 on the new one. That sample is small and mostly my own testing, so it is a real price for real calls, not a production average or a production hit rate. The detail from the very first cache check still holds: the invented catalogue I use for grounding is below OpenAI's 1,024-token minimum for caching, so caching can only be checked on the real knowledge base.

The 29 of 31 is a single pass of a non-deterministic model, run before my last fixes to the checks. One miss was a valid refusal of an off-topic question, worded differently from what my pattern expected, so I widened the pattern. The other was the "reply in Hindi" case from decision 3, where the first draft switches language about half the time and the guard rewrites it. In September all 27 conversations I had then passed in one run, and an earlier set of 20 passed on 14 September. I did not keep the output of those runs.

The September run gives the only timing data I have. Over the 25 conversations that made one model call, the round trip had a median of 3.6 seconds and ranged from 1.3 to 5.7. The two that made two calls took 8.3 and 8.7 seconds. That was measured from my machine in India, on the larger September prompt. It is not WhatsApp latency: the live path adds the 1.5-second merge window and the send, and any reply may include a silent guard rewrite.

On 15 September I looked at the Langfuse project, saw 50 generations all tagged development, and concluded that production had never sent a trace. That was wrong. The deployed service had been tracing all along, each trace tagged with the version it ran, and they were labelled development only because one environment variable was never set on the host. On 1 October those traces showed conversations on the live number: greetings, recommendations, installer lookups for Pune and Mumbai, and hand-overs to the support line.

Looking back

Tests passing is not tone passing. On the first round of the rework every offline check and almost every scripted conversation passed, and the replies still did not sound like a person. What moved it forward was replaying real conversations and reading them. A pattern match can enforce a rule; it can't judge warmth.

I never save the results of a test run. Each one is pass or fail on my screen and then it is gone. There is no dataset, no history and no way to show whether the bot is getting better or worse. Moving two checks from "first draft is clean" to "final text is clean" was reasonable, and it also hid how often the guard was doing the work until I measured one case by hand.

Guard bugs arrived with each change to the rules. Every new behavioural rule needed a matching guard and a matching test conversation, and I found that coupling by accident, late at night, one bug at a time. A checklist for rules changes would have caught all three.

The publish tool was broken for four days and I did not notice, because I had never tried it in the order I actually work in. Nothing runs the checks automatically either; they run when I remember to run them. Code and prompts reach customers by two different routes, and on 1 October I tested between the two and acted on what I saw. Two release routes need one checklist. And the first version's fallback message pointed customers at a support page that does not exist.

Configuration needs checking as much as code. One missing environment variable made production look untraced for two weeks, and I believed it.

All state is process memory: sessions, the dedupe set, rate limits and merge windows. The service redeploys on every code change, so a deploy drops conversation context in the middle of a chat, and now those chats are real. There is also a privacy gap between logs and traces. Logs mask phone numbers to the last four digits, but the customer's full number is the Langfuse user id and the full prompt is stored with every generation. That was designed for a self-hosted Langfuse, and the documented production option is Langfuse Cloud.

Smaller things. Langfuse's cost column used to under-report a cold call, showing about $0.00003 where the arithmetic said about $0.004, because its model definition and the OpenAI wrapper disagreed on a usage key. It now prices cache writes, and the cold-call costs above come from it. The bot can send product images, and that feature is finished and switched off, because there are no images to send yet. And the client's two rules documents are kept apart from the code on purpose, which also means I have no record of which rule drove which prompt change.

What I would change. Set the environment label on the host so production traffic can be counted on its own, then record the first real weeks: customers, cache hits, cold starts, latency and hand-overs. Save the output of every test run, with token usage, so a pass rate can exist. Write the rules-change checklist, so each new rule gets a guard, a test conversation and a line in the budget, and one release checklist that covers both code and prompts. Count the cap in tokens. Hash the phone number before it becomes a user id. Move sessions out of process memory, or at least stop deploying on every change. Run the checks automatically. After a few weeks live, one sentence from the owner about his time would be worth more than any number on this page.