July 2, 2026

Building an AI chatbot for small business that actually knows your products

The RAG stack we ship for small businesses that need a chatbot to answer questions about their actual products, not hallucinate a generic FAQ. Real code, real costs.

KEY TAKEAWAYS

↳Most 'AI chatbot' failures are not model failures, they are retrieval failures. Fix the index before you fix the prompt.
↳For a catalog under 5,000 products, pgvector on Postgres beats a dedicated vector DB on cost and ops overhead.
↳Chunk by product, not by page. One row per SKU with structured fields (price, stock, variants) plus a text blob.
↳Always return the source. If the bot can't cite a product page or doc, it shouldn't answer.
↳Budget around $30-80/month in infra plus model costs for a small business bot handling a few thousand messages.

A client came to us last month with a Shopify store, 800 SKUs, and a chatbot from one of those “train your AI on your website” SaaS tools. It confidently told a customer their dog harness came in size XXL. It does not come in XXL. The refund and the angry email cost more than a year of the SaaS subscription.

This is the whole problem with generic AI chatbots for small business. They scrape your site, dump it in a vector database, and call it a day. Then they hallucinate sizes, prices, return windows, and shipping cutoffs, because retrieval-augmented generation is only as good as the retrieval.

Here is the stack we actually ship for small businesses that need a bot to answer real questions about real products. It runs for about $30-80/month in infra plus whatever you spend on model calls.

The stack, in one screen

Postgres with the pgvector extension (Supabase or a $5 Hetzner box, both work)
OpenAI text-embedding-3-small for embeddings, or Voyage voyage-3-lite if you want cheaper
Claude Sonnet 4 or GPT-4o-mini for the actual chat response
A tiny Node/TypeScript API on Cloudflare Workers or a Hetzner VPS
Cloudflare Turnstile in front so bots don’t burn your token budget
Resend for the “escalate to a human” email path

That is the whole thing. No LangChain, no vector DB SaaS, no orchestration framework. I have shipped enough of these to know that every abstraction layer you add is a layer you will debug at 2am.

Why pgvector and not Pinecone or Weaviate

A small business catalog is usually 200 to 5,000 products. That is nothing. pgvector with an IVFFlat or HNSW index handles it in under 50ms on a $10 machine. You already need Postgres for orders, sessions, and logs. Adding a second database to sync is a whole class of bugs you don’t need.

The rule I use: if the client has fewer than 100k documents, pgvector. Above that, we start talking.

Chunk by product, not by page

This is where most tutorials get it wrong. They tell you to scrape the site, split by 500 tokens, and embed. For a product catalog, that is nonsense. You end up with half a product on one chunk and the return policy on another.

One row per product. Structured columns for the things that must be exact (price, stock, SKU, variants), and a text blob for the fuzzy stuff (description, materials, use cases).

create extension if not exists vector;

create table products (
  id text primary key,
  title text not null,
  url text not null,
  price_cents int not null,
  in_stock boolean not null,
  variants jsonb,
  body text not null,
  embedding vector(1536)
);

create index on products using hnsw (embedding vector_cosine_ops);

When you embed, embed a formatted string that includes the structured fields, because customers ask about them:

const textForEmbedding = `
${product.title}
Price: $${(product.price_cents / 100).toFixed(2)}
In stock: ${product.in_stock ? "yes" : "no"}
Variants: ${product.variants.map(v => v.name).join(", ")}

${product.body}
`.trim();

This is unglamorous and it works. When someone asks “do you have the blue one in medium”, the retrieval hits because the variant names are in the embedded text.

The retrieval query

const embedding = await openai.embeddings.create({
  model: "text-embedding-3-small",
  input: userMessage,
});

const { rows } = await sql`
  select id, title, url, price_cents, in_stock, variants, body,
         1 - (embedding <=> ${embedding.data[0].embedding}::vector) as score
  from products
  where 1 - (embedding <=> ${embedding.data[0].embedding}::vector) > 0.35
  order by embedding <=> ${embedding.data[0].embedding}::vector
  limit 6
`;

That 0.35 threshold matters. Without it, every query returns six products, even when the user asks “what is your return policy”. Then the model tries to answer with product data it should not use. Return zero rows and let the model say “I don’t have that info, want me to email the team?” That is the correct behavior.

Docs are a separate table

Return policy, shipping, hours, contact. Different table, same pattern. Query both, combine results, feed to the model. Do not mix them into one big documents table with a type column. You will want to weight them differently later and you will regret the merge.

The prompt is boring on purpose

const system = `You are the support assistant for ${business.name}.

Rules:
- Only answer using the CONTEXT below.
- If the context does not contain the answer, say so and offer to email the team.
- Always include the product URL when you mention a product.
- Never invent prices, stock, sizes, or policies.
- Keep responses under 4 sentences unless the user asks for more.

CONTEXT:
${formattedRows}

BUSINESS INFO:
${business.hours}, ${business.contact_email}`;

No chain-of-thought scaffolding, no role-play. The rules that matter are “only use context” and “always cite”. Everything else is decoration.

The best AI chatbot for small business is the one that knows when to shut up

I have watched clients get burned by bots that answer everything. A good customer service AI chatbot should refuse cleanly. When retrieval scores are low, we do not call the LLM at all. We reply with:

I don’t have that info handy. Want me to send this to the team? They usually reply within a few hours.

One button. Sends the transcript to the owner via Resend. Conversion on “escalate” is around 40% in our data across three clients (n is small, but the pattern holds). That is a lead you would have lost.

What it costs to run

For a store doing 2,000 chatbot conversations a month, averaging 4 turns each:

Embeddings (one-time reindex + queries): ~$2/month
Claude Sonnet 4 or GPT-4o-mini for responses: $15-40/month depending on model
Hetzner CX22 VPS with Postgres: 6 euro/month
Cloudflare Workers and Turnstile: free tier
Resend: free tier for that volume

Call it $30-60 all-in. The SaaS “AI chatbot for small business” tools charge $99-299/month for something that hallucinates your product catalog. Do the math.

The part nobody talks about

Reindexing. Products change. Prices change. Stock changes. If you embed once and forget, the bot will be wrong within a week.

We run a nightly cron that pulls from Shopify (or Stripe, or whatever the source of truth is), diffs against the products table, and re-embeds only the rows that changed. Costs pennies. Prevents the XXL harness incident.

const changed = await getChangedProducts(since);
for (const p of changed) {
  const emb = await embed(formatForEmbedding(p));
  await sql`update products set ... , embedding = ${emb} where id = ${p.id}`;
}

If you want us to build one of these for your business, or you have a half-finished bot that keeps making things up, get in touch. Or if you want to see what else we do with agents, our agents page has more.

FREQUENTLY ASKED

Common questions

▸How long does it take to set up an AI chatbot for a small business?

For a straightforward store with a clean product feed and a handful of policy docs, we usually ship a working bot in 5 to 10 business days. Most of that is not code, it is cleaning the client's product data and writing the fallback flows. If your product descriptions are two words each, the bot will be two words smart. Content quality drives everything downstream.

▸Can I use ChatGPT or Claude directly instead of building a RAG stack?

You can, and it will confidently make up your prices. The base models have no idea what is in your catalog. RAG is the piece that grounds the answer in your actual product data. If your business is simple enough that a static FAQ covers 90% of questions, skip the bot and use a good FAQ page. If customers ask about specific SKUs, stock, or variants, you need retrieval.

▸What's the best AI chatbot for small business if I don't want to build one?

Honest answer: I don't have a favorite. The off-the-shelf tools all have the same problem, which is that they scrape your site once and treat everything as equal-weight text. Intercom Fin works if you already pay for Intercom. Chatbase is fine for pure FAQ bots. For product catalogs, custom is almost always cheaper past month two and it actually knows your data.

▸How do I keep the bot from hallucinating product details?

Three things. First, embed structured fields (price, stock, variants) as part of the text so retrieval catches them. Second, set a minimum similarity threshold so the bot refuses to answer when confidence is low. Third, put a strict system prompt that says only use the provided context. And reindex when products change, otherwise the bot will confidently quote last month's price.

▸Does this work for a service business, not just ecommerce?

Yes, the pattern is the same. Instead of products you index services, pricing tiers, availability, and policy docs. A dental office might index procedure descriptions, insurance accepted, and appointment policies. A law firm indexes practice areas and intake criteria. The retrieval-plus-refusal pattern matters more for service businesses because the cost of a wrong answer (a bad lead, a missed conflict check) is higher.

▸What happens when the bot can't answer?

It should say so and offer a human handoff. We wire it to send the transcript plus contact info to the business owner via Resend, and optionally drop a row in a CRM or Notion. Do not let the bot make things up to seem helpful. A clean 'let me get someone on this' converts better than a confident wrong answer that ends in a refund.

SOURCES

[1]
pgvector: Open-source vector similarity search for Postgresgithub.com
[2]
OpenAI embeddings pricing and model referenceplatform.openai.com
[3]
Anthropic Claude API documentationdocs.anthropic.com
[4]
Cloudflare Turnstilecloudflare.com

6/2/2026