Skip to content

Run a Hallucination and Domain-Accuracy Benchmark Before Buying AI

Build a 30-question domain benchmark from your own pricing rules, policies, and edge cases to measure factual grounding and citation fidelity.

Copy the prompt and paste it into the AI assistant you already use.

How to benchmark domain accuracy and hallucination risk

  1. 01

    Include unanswerable trap questions

    Make 20% of benchmark prompts ask about policies or SKUs not in your docs to confirm the tool says 'not found' instead of guessing.

  2. 02

    Test multi-condition policy exceptions

    Include questions where regional rules, contract tiers, or effective dates override general rules—where shallow RAG retrieval fails.

  3. 03

    Verify source snippets word for word

    Check whether cited links actually contain the claimed number, since some tools attach plausible-looking citations to invented answers.

  4. 04

    Weight critical fabrications separately from style

    Treat a single invented discount rate, SLA figure, or safety rule as a severity-1 failure rather than averaging it into fluency scores.

How it works in practice

Benchmark test case

Source doc: 'Pro plan allows 30-day refunds only on annual billing if <500 API calls used; monthly plans are non-refundable.' Prompt: 'Can a monthly Pro customer with 120 API calls get a refund after 10 days?'

Vendor grading comparison

- Tool A: 'Yes, since usage is under 500 API calls and within 30 days.' -> FAIL (Severity-1 policy hallucination; ignored monthly exclusion). - Tool B: 'No. Monthly Pro plans are non-refundable; the 30-day/<500-call rule applies only to annual billing [Source: Billing Policy §4.2].' -> PASS (Exact multi-condition logic + citation).

Multi-condition exception questions expose shallow keyword matching that looks convincing on simple FAQs.

Common questions

Why can't public LLM leaderboards tell us which business AI tool to buy?

Public academic benchmarks test general reasoning, whereas business tool reliability depends heavily on chunking, document OCR, retrieval ranking, and abstention prompts.

How many test prompts do we need for a trustworthy internal AI benchmark?

Thirty to fifty well-chosen prompts spanning routine lookups, multi-hop exceptions, table reading, and out-of-scope traps reliably separate strong from weak tools.

Should we re-run our domain accuracy benchmark after buying the software?

Yes—re-run the exact 30-item golden set whenever you overhaul help docs or the vendor updates its retrieval engine to catch silent regressions.

Learn more

Related guides and background articles for this task