Recent support-agent launches are converging on the same production constraint. Pylon argues that an agent cannot decide well from only the latest message and a help center; its July agentic support launch emphasizes account, product, and support history. OpenAI's Presence takes the complementary position: each agent should receive only the knowledge and system access required for its specific job. Salesforce's July Help Agent launch likewise pairs real-time customer data with a complete context packet when a human takes over.
Turn those principles into a context envelope for every support turn. The envelope should contain six things: verified actor, current request, live customer state, governing policy, allowed operations, and evidence metadata. Each field needs a source, retrieval time, and permitted use. If a field cannot be verified or is no longer fresh, the chatbot must narrow its answer or escalate.
Start With a Context Envelope, Not a Bigger Prompt
A system prompt explains general behavior. A context envelope supplies the facts that are true for this customer, this request, and this moment. Mixing the two creates prompts that are long, hard to audit, and likely to retain private data beyond its useful life.
Use this minimum record:
| Context field | Required contents | Safe failure behavior |
|---|---|---|
| Actor | Verified user, tenant, role, channel, authentication strength | Treat the visitor as anonymous; do not expose account data |
| Request | Current intent, entities, unresolved questions, explicit consent | Ask one clarifying question instead of inferring intent |
| Customer state | Relevant plan, order, invoice, entitlement, configuration, or recent event | Say the live state is unavailable; avoid status claims |
| Governing rule | Canonical policy or product rule, scope, effective date, owner | Escalate conflicts or expired rules |
| Allowed operations | Visible tools, authorization constraints, approval requirements | Offer information or handoff; do not attempt the action |
| Evidence | Source IDs, retrieval times, versions, tool receipts, confidence flags | Mark the answer unsupported and withhold consequential claims |
This is an interface contract, not a transcript dump. A refund workflow may need purchase date, region, plan, refund status, and the applicable policy version. It does not need every page the customer viewed, their entire CRM record, or unrelated conversations from last year.
The distinction also prevents a common mistake: treating conversation history as customer truth. A visitor may say, “I am on the Pro plan,” but the authenticated billing record is authoritative. They may paste an old support reply promising a 30-day refund, but the approved policy and its effective date decide which rule applies now.
Separate Four Kinds of Context
Different context expires, changes, and carries risk at different rates. Keep these categories explicit instead of flattening everything into one retrieval result.
Conversation context captures what the customer asked, what the chatbot already said, confirmations, and unresolved points. Summarize it around decisions rather than retaining every token. The chatbot context-window guide explains how to preserve commitments and open questions without carrying an unlimited transcript.
Identity context establishes who is asking and what they may see or do. It includes tenant, role, verification method, and authentication age. A matching email address in a message is not identity proof. For account-specific answers, use the boundaries in the authenticated chatbot guide: resolve identity server-side, scope every lookup, and fail closed when the binding is missing.
Business context is live operational state: subscription, shipment, case, product configuration, usage, outage, or entitlement. Fetch it from the system that owns the fact. Do not copy a dynamic balance or status into a long-lived knowledge base and expect it to stay correct.
Policy context defines what the business permits. Refund rules, cancellation windows, warranty exceptions, escalation thresholds, and approval requirements need an owner and effective date. Policies often look static but change at product launches, contract renewals, and regulatory deadlines.
The chatbot should know which category supplied every claim. That provenance determines whether it may answer, must refresh a lookup, or needs a human decision.
Match Authority and Freshness to the Claim
“The source is trusted” is too vague. Authority depends on the claim. A CRM may own the account tier, a billing service the payment state, a product entitlement service feature access, and an approved policy repository refund eligibility.
Define a claim-to-source map before connecting tools:
- Account identity: authentication and account service, retrieved for the current session.
- Transaction state: billing, order, or case system, fetched immediately before the answer or action.
- Product behavior: maintained documentation or a verified product API, versioned to the deployed product.
- Business permission: approved policy, including its jurisdiction and effective date.
- Prior commitment: conversation log or case history, linked to the exact message and author.
Give each source a freshness rule. “Current” can mean seconds for a payment, hours for an incident flag, a release cycle for product documentation, or a fixed review date for policy. Store retrieved_at and, where possible, valid_until or the event that invalidates the value.
When sources disagree, do not ask the model to choose the most plausible sentence. Use a deterministic precedence rule or emit a named conflict such as billing_state_mismatch or policy_scope_unknown. The ground-truth guide shows how to record accepted outcomes and forbidden claims when edge cases depend on multiple sources.
Minimize Context Before the Model Sees It
More context can improve recall while making authorization, privacy, and relevance worse. Minimize at three stages.
Before retrieval, decide which facts the intent could legitimately require. A public delivery-times question needs destination and service level, not an account lookup. An invoice-status question requires authentication before any billing data is fetched.
After retrieval, transform raw records into purpose-limited fields. Pass plan: standard rather than the whole customer row. Pass invoice_status: unpaid and invoice_id_suffix: 1842 rather than full payment details. Exclude secrets, internal notes that are not customer-visible, and data about other users in the account.
Before logging, remove transient sensitive values that were needed for the lookup but are not needed for review. Keep source IDs, reason codes, and action receipts so an operator can reproduce the decision without storing credentials or full payloads.
This minimization should be enforced in code or tool schemas. A prompt instruction such as “ignore unrelated private data” runs after the data has already crossed the boundary.
Worked Case: A Cancellation Request With Conflicting Signals
Suppose a customer writes:
Cancel our account today. I am the workspace owner, and support promised there would be no early-termination fee.
The email address belongs to the workspace, but the session is not authenticated. The CRM labels the sender “Executive Sponsor,” the account service lists someone else as owner, the contract API reports an annual commitment, and a six-month-old support message mentions a possible fee waiver without approval.
A useful envelope looks like this:
actor:
tenant_id: acct_482
user_id: null
authentication: unverified_email_claim
roles: []
request:
intent: cancel_account
requested_effective_date: 2026-08-13
claims:
- workspace_owner
- fee_waiver_promised
customer_state:
contract_term: annual
renewal_date: 2027-02-01
cancellation_status: active
policy:
rule_id: cancellation_v6
effective: 2026-07-01
owner: billing_operations
operations:
cancel_account: denied_until_owner_verified
open_billing_review: allowed
evidence:
- account_role_lookup: current
- contract_lookup: current
- support_message: historical_unapproved_statement
The correct response does not confirm the contract details or execute cancellation. It can acknowledge the request, explain that the workspace owner must sign in or use the approved verification path, and offer to open a billing review for the claimed waiver.
Compare three possible replies:
“Your annual account is now canceled with no fee.” Fails twice: the actor lacks authority, and no approved waiver or cancellation receipt exists.
“You are not the owner, so I cannot help.” Avoids the destructive action but exposes an account-role conclusion to an unverified visitor and discards a valid support route.
“I can help route this, but I cannot change the account from an unverified session. Please sign in as an authorized owner; I can also send the fee-waiver claim to billing for review.” Preserves privacy, states the boundary, and offers the allowed next step.
If a human takes over, pass the request, verification gap, policy ID, contract lookup timestamp, disputed prior message, and actions not taken. The handoff-routing guide provides the routing fields; the context envelope supplies the evidence the receiving team needs.
Test Missing, Stale, and Forbidden Context
Happy-path tests prove little because every field is present and consistent. Build tests around the ways context fails:
| Test condition | Expected behavior |
|---|---|
| Identity missing | No account data disclosed; verification path offered |
| Customer state lookup times out | No status or completion claim; retry or handoff with outage reason |
| Policy expired | No eligibility decision; route to policy owner or support |
| Two authoritative sources conflict | Conflict named and escalated; no blended answer |
| User requests another tenant's record | Lookup denied before retrieval; event logged |
| Tool returns success without a receipt | Outcome remains unknown; do not tell the customer it completed |
| Old transcript contradicts current policy | Preserve the dispute as evidence; use approved review path |
| Irrelevant sensitive field is returned | Field removed before model and logs; schema test fails |
For each case, score both the answer and the data path. A polite refusal can still fail if the system fetched another tenant's record. A correct answer can still fail if it came from an expired cache. Record whether retrieval was authorized, the chosen source was authoritative, freshness passed, the response stayed within the evidence, and the handoff carried enough context.
Measure Context Quality Separately From Answer Quality
An answer-quality score alone cannot tell whether the model reasoned badly or received the wrong facts. Track the context layer with four ratios:
Context completeness = turns with all required fields / eligible turns Freshness pass rate = time-sensitive fields inside their freshness window / checked fields Provenance coverage = customer-facing factual claims linked to evidence / factual claims Overfetch rate = retrieved fields not permitted for the intent / all retrieved fields
Suppose 500 authenticated billing turns require five context fields, creating 2,500 field checks. Review finds 2,425 present and 2,350 within their freshness windows. Across sampled answers, reviewers identify 800 factual claims and can link 776 to evidence. Tool traces show 60 fields that the intent did not permit among 3,000 retrieved fields.
Context completeness = 2,425 / 2,500 = 97.0% Freshness pass rate = 2,350 / 2,425 = 96.9% Provenance coverage = 776 / 800 = 97.0% Overfetch rate = 60 / 3,000 = 2.0%
Those numbers create specific work. Missing fields point to connector reliability or intent classification. Stale fields point to cache rules. Unsupported claims point to answer grounding. Overfetch points to authorization and schema design. One blended “chatbot accuracy” percentage would hide all four.
Give Every Answer the Smallest Sufficient Truth
Reliable support answers need more than a long transcript and a search result. They need a bounded package of verified identity, current state, applicable policy, permitted actions, and reproducible evidence. The safest envelope is the smallest one that can support the requested decision—and it makes missing truth visible instead of inviting the model to fill gaps.
Start with one high-volume workflow. Define its six context fields, authority map, freshness rules, and safe failures. Test the missing and conflicting cases before adding more connectors. A chatbot that can explain why it must pause is safer than one that silently substitutes stale or unauthorized context.
In Agentkit, conversation logs expose the questions and source references behind answers, while reviewed Q&A pairs can pin critical policy responses that need exact control.
No credit card required.
