AI products can disappear even when the underlying models remain. OpenAI's June 3 update says its Agent Builder and Evals products will be unavailable after November 30, 2026. A different kind of disruption hit Anthropic in June: access to Claude Fable 5 and Mythos 5 was suspended, then restored after export controls changed.
A website chatbot needs an exit plan that covers more than its model. Before moving, save this checklist:
- Export every approved source and record its owner, URL, version, and visibility.
- Copy the system prompt, welcome messages, fallback copy, and escalation rules.
- Inventory actions, forms, webhooks, API calls, secrets, and permission scopes.
- Build a test set from real questions, failures, and high-risk requests.
- Record the current bot's answers, citations, latency, handoff rate, and cost.
- Map visitor, contact, conversation, and account identifiers between systems.
- Run both bots against the same test set before changing the live embed.
- Keep the old bot and configuration recoverable until the new one clears exit criteria.
The deliverable is a migration evidence package: source manifest, behavior specification, integration map, baseline results, and cutover record. Here is how to build it.
Capture the Bot You Actually Run
The settings page is not the whole chatbot. Its behavior is the result of content, instructions, retrieval settings, model behavior, UI copy, identity, and external systems working together. If you copy only documents and a prompt, the new bot may look familiar while answering differently.
Start with a read-only inventory. Do not clean it up yet. A duplicate PDF, stale URL, or unexplained instruction is evidence about the production system. Removing it during discovery makes it harder to explain why an answer changes later.
Your inventory should cover six layers:
- Knowledge: crawled pages, uploaded files, pasted text, approved Q&A pairs, exclusions, and source freshness.
- Behavior: system instructions, tone rules, refusal boundaries, citation settings, fallback language, and conversation starters.
- Interface: widget placement, color and typography settings, launcher behavior, mobile layout, and accessibility labels.
- Workflow: lead forms, buttons, API actions, webhooks, scheduling, CRM writes, and human escalation.
- Identity: anonymous sessions, verified contacts, account membership, consent, retention, and deletion behavior.
- Operations: models, limits, analytics definitions, alerts, owners, incident notes, and billing assumptions.
The open-source versus SaaS comparison treats migration as one factor in a platform decision. For an active support bot, this inventory is the work hidden inside that factor.
Write a Portable Migration Manifest
A spreadsheet is fine for discovery, but a versioned text file is easier to review, diff, and hand to another team. Use stable names that describe business purpose instead of vendor screen labels.
chatbot: help-center-production
owner: support-operations
snapshot_date: 2026-07-12
knowledge:
- id: billing-policy
type: webpage
canonical_url: https://example.com/help/billing
visibility: public
owner: finance-ops
reviewed_at: 2026-07-01
- id: cancel-account
type: approved_qa
visibility: signed-in-customers
owner: support-ops
reviewed_at: 2026-06-28
workflows:
- id: refund-escalation
trigger: refund requested after 30 days
action: create_support_ticket
required_identity: verified_customer
rollback: disable action and show handoff form
release_gate:
required_answer_pass_rate: 0.95
max_p95_latency_ms: 4500
forbidden_action_failures: 0
Do not put API keys, webhook secrets, customer transcripts, or personal data in this file. Store a secret's name and rotation owner, then transfer the value through your normal secrets process.
The manifest also exposes ownership gaps. A policy page with no named owner cannot be confidently migrated because nobody can approve the answer it produces. Assign the owner or exclude the source explicitly; silent ambiguity turns into a production dispute.
Decide What Can Move, Be Rebuilt, or Be Retired
Platform exports rarely map one-to-one. Classify each component before anyone starts recreating settings.
| Component | Preserve as evidence | Rebuild in target | Acceptance check |
|---|---|---|---|
| Sources | Original file or URL, checksum, owner, visibility | Ingest through the target's supported source type | Expected source is retrieved for a known question |
| Instructions | Exact text and version history | Adapt only where the target requires it | Behavior tests pass without unexplained exceptions |
| Q&A pairs | Question, approved answer, owner, review date | Recreate as highest-priority controlled knowledge | Exact policy questions use the approved meaning |
| Actions | Request/response examples, permission scope, failure copy | Reauthorize and implement with new credentials | Allowed call succeeds; denied and malformed calls fail safely |
| Identity | Identifier definitions and verification flow | Map IDs without copying unnecessary personal data | Cross-user and signed-out tests remain isolated |
| Analytics | Metric formulas and baseline period | Recreate dashboards from target events | Same sample conversations classify consistently |
Retire components deliberately. An unused webhook should have a retirement ticket, owner, and date rather than simply vanishing from the new configuration. That creates a record if another system was quietly depending on it.
Build Tests From Conversations, Not Features
Feature checklists say both platforms support documents, citations, forms, and webhooks. They do not tell you whether a customer asking “Can I get a refund?” receives the same safe and useful outcome.
Build a migration test set from recent conversation logs. Strip personal data, retain the user's meaning, and label each case with:
- expected answer facts;
- acceptable source or citation;
- facts the bot must not claim;
- whether identity is required;
- allowed action or handoff;
- severity if the result is wrong.
Include frequent questions, high-value conversion questions, known failures, ambiguous wording, multilingual examples, and action attempts. Add adversarial cases for instructions found inside uploaded documents and requests to expose another customer's data.
The AI chatbot QA guide shows how to turn reviewed conversations into a continuing test set. For migration, freeze a dated copy so the old and new systems face identical inputs.
Score facts separately from style. A new bot can sound warmer and still invent a refund deadline. Use pass/fail for required facts and prohibited claims, then score tone or formatting as a secondary dimension.
A Worked Cutover for a Billing Bot
Suppose a support team migrates a billing chatbot with 420 source pages, 38 approved Q&A pairs, one lead form, and one ticket-creation action. During inventory, it finds 27 duplicate URLs and 11 pages with no content owner.
The team does not delete those 38 questionable sources immediately. It creates a baseline first, then excludes them in a separate, reviewable source-cleanup change. That keeps “new platform behavior” separate from “new knowledge set behavior.”
It selects 120 anonymized questions:
- 60 common billing and plan questions;
- 20 cancellation and refund edge cases;
- 15 signed-in account questions;
- 10 ticket-creation attempts;
- 10 known historical failures;
- 5 prompt-injection and cross-customer requests.
The old bot passes 108 of 120. The new bot initially passes 103. A single percentage would make both look close, but the failures tell a different story: four of the new bot's misses are refund-policy errors and two are unauthorized ticket attempts. Those are release blockers.
After the team recreates the approved refund Q&A, tightens the action's identity requirement, and retests, the new bot passes 116 of 120 with zero prohibited actions. Its remaining four misses are formatting differences marked low severity. The migration record contains both runs, the configuration changes between them, and the four accepted differences.
This worked example also prevents a misleading comparison. Cleaning duplicates improved retrieval, but that improvement is recorded as a separate change. If accuracy later falls, the team can tell whether to reverse the content cleanup, the platform cutover, or both.
Reauthorize Every Action
Never copy production credentials from one chatbot platform into another and assume the permission boundary stayed intact. Create new credentials where possible, limit them to the target system, and rotate or revoke the old ones after rollback closes.
For every action, test three paths:
- Allowed: a verified user provides valid input and the downstream system accepts exactly one request.
- Denied: an anonymous or wrong-account user cannot trigger the action, including through prompt manipulation.
- Failed: timeouts, malformed responses, and duplicate submissions produce safe user copy and an observable operational error.
Record idempotency behavior too. If the user retries after a timeout, will the bot create a second ticket or refund? Conversation wording cannot guarantee transaction safety; the downstream API needs a stable idempotency key or equivalent duplicate protection.
The chatbot tool-permissions checklist is useful here because a migration is the right moment to replace broad legacy tokens with purpose-specific access.
Preserve Identity Boundaries
Identifier names often look interchangeable while meaning different things. One platform's user_id may identify a browser contact; another may require a verified application user. Write down the proof behind every identifier before mapping it.
Run isolation tests with two anonymous browsers, two verified customers, two tabs, an account switch, a deleted conversation, and a removed team member. The chatbot memory-isolation test suite provides the detailed cases. Migration adds one more: confirm that legacy IDs cannot retrieve records created under a different target identity.
Move the minimum data needed. Historical transcripts are valuable for tests and support continuity, but they may not belong in the new bot's live retrieval index. Keep operational history separate from approved knowledge unless an owner has reviewed and authorized it as a source.
Cut Over With a Reversible Sequence
Run the new bot in a non-public environment first. Once it clears offline tests, use a small live slice if your delivery setup supports traffic splitting; otherwise choose a low-volume window and keep the previous embed configuration ready.
Record these values immediately before cutover:
- old embed or endpoint version;
- new embed or endpoint version;
- source and instruction snapshot IDs;
- credentials activated and credentials awaiting revocation;
- dashboard links and alert owners;
- rollback decision-maker and communication channel.
Watch question-level outcomes, not only uptime. A bot can return HTTP 200 responses all day while citation quality falls or handoffs stop creating tickets. Compare answer pass rate, prohibited claims, action failures, fallback rate, handoff completion, p95 latency, and cost using the same definitions captured in the baseline.
Keep the rollback window tied to evidence rather than a calendar habit. Close it only after representative traffic has covered high-risk intents, every integration has produced a successful live event, no identity-isolation failure remains, and the team has accepted the measured differences. The model rollback checklist covers the narrower case where the platform stays put but the model must change.
Portability Is a Release Requirement
A migration is successful when customers can still get correct answers and complete safe workflows, while operators can explain every accepted difference. The export archive alone proves very little. The evidence package makes the bot reproducible: owned sources, specified behavior, bounded permissions, comparable tests, and a reversible cutover.
Build that package before a vendor deadline or access change forces the schedule. It improves the current chatbot immediately by exposing stale knowledge, ownerless policies, broad credentials, and metrics nobody can define.
No credit card required.
