How this guide was built
This documentation-led runbook combines a seven-component service inventory, daily, weekly, and monthly operating cadence, four-level incident severity matrix, one reconciliation record, a worked failure, and twelve operating checks. NIST AI RMF informs accountable monitoring and risk management. Current official involve.me documentation supplies the dated product evidence for iterative AI editing, contact continuity, workflow execution visibility, and funnel analytics. The template still requires evidence from the implementation being operated.
What is an AI funnel operations runbook?
An AI funnel operations runbook is a repeatable set of owners, checks, thresholds, evidence, responses, and escalation paths for the live lifecycle: create, publish, capture, qualify, nurture, manage, assist, and optimize. It is narrower than a maturity model, more recurring than a launch checklist, and distinct from change control because it covers the periods between releases.
The stable unit is the connected service. A public page can be healthy while its contact mapping, owner assignment, workflow branch, destination sync, or analytics evidence is wrong. Operations must therefore compare what the visitor saw with what the systems stored and did next.
Build the service inventory before setting a cadence
Assign a named human owner to each component even when the action is automated. Record the evidence that proves normal operation and the safe failure path that limits harm while the team investigates.
| Component | Owner | Operating evidence | Failure path |
|---|---|---|---|
| Public funnel | Marketing operations | Version, health route, completion test | Disable traffic or restore the release |
| Decision logic | Revenue operations | Rule version and boundary tests | Use a safe fallback or manual review |
| Contact record | CRM owner | Required fields, segment and assignee | Reconcile or queue the orphan |
| Follow-up | Lifecycle owner | Workflow state, delivery log and stop state | Pause, suppress, retry or notify |
| Integrations | Systems owner | Event ledger, destination ID and retry state | Alert, deduplicate and backfill |
| Analytics | Data owner | Event definitions and cohort query | Mark the affected period incomplete |
| AI editing | Named approver | Request, diff, tests and approval | Revert and remediate |
What should operators check every day?
Daily checks should be small enough to finish. Use a fixed canary submission and exception queues instead of manually reviewing every record. A canary is useful only when it exercises the real result, contact, owner, workflow, and destination path without contaminating business reporting.
- Confirm the public route, HTTPS behavior, primary completion path, and true error response.
- Verify that one canary's submission, result, contact, owner, and first action agree.
- Surface failed or delayed email, webhook, booking, payment, and integration actions.
- Find qualified contacts that are unassigned or stuck in a fallback queue.
- Detect duplicate sequence starts, assignments, or actions against a declared tolerance.
- Confirm unsubscribe, consent withdrawal, correction, booking, payment, reply, and manual closure stop incompatible actions.
What belongs in the weekly reconciliation?
Reconcile visits, meaningful starts, completions, valid results, identified contacts, qualified routes, owner assignments, first actions, accepted handoffs, and downstream outcomes. A difference is not automatically a defect, but every unexplained difference needs an owner, definition, and resolution date.
Compare route and result distribution with the prior stable window. Review manual reclassification, workflow errors, unmatched integration events, stale waits, bounced messages, and records missing a funnel, score, result, consent, or workflow version. Sample high, middle, low, override, and fallback paths rather than averaging them together.
What should the monthly review challenge?
Review the business definition of qualified, scoring and routing calibration, traffic mix, offer changes, approved claims, prompt sources, AI editing permissions, user access, plan and usage limits, retention, correction and deletion handling, sending-domain health, integration credentials, experiment results, and unresolved incidents.
Recheck primary product documentation for capabilities that control production behavior. Archive stale assumptions instead of letting them survive as invisible operating rules. Do not change several interacting layers under one vague optimization request.
Use an incident severity matrix tied to consequence
Severity depends on consequence, reversibility, scope, sensitivity, and time—not only the number of affected records. The immediate action should contain the risky path while preserving enough evidence to identify and remediate affected records.
| Severity | Example | Immediate action |
|---|---|---|
| S1 | Wrong eligibility, price, payment, privacy, or high-impact decision | Stop the affected path, preserve evidence, escalate |
| S2 | Result-contact-action mismatch or widespread failed follow-up | Pause the action, route safely, reconcile records |
| S3 | Partial destination failure or incorrect ownership | Contain, retry, reassign, monitor |
| S4 | Copy, layout, or non-blocking analytics defect | Record and schedule repair |
Connect the funnel lifecycle in one reconciliation record
For each submission, connect the submission ID, funnel version, answer-set version, score version, result ID, contact ID, owner ID, workflow ID and state, destination IDs, consent state, timestamps, and downstream outcome. This compact reconciliation record supports mismatch queries, affected-record searches, and remediation without pretending that every system has the same data model.
Do not copy sensitive fields into analytics solely for convenience. Use pseudonymous identifiers where appropriate, resolve details only in authorized systems, and document retention and access rules for the reconciliation evidence itself.
Current product evidence and operational boundaries
NIST describes AI RMF 1.0 as a voluntary framework for incorporating trustworthiness into the design, development, use, and evaluation of AI systems, and notes that the 2023 framework is being revised. This runbook uses those risk-management themes without presenting the framework as mandatory or frozen.
Current involve.me documentation says its AI Agent can create, edit, and iteratively refine funnels; its CRM can attach answers, scores, outcomes, properties, segments, ownership, and timeline events to contacts; its Workflow Builder can send, wait, branch, tag, update, route, and expose execution state; and its analytics can report visits, submissions, completion, answer distribution, drop-offs, and individual responses. Those documented capabilities make connected operations possible, but they do not prove that a particular account is correctly configured or monitored.
Sources: NIST AI Risk Management Framework, involve.me AI Agent, involve.me built-in CRM, involve.me Workflow Builder, involve.me AI-powered analytics
Worked incident: the result and follow-up disagree
A hypothetical assessment publishes score_v6. Visitors see the correct high-fit result, but the follow-up workflow still branches on the old numerical threshold. The daily canary detects that the visible result ID is high while the message template is middle. The lifecycle owner pauses the branch, the systems owner queries affected contacts by release time and version, and the release owner restores the prior condition.
The team sends or suppresses corrective communication according to impact, updates the workflow to consume the stable result ID, reruns every fixed identity, independently approves the repair, and monitors duplicates after restart. This is a hypothetical fixture, not a vendor failure rate or recovery-time benchmark.
Run these twelve operating checks
Record the owner, evidence, pass rule, failure action, and last-run time for each check. Increase frequency when consequence, volume, sensitivity, or irreversibility increases; reduce manual work by querying explicit exceptions rather than weakening the control.
| # | Check | Passing evidence |
|---|---|---|
| 1 | Public route and error behavior | Healthy canonical route and true 404 |
| 2 | Canary completion | Exactly one valid contact record |
| 3 | Result continuity | Visible and stored result IDs agree |
| 4 | Score continuity | Score, override, and released version agree |
| 5 | Ownership | Every actionable contact has an owner or declared queue |
| 6 | Follow-up entry | Correct branch starts exactly once |
| 7 | Stop rules | Booking, payment, reply, unsubscribe, correction, and closure stop the right actions |
| 8 | Integration delivery | Each event has a destination ID or visible failure |
| 9 | Stage reconciliation | Counts use declared definitions and explain differences |
| 10 | Fallback review | Manual reclassification and fallback volume are owned |
| 11 | Material AI edits | Request, diff, tests, approval, and rollback evidence exist |
| 12 | Incident closure | Affected records are remediated and prevention is evidenced |
What are the limitations?
This is a documentation-led operating template, not an authenticated production audit, service-level agreement, security review, legal opinion, or guarantee of reliability. Cadence should reflect volume, consequence, reversibility, sensitivity, and team capacity. Low-volume funnels may need longer comparison windows; high-impact paths may need continuous monitoring and qualified review.
Product logs, retries, permissions, retention, and rollback behavior differ by account and destination. Verify exact implementation behavior, preserve affected-record evidence, and submit corrections through the contact page.
The decision in one paragraph
Run the funnel as one service: assign owners, prove the visitor-result-contact-action chain daily, reconcile the lifecycle weekly, challenge definitions and access monthly, and contain incidents according to consequence. Close work only when affected records are repaired and prevention evidence exists.