Playbook
Control design and vendor evaluation.
Who is asking for access?
Create stable IDs for the applicant, business, owners, accounts, and application. Record the source and time of each fact.
Owner Product + identity operations
Inputs
- Application and declared activity
- Company, owner, and document records
- Consent and collection context
Outputs
- Versioned customer and entity profile
- Evidence references and missing fields
- Ownership relationships
Failure handling
Missing required evidence routes to a recoverable review state. A provider timeout is distinct from a failed check.
Shared data and event contracts
Use stable entity IDs, versioned events, source references, permissions, and audit history across the system. These synthetic schemas illustrate the contract; they are not vendor APIs.
A stable subject record. Store sensitive evidence in an access-controlled store; carry a reference here.
{
"schema_version": "customer.v1",
"event_id": "evt_customer_001",
"subject_id": "business_1042",
"subject_type": "business",
"occurred_at": "2026-09-01T10:00:00Z",
"recorded_at": "2026-09-01T10:00:02Z",
"profile_version": 3,
"legal_name": "Example Market LLC",
"jurisdiction": "US",
"owner_subject_ids": ["person_204"],
"expected_activity": { "monthly_volume_minor": 5000000, "currency": "USD" },
"evidence_refs": ["evidence://registry/1042/v3"]
}Validation checks
- Use stable subject IDs across vendors.
- Preserve old profile versions.
- Keep ownership percentages, control roles, and sources on relationship records.
Real-time decisions and ongoing monitoring
Before an action
Set the latency budget, decision expiry, fallback, and execution receipt for a payment, account, or session decision.
Request → Decision → Enforce
Test a provider timeout, a repeated request, a stale response, and a policy update during the request.
Across time
Track event time and processing time separately for historical patterns, changed profiles, investigations, and quality review.
Events → Pattern → Case
Test missing events, delayed feeds, corrections, a replay, and a full rebuild from a known snapshot.
Trace one customer and one payment from original evidence to decision, action, and review. Each gap is an integration requirement.
Verification and acceptance
| Layer | Evidence | Output | What remains |
|---|---|---|---|
| KYC / identity | Who is this person? Identity records, document checks, liveness where used, fraud signals, and review evidence. | An identity result with uncertainty and provenance. | Identity confidence does not establish business viability or repayment capacity. |
| KYB / ownership | What is this business, and who owns or controls it? Registry facts, status, address, owners, controllers, operating evidence, and entity relationships. | A business profile with resolved gaps and an ownership map. | A registered business can still have a risky, prohibited, or misrepresented operating model. |
| Screening / due diligence | What external risk needs review? Sanctions, political exposure, relevant adverse media, and applicable ownership research. | Match decisions and a documented due diligence outcome. | A screening score is not a legal conclusion, credit score, or fraud verdict. |
| Merchant underwriting | What financial and operating exposure can we accept? Business model, product delivery, volume, ticket size, refunds, disputes, financials, and processing history. | Acceptance decision, approved activity, conditions, limits, and review schedule. | A workflow engine needs the policy, evidence, and downstream controls to carry out this job. |
| Ongoing risk | Has the accepted risk changed? Actual activity, returns, disputes, ownership changes, new screening records, and customer contact. | Updated risk, investigation, revised limits, or a new approval review. | The original onboarding decision becomes stale when the business changes. |
Acceptance gates
Business eligibility
Is the activity allowed under the approved acceptance policy?
- Evidence
- Products, website, locations, licenses where relevant, and stated use of funds.
- Outcome
- Ineligible activity → policy decline or authorized exception process.
Identity & ownership
Do the person, entity, owner, and controller checks have sufficient evidence?
- Evidence
- Resolved verification results, source records, ownership graph, and review rationale.
- Outcome
- Unresolved evidence → request information or manual review.
Exposure assessment
Do delivery timing, volume, financial condition, and loss history fit the proposed terms?
- Evidence
- Processing statements, financial records, order fulfillment, refunds, and dispute history.
- Outcome
- Exposure above appetite → revise terms, lower limits, review, or decline.
Decision & enforcement
Can the operating systems enforce every approved condition?
- Evidence
- Decision record, owner approval, limit write, reserve configuration where used, and execution receipt.
- Outcome
- Unenforced condition → do not treat conditional approval as complete.
Review triggers
Which changes reopen the approval decision?
- Evidence
- Volume and ticket drift, product changes, ownership change, adverse signals, and loss outcomes.
- Outcome
- Trigger → investigation or re-underwriting with prior decision context.
Test a conditional approval
- Create an approval with a lower volume limit and a required follow-up.
- Send the limit to the processor or ledger and show the execution receipt.
- Attempt activity above that limit and confirm the actual result.
- Change the owner record and show how the approval review reopens.
- Export the full decision and change history.
Measure friction, exposure, and reliability
- Approval quality
- Loss and dispute outcomes by accepted cohort, with enough time for outcomes to mature.
- Applicant friction
- Completion, abandonment, extra evidence requests, and manual review by segment.
- Decision reliability
- Time to decision, reviewer agreement, policy exceptions, and successful condition enforcement.
Confirm who supplies financial analysis, website review, processing history, reserve calculations, settlement controls, and portfolio monitoring. A configurable workflow still needs these inputs and control points.
Detection scenarios
Rapid movement of funds
Inflow followed by outflow with little retained balance or expected business purpose.
- Data needed
- Timestamped flows, counterparties, balances, and stated business activity.
- Legitimate explanations
- Payroll, treasury sweeps, and settlement businesses can show similar patterns.
- Pilot test
- Seed known sequences and legitimate flow-through accounts. Compare detection by customer type and time window.
Distributed activity
Related payments or accounts form a pattern that a single-transaction check misses.
- Data needed
- Customer and linked-party IDs, cross-account activity, time windows, and payment context.
- Legitimate explanations
- Splits, installments, and batching can be legitimate. A pattern alone does not establish intent.
- Pilot test
- Test linked events across accounts, channels, and time. Include legitimate payment splits as controls.
Funnel or mule networks
Many sources concentrate funds into a small set of recipients or connected accounts.
- Data needed
- Sender and beneficiary graph, device or account links where permitted, and activity timing.
- Legitimate explanations
- Marketplaces, payroll processors, and shared household devices can create dense links.
- Pilot test
- Measure entity-level detection on seeded networks. Examine whether one weak link expands into unrelated accounts.
Dormant account activation
A quiet account changes to sustained activity that does not fit its known purpose.
- Data needed
- Account age, inactivity periods, login or device changes, new beneficiaries, and payment history.
- Legitimate explanations
- Seasonality, a new product launch, or a genuine business restart can explain the change.
- Pilot test
- Replay seasonal and nonseasonal histories. Check alert timing and the context shown to the analyst.
Circular or reciprocal flows
Funds move through related parties and return to the origin or a linked entity.
- Data needed
- Directed payment graph, ownership links, amount and timing, and currency conversion.
- Legitimate explanations
- Intercompany transfers, loan repayments, and refunds can be reciprocal.
- Pilot test
- Use loops of different lengths. Include partial returns, currency conversion, and legitimate treasury flows.
Profile or corridor mismatch
New countries, counterparties, products, or volumes differ from the accepted activity profile.
- Data needed
- Expected activity, actual flows, customer segment, corridor, and updated customer evidence.
- Legitimate explanations
- Business expansion and one-time large orders can change activity without wrongdoing.
- Pilot test
- Separate new business growth from unexplained change. Inspect the history used to form the baseline.
Merchant activity mismatch
Payment patterns, customer complaints, or fulfillment evidence conflict with the declared business.
- Data needed
- Merchant profile, transaction descriptors, website evidence, refunds, disputes, and customer contact.
- Legitimate explanations
- Processors may not have item-level sales data. A legitimate pivot can resemble concealed activity.
- Pilot test
- Ask the vendor to state missing evidence. Check links from a merchant alert to the actual source events.
Unexplained high-risk relationships
Activity involves a party or relationship that needs further due diligence under the firm’s policy.
- Data needed
- Counterparty identity, ownership, screening context, geography, and business purpose.
- Legitimate explanations
- Name collisions and unresolved entity links can create unsupported risk labels.
- Pilot test
- Use ambiguous identities and corrected links. Verify that the system can retract an incorrect relationship.
Investigation workflow
Triage
Check the trigger, data quality, subject, prior alerts, and urgency. Keep duplicated alerts linked to the same case.
Build the timeline
Bring together customer history, money movements, linked entities, and changes in the accepted risk profile.
Test the explanation
Compare the activity with its stated purpose. Record both supporting and conflicting evidence; request missing facts.
Make the decision
An authorized reviewer records the rationale, uncertainty, and next action. A suspicious pattern is a prompt for review.
Execute & confirm
Route account actions and any applicable reporting to their approved workflows. Record execution and submission receipts.
Quality & feedback
Sample decisions, correct labels, track repeat cases, and feed reviewed outcomes into controlled rule or model changes.
Relationship analysis
Review the transaction sequence and related entities. A shared recipient, device, address, or owner can add context. Record the source and confidence of each inferred link.
Test concentrated flows, linked accounts, and circular transfers. Include legitimate marketplaces, payroll processors, households, and treasury activity as controls. Confirm that the system can retract a wrong relationship.
Human and AI responsibilities
AI assistance
- Retrieve permitted evidence and build a cited timeline.
- Explain a rule or model trigger.
- Propose questions about missing facts.
- Draft summaries and identify conflicting evidence.
- Flag incomplete fields for review.
Human decisions
- Set policy, test criteria, and acceptable uncertainty.
- Confirm identity links and evidence interpretation.
- Authorize account and reporting actions.
- Approve rule, model, and prompt changes.
- Review sampled closures and correct weak labels.
Measure detection on a fixed labeled sample, AI summaries for factual support, and investigations through reviewer quality checks. Faster closure alone does not prove a stronger control.
Define the coverage
Record which lists, programs, entity types, ownership data, and jurisdictions the policy needs. Keep sanctions, PEPs, and adverse media as distinct result types.
- Evidence to request
- A source-by-source coverage inventory with license rights and a named owner.
Normalize without losing evidence
Retain the original input. Test name order, punctuation, script, transliteration, legal suffixes, aliases, birth dates, and identifiers.
- Evidence to request
- Original and normalized fields with field-level match explanations.
Match, then resolve
Generate candidates and inspect identity attributes. Document contradictory evidence and ambiguity. A similarity score does not establish that two parties are the same.
- Evidence to request
- An independently labeled test set with positive matches, difficult negatives, and reviewer agreement.
Make refresh observable
Measure source publication, provider ingestion, screening index availability, rescreening completion, and alert creation. A vendor’s “daily updates” statement is not an end-to-end service measure.
- Evidence to request
- Versioned snapshots, change logs, freshness alerts, and a catch-up test after an outage.
Control the disposition
Separate false-positive clearance from the decision about a confirmed exposure. Have the authorized owner determine the applicable action and record the reason.
- Evidence to request
- Permissions, dual review where policy requires it, expiring suppressions, and an action audit trail.
Data refresh
List change is available.
A versioned snapshot is ready.
New checks use the change.
Existing records are checked.
New cases enter the queue.
Measure the elapsed time at every step. Set the budget for each source and operating model. These are measurement points, not regulatory deadlines.
Example: resolve a potential match
Compare identity attributes, source records, and missing evidence. Name similarity alone does not establish a match. All records in this example are fictional.
| Attribute | Customer | List candidate |
|---|---|---|
| Name | Mariam Haddad | Maryam Al-Haddad |
| Year of birth | 1984 | 1984 or 1985 |
| Country | Not provided | Multiple source values |
| Identifier | In the evidence store | Not in this source record |
Inspect aliases and the full source records. Resolve the missing attributes and record the basis for the decision.
Matching and recovery tests
Matching
- Exact identifiers, aliases, reordered names, and transliteration.
- Common names with conflicting dates or countries.
- Incomplete records, stale profiles, and corrected identities.
- Business names, legal suffixes, and ownership links.
- Records added, changed, removed, and added again.
Recovery and review
- Change a list record after a customer was cleared.
- Change customer details after a false-positive suppression.
- Interrupt the feed and verify alarms and recovery.
- Replay a decision with its original list snapshot.
- Test who can clear, override, or change a rule.
Ownership, geography, services, and transaction context can require separate analysis. See OFAC’s potential-match guidance for the U.S. context.
10 of 10 areas
Scope & product boundariesProduct
- Name the exact products, modules, data sources, and licenses in this proposal.
- Which capability is native, partner-provided, custom work, or on a future roadmap?
- Who owns the control, contract, support ticket, and service level for each partner dependency?
A signed scope map that connects each required control to a current SKU and accountable provider.
Data ingestion & reconciliationEngineering
- Show the schemas for entities, ownership, transactions, decisions, cases, and corrections.
- How do you handle duplicates, late events, backfills, source outages, and changed customer records?
- Can we reconcile accepted and rejected events to our ledger and replay a historical decision?
API schemas, a sample reconciliation report, and a recorded replay using the pilot dataset.
Detection & tuningProduct
- Which of our risks require rules, models, graph analysis, or external data?
- Can we backtest a change against a locked dataset before release and roll it back?
- Show quality by customer segment, payment rail, country, and typology, with sample counts.
Versioned rules, validation results, explainable triggers, and a change approval log.
Screening coverage & freshnessCompliance
- What are the list and ownership sources, coverage gaps, and source rights?
- What elapsed times are measured from source update to customer rescreening?
- How do you test aliases, scripts, sparse identity data, and the expiry of false-positive suppressions?
Source inventory, update timestamps, matching benchmark, and rescreening failure recovery.
AI authority & evidenceAI & governance
- What exact tasks use ML, generative AI, or agents, and what may each system change?
- Can each factual AI statement link to accessible source evidence? What happens when evidence is missing?
- Show prompt, model, and retrieval versioning; tenant isolation; training-data terms; and human override.
- How do you test prompt injection, incorrect citations, permission escalation, and silent model changes?
Task-level permission matrix, evaluation set, error analysis, and a complete action trace.
Investigations & reportingCompliance
- Can we export the full case, linked evidence, decision history, and reviewer edits?
- Which reporting forms and jurisdictions are supported, and what submission path is included?
- Show rejection, correction, resubmission, receipt, confidentiality, and access controls.
A sample closed case and end-to-end reporting demonstration with sensitive data removed.
Runtime & failure behaviorEngineering
- What p50, p95, and p99 latency applies to our payload, region, and sustained load?
- How do timeouts, retries, rate limits, repeated requests, and a regional outage behave?
- Who operates the fallback, queues delayed work, resolves conflicts, and confirms recovery?
Measured load results, service objectives, incident history, and a completed recovery drill.
Security, privacy & governanceAI & governance
- Show SSO, role design, service account scopes, tenant boundaries, and audit export.
- Where is data stored and processed, who can access it, and which subprocessors receive it?
- What retention, deletion, encryption, incident notice, and model-training terms will be in the contract?
Current assurance reports, architecture and data-flow diagrams, contract terms, and access tests.
Implementation & operating costCommercial
- What data preparation, historical migration, rule conversion, and specialist work is excluded?
- Which internal roles and hours are needed for launch, tuning, and ongoing support?
- Price normal usage, peak volume, manual reviews, extra environments, storage, API retries, and partner calls.
A work breakdown with acceptance criteria, staffing assumptions, and an itemized three-year cost model.
Portability & exitCommercial
- Can we export rules, cases, evidence, audit logs, models where licensed, and historical decisions?
- What formats, fees, time limits, and assistance apply at termination?
- What continues to work if an external data source, model provider, or subcontractor changes?
A tested bulk export, data dictionary, transition plan, and contractual exit terms.
Maintain an evidence register
For each answer, record the claim, product version, source, reviewer, test result, open dependency, and contract commitment.
- A marketing statement is a claim to test.
- Documentation and scope define the contracted capability.
- A demonstration shows behavior on a known case.
- Your pilot establishes behavior on your data and controls.
A roadmap item remains unproven until you can inspect the delivered behavior.
Day 30Prove the inputs
Replay / shadow
- Confirm scope, owners, and decision authority.
- Load a representative, access-controlled dataset.
- Reconcile source counts, amounts, and rejected events.
- Lock the test set and baseline measures.
Exit when data defects are resolved and the baseline can be reproduced.
Day 60Prove the decisions
Controlled evaluation
- Compare results on the same labeled cases.
- Test difficult negatives, missing data, and edge cases.
- Measure reviewer time and unsupported AI statements.
- Run permission, outage, and replay drills.
Exit when quality is understood by segment and critical controls pass.
Day 90Prove the operation
Limited rollout, if approved
- Use a defined cohort with an approved fallback.
- Verify actions in the real payment and case systems.
- Observe queues, freshness, exceptions, and cost.
- Approve rollout, extend the pilot, or stop.
Exit with signed evidence, operating owners, and a tested recovery plan.
Evidence score 0/100
0 of 7 areas assessed · 0 of 4 mandatory gates passed
Data integrity20%Mandatory
Measure and criteria
Required records reconciled; schema, identity links, amounts, and event states tested.
Zero unexplained reconciliation gaps in the agreed test set. Every rejected event is visible and recoverable.
Detection quality20%
Measure and criteria
Recall on labeled known cases; precision on reviewed alerts; false positives by segment.
Meet the pre-agreed baseline or improvement target on a locked, representative sample. Show sample size and uncertainty.
Screening controls15%Mandatory
Measure and criteria
Coverage, tested match recall, difficult-negative results, and end-to-end update lag.
Pass the agreed critical match cases. Every source has a freshness measure, owner, and tested recovery path.
Analyst operation15%
Measure and criteria
Median and p90 review time; rework; queue age; evidence completeness; reviewer agreement.
Meet agreed queue and handling-time targets without lower review quality. Each sampled decision has a traceable rationale.
AI & action authority10%Mandatory
Measure and criteria
Factual support, incorrect citations, permission tests, overrides, and action history.
Zero unauthorized actions in the test suite. Unsupported outputs are visible; consequential actions follow the approved authority model.
Resilience & latency10%Mandatory
Measure and criteria
p95 / p99 latency at agreed load; timeout behavior; replay, rollback, and restore.
Meet the flow-specific latency and recovery budgets. Pass each required outage and repeated-request drill.
Cost & portability10%
Measure and criteria
Total cost per evaluated event and reviewed case; implementation effort; export completeness.
Fit the agreed budget at normal and peak usage. Successfully export the agreed evidence and operational records.
Partial credit earns half the weight. A score cannot override a mandatory gate. Ratings remain in this page session; export to keep a record.
Test design and the final decision
Keep the comparison fair
- Use the same population, period, outcomes, and review rubric.
- Separate tuning data from the final evaluation sample.
- Report segment results and sample counts.
- Track unknown outcomes and label quality.
- Include staff effort, partner fees, and support.
Approve, extend, or stop
- Approve: criteria met, owners assigned, scope and fallback signed.
- Extend: a specific gap has an owner and a bounded test.
- Stop: a critical gap remains or the cost does not fit.
- Retain the evidence and rationale.
- Revalidate when scope, models, data, or controls change.