AI Implementation Checklist: Stage by Stage

Most AI implementation checklists are organized by category: strategy, data, governance, security, talent.

That structure is easy to read and nearly impossible to use. Categories do not tell you when to do things or what happens if you get it wrong.

This checklist is organized by stage. Each stage has a specific set of items to complete before moving to the next.

Each stage also has failure signals: the specific conditions that mean the implementation is heading toward a problem that will be expensive to fix later.

Use this checklist for an organization-wide AI implementation. For a single-tool deployment, collapse stages 2 and 3 and skip the multi-workflow items in stage 5.

Stage 1: Business Case and Scope

Before any technical work begins, the business case for AI needs to be defined at a level of specificity that survives contact with the finance team.

Vague goals (“improve operations with AI”) produce vague implementations that fail at the adoption stage.

Stage 1 Checklist

• Specific problem defined. The problem AI is solving is stated as a measurable outcome: “reduce invoice processing time from 4 days to 1 day for the AP team” not “improve efficiency.”
• Executive sponsor identified. A named executive owns the outcome and has budget authority. A steering committee without a single owner is not an executive sponsor.
• Baseline metrics established. The current state is measured. You cannot demonstrate ROI without a before number.
• Scope bounded. The first implementation targets one workflow, one team, or one function. AI implementations that start with “the whole company” collapse under their own weight.
• Success criteria agreed. The specific metric that defines success is agreed on before any vendor is contacted. This prevents goal-post movement after launch.
• Risk appetite assessed. The organization has discussed what happens if the AI system makes an error in this workflow. Is the output reviewed by a human before action is taken? Who is accountable for errors?
• Build-vs-buy decision made. Custom AI development, off-the-shelf AI tools, and embedded AI consulting are three different paths with very different cost structures and timelines.

Stage 1 failure signals:

• The problem statement changes every time you ask a different stakeholder what AI is supposed to solve
• No one can name the specific metric that would prove the implementation succeeded
• The scope includes more than two workflows in the initial phase

Stage 2: Data Readiness

AI systems are only as good as the data they run on. Data readiness is consistently underestimated and is the most common reason AI pilots fail to reach production.

Stage 2 Checklist

• Data inventory completed. You know where the relevant data lives: which systems, which formats, who owns it, and how it is currently accessed.
• Data quality assessed. The data has been evaluated for completeness, consistency, and accuracy. Missing fields, duplicate records, and inconsistent formats are quantified, not assumed.
• Data access confirmed. The teams and systems that need to use the AI output can actually access the underlying data. This includes checking permissions, API availability, and data residency requirements.
• PII and sensitive data mapped. Any personally identifiable information, protected health information, or other sensitive data in the relevant dataset is identified and a handling policy is in place before any AI system is trained or fine-tuned on it.
• Data pipelines documented. How data flows from source systems to the AI system is documented and tested. Pipelines that break silently are one of the most common sources of AI system degradation after go-live.
• Retention and deletion requirements addressed. If the AI system stores or processes data, the organization knows how long that data is retained, who can access it, and how it is deleted when required.
• Historical data volume confirmed. There is sufficient historical data to train, fine-tune, or provide meaningful context for the AI system. “Sufficient” depends on the use case and the model approach, but this is evaluated, not assumed.

Stage 2 failure signals:

• No one person can describe the complete data flow from source to AI system output
• Data quality assessment has not been done; the assumption is that “the data is fine”
• The relevant data lives in a system that cannot be accessed via API or direct query

Stage 3: Infrastructure and Security

Stage 3 Checklist

• Deployment model selected. SaaS, private cloud, on-premises, or hybrid. The deployment model is chosen based on data sensitivity, regulatory requirements, and total cost of ownership, not just convenience.
• Model hosting decision made. Which AI models will be used (commercial API, open-source self-hosted, or proprietary fine-tuned), and where inference will run.
• Access controls designed. Who can use the AI system, what they can do with the output, and how access is provisioned and revoked. Role-based access control is documented before deployment.
• Authentication integrated. The AI system is connected to the organization’s SSO and identity provider. This is consistently the largest single technical task in enterprise AI deployment; it cannot be deferred to after launch.
• Audit logging enabled. Every significant action the AI system takes or recommends is logged. The log is tamper-resistant and accessible for compliance review.
• Data residency confirmed. The organization knows which country or region the AI system stores and processes data in. This matters for GDPR, HIPAA, and US state privacy laws.
• Vendor security reviewed. If a third-party AI vendor is used, their security posture has been reviewed: SOC 2 Type II report, data processing agreement, subprocessor list, and incident response commitments.
• Incident response plan updated. The existing incident response plan covers AI-specific scenarios: a model producing harmful output, a data exfiltration through a prompt injection, or an AI agent taking an unintended action.

Stage 3 failure signals:

• SSO integration is listed as “to be completed after launch”
• The vendor cannot provide a current SOC 2 Type II report
• Audit logging is not included in the deployment plan

Stage 4: Governance and Policy

Governance that is written after a problem occurs is remediation, not governance. The policies in this stage need to be in place before the AI system goes live with real users.

Stage 4 Checklist

• Acceptable use policy published. A written policy defines what employees may and may not do with AI tools: what data can be entered into AI systems, what outputs can be used without human review, and what uses are prohibited.
• AI output review requirements defined. For each workflow the AI system touches, there is a defined standard for when AI output requires human review before action is taken. High-stakes outputs (financial decisions, hiring decisions, customer-facing communications) require review; low-stakes outputs (internal draft summarization) may not.
• Vendor AI use policy reviewed. If employees are using AI tools that send data to third-party models, the organization has reviewed how those vendors use input data for model training. Most major vendors offer data opt-out for enterprise contracts; confirm it is enabled.
• Model documentation maintained. The organization knows which AI models are in use, what version, what training data they were trained on (to the extent the vendor discloses), and what the model’s known limitations are.
• Bias and fairness review conducted. For any AI system that affects people (hiring, lending, customer service prioritization, performance evaluation), a bias and fairness review has been conducted before go-live.
• Escalation path defined. When an employee disagrees with an AI system’s output or suspects an error, there is a defined escalation path. This path is documented and communicated before launch.
• Regulatory compliance reviewed. The legal team has reviewed the AI implementation for compliance with applicable regulations: EU AI Act (if the organization has European operations or customers), CCPA/CPRA, HIPAA (if health data is involved), and any sector-specific requirements.

Stage 4 failure signals:

• There is no written acceptable use policy for AI tools
• The legal team has not been involved in the implementation
• Employees do not know who to contact if they think the AI system is producing incorrect output

Stage 5: Pilot

The pilot is not a demo. It runs with real users, real data, and real workflows.

The purpose is to surface integration failures, adoption barriers, and output quality issues before the system is deployed at scale.

Stage 5 Checklist

• Pilot scope defined. A specific team, workflow, and time period is selected. The pilot is not “a few people try it” but a structured test with defined participants, defined usage expectations, and defined measurement criteria.
• Baseline measurement confirmed. The before metrics from Stage 1 are captured for the pilot team specifically, so the pilot improvement (or lack of improvement) can be measured accurately.
• Feedback mechanism in place. Pilot users have a defined way to report problems, confusion, or disagreements with AI output. This is not a suggestion box; it is a structured feedback process with a defined review cycle.
• Output quality monitored. Someone is reviewing AI outputs during the pilot period, not just collecting user satisfaction scores. Hallucinations, retrieval failures, and output drift need to be identified and documented.
• Integration tested under load. The AI system has been tested with the actual data volume and query patterns of the target workflow, not just with sample data in a demo environment.
• Adoption tracked. The organization is measuring whether pilot users are actually using the system, not just whether they have access to it. Low adoption during the pilot is a signal to investigate before scaling.
• Go/no-go criteria defined. Before the pilot starts, the specific criteria for proceeding to full deployment are agreed on. Pilots that have no exit criteria almost always proceed to deployment regardless of results.

Stage 5 failure signals:

• The pilot group is self-selected (only enthusiastic volunteers)
• No one is reviewing actual AI outputs during the pilot period
• The pilot is declared a success based on user satisfaction scores alone, without measuring the workflow outcome metric from Stage 1

Stage 6: Rollout and Training

Stage 6 Checklist

• Training delivered before access. Employees receive training on the AI system before they are given access, not after. Training covers both how to use the system and the acceptable use policy from Stage 4.
• Change management addressed. The rollout plan addresses the human side of the implementation: what changes about daily work, what does not change, and what support is available. Implementations that skip this consistently have lower adoption.
• Rollout sequence planned. The deployment follows a defined sequence: which teams or functions go live in which order, and why. Simultaneous organization-wide deployment without a sequence is high-risk.
• Rollback plan documented. If the system produces a critical error or the output quality degrades significantly after a model update, there is a documented plan for reverting to the previous workflow state.
• Support structure in place. There is a defined internal team or external partner that handles post-launch questions, bugs, and escalations. “Submit a ticket to IT” is not a support structure for AI systems.
• Communication sent. Employees who are affected by the AI implementation have been communicated to before go-live. Surprises destroy adoption.

Stage 6 failure signals:

• Training is scheduled for after the system goes live
• No rollback plan exists
• The implementation team dissolves immediately after launch day

Stage 7: Monitoring and Ongoing Governance

An AI system that is not monitored after go-live will degrade.

Models update, data distributions shift, user behavior changes, and the gap between what the system was designed to do and what it is actually doing widens over time without active monitoring.

Stage 7 Checklist

• Output quality monitored on a schedule. A defined team reviews AI output quality on a regular cadence (weekly for high-stakes applications, monthly for lower-stakes ones) against the baseline established in Stage 1.
• User feedback reviewed. The feedback mechanism from the pilot continues in production. Feedback is reviewed and acted on with a defined response time.
• Model version changes tracked. When an underlying model is updated by a vendor, the organization is notified, the change is logged, and output quality is spot-checked before the update is accepted into production.
• Usage metrics reviewed. Adoption, query volume, error rates, and cost per query are tracked and reviewed. Unexpected spikes or drops in any of these are investigated.
• Policy reviewed annually. The acceptable use policy and governance framework are reviewed at least annually, and any time a significant new AI capability is added to the implementation.
• Incident log maintained. Every significant AI system error, unexpected output, or policy violation is logged, investigated, and documented.
• ROI measured against Stage 1 baseline. The outcome metric defined in Stage 1 is measured at 30, 90, and 180 days post-launch and compared to the baseline. If the metric is not moving, the implementation is investigated before it is accepted as working.

Stage 7 failure signals:

• No one owns ongoing AI system monitoring after the implementation team disbands
• Model version changes are applied automatically without output quality review
• The ROI measurement from Stage 1 has never been revisited

Getting the Implementation Right

Phos AI Labs is an embedded AI consulting firm that works with US businesses with $5M+ revenue.

We build AI strategy, install the foundations, train teams, and stay through implementation until AI is actually running the workflows it was designed to run.

Phos AI Labs is one of the first 10 OpenAI Select partners worldwide and one of the first Anthropic partners with CCA-F certification.

Our team of 10+ CCA-F certified forward deployed engineers has completed 400+ builds.

Two paths:

• Path one: if you are in Stages 1 through 3 and need help building a strategy and scoping the right first implementation, we start with an AI Readiness Audit that identifies your specific failure risks before any engineering begins.
• Path two: if you are past scoping and ready to build, we embed with your team and deliver the implementation, governance framework, and post-launch monitoring infrastructure.

Engagement pricing:

• AI Readiness Audit: from $10,000
• Ongoing embedded delivery: from $15,000/month
• Full embedded AI department: up to $50,000/month

No self-serve signup. All engagements scoped on a call.

Talk to the team at Phos AI Labs.

FAQs

How Long Does an AI Implementation Take?

A single-workflow AI implementation typically takes eight to sixteen weeks from scoping through go-live. Organization-wide implementations typically take twelve to twenty-four months.

The timeline is driven primarily by data readiness and integration complexity.

What Is the Most Common Reason AI Implementations Fail?

The three most common failure modes are: unclear success criteria, underestimated data quality issues, and low adoption after launch.

All three are preventable with Stage 1 and Stage 5 work.

Do Small and Mid-Market Companies Need All Seven Stages?

Yes, but scope scales to the organization. A mid-market company deploying one AI tool can complete all seven stages in eight to twelve weeks.

The stages do not change. Failure signals apply regardless of size.

What Should Be in the Scope of the First AI Implementation?

The first AI implementation should target a single workflow with a measurable outcome, a willing team, and clean data.

The goal is not maximum ROI. It is learning how AI deployment works inside your organization.

How Do You Measure ROI on an AI Implementation?

Measure the operational metric defined in Stage 1 at 30, 90, and 180 days post-launch. Compare to the baseline from before implementation.

Common metrics include time per task, error rate, throughput volume, and hours saved.