A CI/CD pipeline turns a reviewed commit into a tested immutable artifact, then promotes that same artifact through environments under explicit policy. It should use isolated jobs, short-lived credentials, reproducible dependencies, layered tests, protected production approval, observable rollout, and a recovery plan for code, configuration, and data.
Who this is for: Development and platform teams replacing manual builds and deployments with a controlled, repeatable delivery pipeline.
- Build once and promote the same artifact digest so later environments test the bytes production receives.
- Grant each job only the credentials and network access needed for its specific stage.
- Make deployment health and recovery part of the pipeline instead of stopping at a successful upload.
Map the path from commit to customer
Begin with triggers, required reviews, build inputs, artifacts, environments, deployment mechanisms, database changes, and owners. Define which branch or tag can start a release and how the source revision is identified. Pull-request checks should provide fast feedback, while protected release stages can run broader tests and policy checks.
Draw trust boundaries around source code, third-party dependencies, runners, artifact storage, cloud accounts, and production. A pipeline is part of the software supply chain and can become a privileged attack path. Restrict who can edit workflow files, pin reusable actions or tools according to platform guidance, and review dependency provenance.
Create one verifiable artifact
Use locked dependencies and an isolated build environment. Run formatting, static analysis, unit tests, and artifact construction in a deterministic order. Produce a container digest or versioned package plus metadata linking it to the commit, build identity, dependencies, and test results. Do not rebuild from the branch separately in every environment.
Store artifacts in a restricted registry with retention and immutability controls. Scan and sign or attest them when policy requires it. Define how findings are evaluated and exceptions expire. Secrets must enter only the jobs that need them and must never be printed, embedded in artifacts, or exposed to untrusted pull-request code.
Promote through meaningful gates
Deploy the artifact to a test environment and run integration, contract, migration, and user-journey checks suited to the system. A staging environment is valuable when it reproduces critical interfaces and controls, not merely because it has a production-like name. Keep test data synthetic or approved and reset it safely.
Production should require branch protection, successful checks, and accountable authorization proportionate to risk. Prefer workload identity or short-lived credentials bound to an environment. Pin account, Region, cluster, namespace, and artifact explicitly in deployment jobs. Show these destinations in the run summary before mutation so an operator can catch wrong-environment configuration.
Observe deployment and recover
A deployment job should wait for rollout health and evaluate service indicators, not report success when an API merely accepts a manifest. Use progressive exposure where appropriate, with stop conditions based on errors, latency, saturation, and critical business outcomes. Record the release marker in observability systems for correlation.
Recovery may mean routing back to the prior artifact, rolling forward a fix, restoring configuration, or completing a compatible database repair. Practice the path. Database migrations need expand-and-contract sequencing because old code may run during rollout or rollback. Preserve pipeline evidence and require an incident review when automated safeguards fail.
Deliver a containerized API
A team currently builds locally, copies an image tag to staging, and manually redeploys the same tag in production.
- Trigger on a reviewed commit, install locked dependencies, run tests, and build one image whose digest is recorded with the commit.
- Scan and store the immutable digest, then deploy it to staging with short-lived staging credentials and run API plus migration checks.
- Require protected production approval that displays the exact account, cluster, namespace, commit, digest, and plan of change.
- Deploy progressively, watch service indicators, and route back to the prior digest if an agreed stop condition is crossed.
Pipeline stage contract
Define this contract for each job rather than granting the pipeline one broad identity.
- Trigger, source revision, protected files, runner trust, dependencies, and expected inputs.
- Command, checks, timeout, network access, secrets, identity, and least-privilege permissions.
- Artifact digest, provenance, test evidence, scan result, retention, and promotion rule.
- Exact environment, account, Region, cluster, namespace, approval, and mutation preview.
- Rollout indicators, stop conditions, recovery action, audit record, and accountable owner.
Common mistakes
- Rebuilding from the same branch in production and assuming it produces the artifact tested in staging.
- Giving pull-request jobs access to production credentials or broadly trusted persistent secrets.
- Declaring deployment success before readiness and user-facing service indicators are checked.
Try one
A pipeline deploys the tag latest to production after tests pass against latest in staging. Explain the integrity problem and redesign the promotion.
The tag can point to different bytes at different times, so passing staging tests does not identify production content. A strong answer builds once, records an immutable digest, tests that digest, and promotes it explicitly with provenance and destination checks. It retains the prior digest and observes rollout health for recovery.
Sources
- AWS CodePipeline documentationOfficial AWS overview of staged continuous delivery workflows, approvals, and release automation.
- AWS CloudFormation change setsOfficial AWS guidance for previewing infrastructure changes before execution.