CI/CD Best Practices That Cut Deployment Time by 70%

Slow CI/CD pipelines are a tax on every engineer, every day. A 45-minute pipeline does not just cost 45 minutes — it breaks focus, discourages small commits, and creates merge queues that serialise the entire team.

We took our main pipeline from 45 minutes to 13 minutes without changing the test suite or skipping any checks. Here is exactly how.

Identify Your Bottleneck First

Before optimising anything, instrument your pipeline. GitHub Actions has built-in step timing. Buildkite has a detailed analytics view. Find the single slowest step and start there. In our case, it was the Docker build — 22 minutes for a full rebuild every time.

Layer Caching Aggressively

Docker layer caching is the highest-leverage optimisation for most teams. The key insight: your application dependencies change far less frequently than your application code. Structure your Dockerfile to copy dependency files first, install dependencies, then copy application code:

COPY package*.json ./
RUN npm ci --only=production  # This layer is cached until package.json changes
COPY . .                      # Only this layer rebuilds on code changes

Combine this with a remote cache registry (GitHub Packages, ECR, or Artifact Registry) so cache hits work across different CI runners.

Parallelise Test Suites

Most test suites can be split into parallel jobs with no code changes. GitHub Actions matrix strategy, Jest’s --shard flag, and pytest-xdist all support this. We split our 800-test suite across 4 parallel jobs, reducing test time from 18 minutes to 5 minutes.

Fail Fast

Run your fastest checks first: linting, type-checking, and unit tests before integration tests and Docker builds. If the linter fails, there is no point building a Docker image.

A 30-second lint failure caught before a 20-minute Docker build saves 20 minutes every time. Ordering matters as much as speed.

Skip Unchanged Services

In a monorepo, detect which services were modified and only build and test those. Tools like Nx, Turborepo, and Bazel have first-class support for affected-only builds. This alone cut our pipeline time by 40% on the average PR.

The Results

Layer caching: -22 min. Test parallelisation: -13 min. Fail-fast ordering: -5 min. Affected-only builds: -11 min average. Total reduction: 51 minutes → 13 minutes. Deployment frequency increased 3× in the following quarter.