Catch Breaking API Changes in Pull Requests

To catch a breaking API change in a pull request, compare the OpenAPI spec on the branch against the spec on the base branch, and fail the check when the diff contains a breaking change. oasdiff does this comparison, and its GitHub Action is documented as a way to block pull requests that introduce breaking API changes. This post gives the workflow file, the severity setting to choose, and the branch protection rule that turns a red check into a blocked merge.

The procedure in four steps

The procedure has four steps. Run a spec diff between the base branch and the pull request branch. Choose the severity at which the diff fails. Trigger the job on the pull_request event. Mark the job as a required status check so the merge waits for it.

The tool in this post is oasdiff, a command-line tool to compare and detect breaking changes in OpenAPI specs. Its documentation on breaking changes says the commands are typically used in CI to report or prevent breaking changes.

The method has a boundary worth stating up front. A spec diff compares two descriptions of an API, so it can only report what the description says. It reads a removed endpoint or a new required field. It cannot read a response that changed behind an unchanged schema, which is the job of the request-level checks in REST API testing, step by step. The last section covers that gap.

What counts as a breaking change

oasdiff sorts every change into one of three levels. ERR marks definite breaking changes that should be avoided. WARN marks potential breaking changes that cannot be confirmed programmatically. INFO marks non-breaking changes. The oasdiff breaking command reports changes at the ERR and WARN levels only.

One rule surprises people. A change that makes a previously valid request invalid is breaking even when your server would still accept the request. Adding a required request property is the standard example: a request that omits it is invalid under the new contract, whether or not the property has a default. So a pull request that adds currency as a required field fails the check even if the handler falls back to a default value.

Two commands cover related ground. oasdiff breaking displays the changes that break existing API clients. oasdiff changelog displays changes that can affect API consumers, both breaking and non-breaking, and leaves out documentation-only edits such as descriptions and examples. Use breaking to gate a merge and changelog when you want a readable release note of what changed.

Add the GitHub Actions check

Add the workflow to .github/workflows/oasdiff.yaml. The oasdiff-action quick start gives it as the way to block pull requests that introduce breaking API changes. Replace openapi.yaml with the path to your spec.

name: oasdiff
on:
  pull_request:
    branches: [ "main" ]
permissions:
  contents: read
  pull-requests: write
jobs:
  breaking-changes:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - run: git fetch --depth=1 origin ${{ github.base_ref }}
      - uses: oasdiff/oasdiff-action/breaking@v0
        with:
          base: 'origin/${{ github.base_ref }}:openapi.yaml'
          revision: 'HEAD:openapi.yaml'
          fail-on: WARN
          github-token: ${{ github.token }}

The base input is the old spec and the revision input is the new one. Both accept a file path, a URL or a git ref, and the workflow points base at the spec on the base branch as a git ref. The job compares your spec on the pull request branch against the base branch and fails if it finds breaking changes.

The failure is easy to read. The action writes inline ::error:: annotations on the pull request’s Files changed tab, and it fails the workflow when changes at or above the fail-on severity are found. The author sees which line of the spec broke the contract without opening the job log.

Turn the failing check into a merge gate

A red check does not stop a merge by itself. A run that reports a failure and lets the merge through is a notification, and notifications get merged past. Blocking the merge is branch protection, the second setting covered in keeping API tests current with code changes: check the box for Require status checks to pass before merging, then select the job by name, here breaking-changes. Until that box is ticked, the check is advisory.

Set it once on the base branch. After that, a breaking change in the spec has two ways forward: the author reverts it, or the team decides the break is intended and changes the severity policy or ships a new API version.

Run the same diff locally and choose a severity

Run the same comparison on your machine before you push, so the first failure is not in CI. The --fail-on flag of oasdiff breaking sets the exit code.

The flag has three settings. --fail-on ERR exits with return code 1 if ERR-level changes are found. --fail-on WARN exits with return code 1 if ERR or WARN changes are found. --fail-on INFO exits with return code 1 if any changes are found.

Which one to pick is a policy decision. Failing on ERR blocks only the definite breaks and keeps noise low. Failing on WARN also catches changes the tool cannot confirm, which means a human reads more warnings. Teams adopting the check on an existing API can start at ERR and tighten later.

What a spec diff misses

A spec diff compares two descriptions, so a pull request that changes behaviour and leaves the spec alone passes it. A handler that starts returning a different value for the same field, a validation rule added in code, or a change in how two endpoints interact are not visible in the diff.

Those changes need integration tests that exercise the running API, the same kind of check described in running API tests on every pull request. Stresseur is built for that side of the problem: it records how an API is really used during development, turns those flows into YAML tests, and runs them in CI on every pull request. Keep the spec diff in place and add these tests next to it, starting with a short set as in API smoke tests on every pull request. Contract-level checks are covered in turning an OpenAPI spec into contract tests. The diff catches contract breaks cheaply, and tests built from real usage catch the changes the contract never mentions.

Conclusion

Catching breaking changes in a pull request takes one workflow file and one repository setting. Diff the spec against the base branch with oasdiff, pick ERR or WARN as the failing severity, and require the job as a status check so the merge cannot proceed past it. Then cover the behaviour a spec cannot describe with tests that run against the API itself, such as the approaches in testing an API without writing test code.