API Automation Testing: What to Automate First

Automate the paths that would hurt most if they broke, and automate them in this order: login, the money paths, create-then-read checks, the endpoints a pull request touches, error responses, and only then the rest. A short set that runs on every pull request beats a large set nobody trusts. This post gives the order, explains what to leave out, and shows how to run the set in CI.

What to automate first, in order

API test automation is the practice of using a testing tool to execute API tests programmatically, at set times or in CI/CD pipelines, as Postman defines it. If you want the basics first, start with what API testing is and the request-by-request walkthrough in REST API testing, step by step. The question here is narrower: with limited time, which tests come first?

1. Login and token issue. Every later request depends on it.
2. The money paths. Sign-up, checkout, the action customers pay for.
3. Create, then read it back. Each write is followed by a read that must show the new item.
4. Endpoints the current pull request changes. The place where new bugs enter.
5. Error responses. A wrong password, a missing field, an expired token.
6. Everything else, in the order users hit it.

Treat the list as a budget. Each tier should be passing and running in CI before you start the next, because a half-built set at every tier protects less than a finished set at the first three.

Testsigma lists what an automated API test checks as status codes, schema, data and performance, and the first tiers need only some of those.

Stop at the point where adding another test costs more upkeep than the bug it would catch.

Why login and the money paths come first

A flow is a sequence of requests in which each step feeds the next. The login returns a token and every later request sends it. When login breaks, the rest of the set fails for one reason, so a login test that fails clearly is worth more than twenty that fail together. For the shortest version of the first request and the checks that follow it, see how to test an API.

Money paths come second because their failure costs the most. They also tend to be flows made of several calls, not single calls: sign up, log in, create an order, read the order.

A status code alone is a weak check. Stresseur generates flows that check each answer as well as the status, and that send fresh sign-up values on each run. It also generates the read-back check: after a create with POST, PUT or PATCH, a later read of the list or page must hold the new id, or a value the user typed. A create call can return 200 and store nothing, and only the read-back catches that.

A first set for a small shop API

Take a shop with sign-up, a cart and checkout. A first automated set of five flows covers most of the risk:

1. Sign up, log in, and read the profile with the token you received.
2. Add an item to the cart, then read the cart and confirm the item is in it.
3. Check out, create the order, then read the order back.
4. Log in with a wrong password and confirm the request is refused.
5. Hit the one endpoint the open pull request changes.

Each flow is short and has one reason to fail. The first three map to tiers one to three. The fourth is the error case, and the fifth changes with every pull request. If you want more about how such multi-step flows pass a token between steps, see testing multi-step API flows with auth tokens. For a longer walk through what these checks look like against a REST API, see integration testing examples for a REST API.

What not to automate

Leave out what costs more to maintain than it protects. Automation is a trade of upfront time for repeated checking, and that trade only pays on tests that run often and stay valid.

• One-off exploration. Poking at a new endpoint by hand is how you learn it. Automate it once you know what correct looks like.
• Endpoints that change every day. A test that is rewritten daily gets switched off. Wait until the contract settles.
• Third-party services you do not own. Test that your code handles their answers, not that their service works.
• Every permutation of every field. Cover the cases that have broken before, and add a case when a bug teaches you one.
• Tests you would have to write by hand when a recording would do. Record the flow, then edit it where it needs to differ. The related question of how to keep recorded tests current as the code changes is covered in how to keep API tests current.

One limit applies to Stresseur itself. Writing missing flows from the AI test engineer, with a request like “add tests for this”, is not available yet. Record the flows you need instead.

How to automate the first set

The fastest route is to record real use of the app instead of writing requests from scratch. Stresseur is an open-source CLI, run as npx stresseur, that works with or without a workspace.

npx stresseur init finds the app, starting its dev script if needed, records it in a fresh browser window, and generates the flows. Click through login and your main path, then press Done. Apps that build their pages on the server, such as Flask, Django, Rails and Next.js server actions, work too, because recording also captures page loads and form posts.

The flows land in your repository under api-tests/ as *.yamlflow.yaml files. They are plain YAML, so they are reviewed in a pull request like code. Any recorded flow can also run later as a load test. For how this fits a wider test strategy, see API testing with Stresseur.

Run the set on every pull request

Automated tests only help if they run without anyone remembering to run them. init-ci writes the GitHub Actions workflow.

In CI, the workflow runs each flow 13 times, one warm-up and 12 measured, and uploads per-step timings. Each pull request then gets one check named “Stresseur / performance” and one comment, updated in place on each push, so a long-lived branch gets one comment, not a pile of them.

The larger guide to the workflow is API tests in GitHub Actions. A flow is flagged when its median is at least 10% slower than the baseline and outside its normal run-to-run variation. The baseline is the last 30 comparable runs on the default branch. On the Free plan, pull request checks and slowdown detection warn only and do not fail the check.

A smoke-level subset is worth separating out, and API smoke tests on every pull request shows how to run it on its own. Read smoke test vs integration test to decide which flows belong in a fast gate on every push and which belong in a slower set.

Conclusion

Start small and start with the paths that cost the most when they break: login, the money flows, and a read-back after every write. Skip what you would only maintain. Record the first set instead of writing it, commit it to the repository, and let CI run it on every pull request. Add the next test when a real failure shows what was missing.