Unit Test vs Integration Test: What Each Catches

Unit tests vs integration tests: what each one proves, what it misses, and what 8 real bugs showed when we ran 2,189 unit tests and one API flow.

Mustafa BayramogluMustafa BayramogluAbstract illustration of many small isolated points on one side and a single connected path on the other

A unit test checks one piece of code on its own, with its neighbors faked. An integration test checks that pieces work once they are connected: routes, auth, config, the database, and the shape of the data passed between them. Unit tests are cheap and point at the exact line. Integration tests catch the bugs that live between the lines (see what integration testing is). When we ran 2,189 unit tests and one API flow test against 8 ordinary bugs, the unit tests caught 5 and the flow caught 6, and each caught something the other missed. You want both. This guide covers what each one proves, what each one misses, what they cost to run, and how many of each to write.

Unit tests vs integration tests at a glance

A unit test asks “does this function do the right thing?” An integration test asks “do these parts work together?”

Unit test Integration test
What it tests One function, class or module Two or more parts, connected
Its neighbors Often replaced with mocks or stubs Real, or a faithful stand-in
Speed Fast: no network, no real database Slower: real I/O and setup
What breaks it A change to the code under test, or to how it is called A change anywhere along the path, including wiring and config
Who writes it The developer who writes the code Developers, sometimes with QA

The Microsoft engineering playbook lays out the same split across unit, integration, system and end-to-end tests. The short version: unit tests sit close to the code, and integration tests sit close to how the code is used. The next layer up is covered in integration testing vs end-to-end testing.

What a unit test proves, and what it can’t see

A unit test proves that one piece of logic does what you wrote it to do. It can’t see what happens when that piece meets the rest of the app.

There are two styles. A solitary unit test keeps the real neighbors out, so a fault in another class can’t fail it. A sociable unit test lets it use some real neighbors. Both styles stay inside the process. Neither starts your server and sends it a real request.

That leaves blind spots:

  • Wiring. Middleware order, which router is mounted where, which handler a route points to.
  • Config. Two modules reading two different secrets, or a setting that only exists in one environment.
  • Routes. A path that got renamed on the server but not in the client.
  • Speed. A mocked database answers at once, so a unit test can’t see a change that makes a real request slower. In two slower pull requests, a 199-test unit suite passed while login got about 8 times slower.
  • Serialization. The JSON your API sends out and the JSON your client reads back in. Martin Fowler’s test pyramid guide says to write integration tests for every piece of code that serializes or deserializes data.

The last one is where unit tests most often pass while users get stuck. If a developer renames a field in a response and updates the unit tests that read it, the tests still agree with the code. The client that reads the old field name is not in the test. That is the exact bug that let every unit test pass while nobody could log in.

Unit tests are still the best tool for logic: edge cases, math, parsing, validation rules. They run fast, and when one fails, it names the function. Many of them also end up in a regression suite, and the post on regression testing vs unit testing explains how the two ideas overlap.

What an integration test proves

An integration test proves that parts built separately still work when you connect them. Google’s testing blog describes it as taking a small group of units, often two, and testing their behavior as a whole.

In practice there are three common kinds:

  • Database tests. Your data layer runs real queries against a real (often throwaway) database.
  • Service tests. Your code calls another service, or a stand-in for it, over the network.
  • API-level tests. You start the app and send it HTTP requests, the way a browser or a mobile client would. The request goes through the real routes, middleware, auth and database.

The word “integration” gets used two ways. A narrow integration test checks only the code that talks to one neighbor, often against a test double. Martin Fowler’s note on integration tests draws the line: a broad integration test needs live versions of all the services it touches. Narrow tests are fast and easy to run. Broad tests catch more, and cost more to set up.

API-level tests sit in between. They need your app running, but not every service in your company. For how an integration test differs from a smoke test, see smoke test vs integration test.

Where an API flow test fits

This is what integration tests are for. An API flow test is an API test that sends a sequence of real requests to your running app and checks each response. It tests the connections, which is the job unit tests can’t do.

A flow is more than one request. It logs in, takes the token from the response, and uses it on the next call. It creates a record, reads the ID back, and updates it. The flow in our experiment had seven steps: log in, list categories, list tags, list products, create a category, create a product, update it.

That chaining matters. Most real bugs show up in step two or three, when one response feeds the next request. A single-endpoint check misses them. For more on how tokens and IDs pass between steps, see testing multi-step API flows with auth tokens.

A flow has limits, too. It only fails on what it checks. If a field matters to your users, assert on it.

One bug, two tests

The same bug looks different from each side. Here is the token rename from our experiment, in simplified code.

The server used to answer a login with { "token": "..." }. A commit changes it to { "accessToken": "..." }. The developer updates the unit test that checks the login handler:

test('login returns an access token', async () => {
  const res = await login({ email: 'a@shop.test', password: 'secret' });
  assert.ok(res.body.accessToken);
});

The test passes, and it should. The handler does return accessToken now. The bug is in a different file: the web app still reads res.token, gets undefined, and never stores a session. No unit test covers the gap between the server’s answer and the client’s read.

An API flow test sees it from the client’s side. Step one logs in and saves the token. Step two sends it:

- request:
    name: Login
    method: POST
    url: '{{ #env:APP_URL }}/api/auth/login'
    body: { email: 'a@shop.test', password: 'secret' }
- request:
    name: ListCategories
    depends_on: Login
    method: GET
    url: '{{ #env:APP_URL }}/api/categories'
    headers: { Authorization: 'Bearer {{ Login.response.body.token }}' }
    assertions:
      - response.status == 200

After the rename, Login.response.body.token is empty, so step two goes out with no valid token and gets a 401. The flow fails on the step a user would fail on.

Neither test is smarter. They look from different places. The unit test checks the handler against the developer’s intent. The flow checks the API against how a client uses it.

What 8 real bugs showed

We ran both kinds of test against the same 8 bugs. The unit tests caught 5. One API flow test caught 6. Each caught bugs the other missed, and one bug got past both.

The unit suite was large: 2,189 tests, all passing, covering 99% of the lines in the app’s source. The bugs were ordinary commits, written down before anything ran. The full method and every row are in our unit test experiment. Here is the short version.

Bug Unit tests API flow
Login response renames token, unit tests updated Passed Caught
A product update with no tags wipes the tags Caught Passed
Product list switches field names to camelCase Passed Passed
Total, all 8 bugs 5 of 8 6 of 8

Three rows tell the story.

The rename. The commit renamed the login token and updated the unit tests that read it. All 2,189 tests passed. The web app still read the old name, so nobody could log in. The flow read the token from the real response, sent its next request without one, and got a 401.

The tags. The unit tests caught a bug the flow could not, because the flow always sent tags.

The camelCase change. It got past both. The flow didn’t check that field.

There was a cost on the unit side, too. The big suite went red on three of four refactors that changed nothing a user or client sees. Tests that pin how the app is wired catch wiring bugs, and they also fail when the wiring changes for a good reason.

What each one costs to run

Unit tests are usually cheap per test. A big suite adds up, and an integration test needs the app running. Both fit in a pull request.

In a 2-CPU container sized like a CI runner, the 2,189-test unit suite took 17.6 s, the flow took 0.25 s, and the app started in 0.09 s. Don’t read that as “flows are faster than unit tests.” It compares one big suite with one flow. The unit suite was slow there because its test runner ran one file at a time on 2 CPUs. We don’t claim a speed win.

The bigger cost is upkeep. A test that fails on harmless changes teaches a team to update tests on sight. That habit is how the token rename got through: the commit updated its tests too. When you weigh the two, count the false alarms as well as the run time.

When to write which

Pick the test by where the bug would live. If it lives inside one function, write a unit test. If it lives between two parts, write an integration test.

Write a unit test when:

  • the code has branches, edge cases or math (prices, dates, permissions);
  • you want a failure to point at one line;
  • the code has no I/O, or the I/O is easy to fake.

Write an integration test when:

  • data crosses a boundary: a response a client reads, a row the database stores, a message another service gets;
  • the bug would come from wiring, such as a route path, middleware order or a config value;
  • the behavior depends on a sequence, such as log in, then create, then update.

When you can’t tell, ask what a user would see if it broke. If the answer is “an error on login” or “a blank page”, the bug probably sits between parts, and a unit test with mocks won’t see it.

How many of each: the test pyramid

Write many unit tests, fewer integration tests, and very few end-to-end tests. That is the test pyramid. For where an API layer sits inside it, see the testing pyramid with an API layer.

Martin Fowler’s version of the advice is to write lots of small and fast unit tests, some more coarse-grained tests, and very few high-level tests that test the app from end to end. Google’s testing blog gives a 70/20/10 split, 70% unit, 20% integration and 10% end-to-end, as a good first guess.

Treat the numbers as a starting point. What matters more is what each layer covers. If your API is the product, or several clients depend on it, the middle layer earns more weight. A few API flows that cover your main journeys catch the wiring bugs that a thousand extra unit tests miss.

Integration tests vs end-to-end tests

An integration test checks that a few parts work together. An end-to-end test checks a whole user journey, usually through the real interface.

Broad integration tests and end-to-end tests overlap. Both need live services. The difference is the entry point. An end-to-end test often drives a browser or a mobile app. An API flow test starts at the HTTP layer, which skips the UI. It can run the same journey, such as log in, add to cart, check out, without clicking anything. It won’t catch a button that moved or a page that draws wrong. Keep a small set of browser tests for that, and move the rest of the journey checks to the API layer. The end-to-end testing guide covers both layers, and API vs browser tests shows one journey measured both ways.

Adding API integration tests to a project

You can write API integration tests by hand with any HTTP test library. You can also record them.

Stresseur records how your API is used during development, turns those flows into YAML tests, and runs them in CI on every pull request. npx stresseur record captures the flow while you use your app, and npx stresseur init-ci writes the GitHub Actions workflow. The free plan covers 1 private repo. See API integration tests from real traffic for how it works, or the Stresseur quickstart guide to set it up.

Keep your unit tests. Add integration tests for the connections they can’t see. In our experiment, each side caught a bug the other missed, so the cheapest way to catch more is to run both on every pull request. Start with the one journey your users can’t live without, such as sign-up, login or checkout, and record it as an API flow. Then read what the flow checks, and add an assertion for every field your users depend on.

Frequently asked questions

What is an integration test?

An integration test checks that separately built parts of your software work when they are connected to each other. For a web app, that usually means calling the running app over HTTP and letting the request reach the real routes, auth and database.

What are the four types of testing?

A common way to list them is unit, integration, system and end-to-end testing. Each one tests a bigger piece than the one before it: a single function, a few parts working together, the whole application, and the whole journey across every service a user touches.

What is the definition of unit testing?

A unit test checks one small piece of code, such as a function or a class, on its own. In the solitary style, its neighbors are swapped out, so a fault in another class can't fail it.

Can integration tests replace unit tests?

No. In our experiment, the unit suite caught a bug the API flow could not, and unit tests point at the broken line when they fail. Use integration tests for the wiring between parts, and unit tests for the logic inside each part.

All posts