← Blog

E2E API contract testing

E2E + API Contract Testing

Most flaky end-to-end tests aren't flaky because of the UI. A staging account ran out of balance. A feed returned different items this morning. The chat server was slow to send its welcome message. The flow was fine. The backend under it changed.

This post is about pinning that backend down: record what your API really sends, keep it as the contract your app was built against, and run any scenario on top of it. Plain HTTP, server-sent events and WebSockets alike.

What we mean by API contract testing

An API contract is the agreement between your app and its backend: this request gets this response, in this shape, with these status codes. Classic contract testing tools check that agreement from both sides in unit-test land. They never launch the app.

End-to-end tests have the opposite problem. They run the real app, but against whatever the backend happens to return today. So the test checks two things at once, and when it fails you don't know which one broke.

E2E contract testing splits them apart. The UI is tested against a fixed, recorded contract. If the flow fails, the app broke its side of the agreement, because the backend's side didn't move.

A live backend tells you whether today's data works. A recorded contract tells you whether the app still works.

The loop: record, save, replay

1 · RECORD
Use the app, or run a scenario, with the interceptor on. Every call lands in the Network tab: HTTP, SSE events and WebSocket frames.
2 · SAVE
Record group turns the whole recording into one mock group: a rule per endpoint, streams as scripts timed as they came.
3 · SHAPE
Leave out analytics, pick how repeated calls replay, edit answers into "Empty cart" or "Payment declined".
4 · REPLAY
Turn the group on for any scenario or chain. Runs in the app and on CI get the same answers.
From live traffic to a contract any scenario can run on.

Recording: one line in your debug build

Recording happens inside the app, not behind a proxy. The Dev Scenario HTTP interceptor sits in your app's own network stack, so there are no certificates to install, no proxy settings on the device, and pinned TLS keeps working. It sees each request after your own interceptors, auth headers included, exactly as it goes out.

On Android, the Gradle plugin wires every OkHttpClient in the debug build. Streams are opt-in:

plugins {
    id("io.github.devscenario.httpinterceptor.auto") version "…"  // latest: see Libraries
}

scenarioHttpInterceptor {
    serverSentEvents = true   // send each event live
    webSockets = true         // capture OkHttp WebSockets
}

On iOS it's a Swift package built on URLProtocol. Two Info.plist Booleans turn on the streams: DevScenarioInterceptorServerSentEvents and DevScenarioInterceptorWebSockets. Flutter has its own package for dart:io, http and dio.

Then open the Network tab and use the app. Each call shows its method, path, status, time and size, with the full request and response. A server-sent event stream gets an Events tab that fills as events arrive. A WebSocket gets Messages: every frame with its direction (↑ sent, ↓ received), size and text, and who closed it.

Off in your release app. On Android it's a debug-only dependency. On iOS and Flutter it stays off outside debug builds unless you turn it on yourself.

Saving: a whole recording as one group

In the Network tab, Record group starts from whatever traffic you need:

  • Start recording keeps every call until you press Stop and review. Use the app by hand or run a scenario meanwhile.
  • Save current calls takes what Traffic shows right now, filters and all.
  • Save run takes the calls of one of your last five runs. Nothing runs again.

Each one opens a review before anything is saved:

  • Noise starts unchecked. Analytics, crash-report and ad hosts are left out by default. Your own API is in. Tick anything back in.
  • One rule per endpoint. GraphQL calls split by operation, so GetCart and GetProfile on one URL get their own answers. When the answer depends on the query, ?page=1 and ?page=2 get a rule each.
  • Repeated calls replay in order. A cart that came back empty, then with one item, then with two, plays back exactly like that, and starts over on every scenario run. That's how you test polling without waiting on a real payment. Or keep just the last answer.
  • Only your server's answers. Calls that a mock already answered aren't saved.
  • Re-record when the backend changes. A save makes a new group. Pick an existing group instead, and its rules are replaced with the new recording.

Every saved rule stays editable, and a single call still has Mock this. A recorded rule is a starting point: change 200 to 503, empty the list, drop a field, and you have the edge case your backend can't produce on demand.

Streams: SSE and WebSockets, scripted

Request–response mocking is common. Mocking a conversation isn't. A live price ticker, a chat, a delivery tracker: the server talks first, over time, and the test needs the same words at the same moments every run.

So a stream rule plays a script. A recorded stream writes the script for you: the events (or the server's WebSocket messages) as steps, each delayed the way it arrived. Then you edit it like any rule:

PartWhat it doesExample
StepsPlayed in order once the stream opens, each after its delay. SSE steps can set event, id and retry.After 300 ms: {"status":"courier assigned"}
Replies (WebSocket)When the app sends a matching message, answer after a delay. Match the whole text, part of it, or a regex.App sends "type":"ping" → {"type":"pong"}
EndKeep the stream open, or end it after a delay. A WebSocket can close with a code and reason.Close 1001 "server restart" after 5 s

The script plays on the device, inside the interceptor, so your app reads it through its normal client code. The row in the Network tab then shows exactly what was played and what the app sent back, the same way a real stream would.

That last row in the table is a good example of a test you can't write against a live server: "when the socket drops with 1001, does the app reconnect and show the chat again?" With a script it's one rule and a flow:

tap("Support chat")
assertVisible("Hi! How can we help?")
assertVisible("Reconnecting…")
assertVisible("Hi! How can we help?")

Replaying: mock groups on any scenario

Rules live in mock groups, and groups belong to the project. A scenario or chain picks which groups are on while it runs. That's the whole wiring:

One flow, three contracts

The same checkout flow units, three scenarios: Happy path, Empty cart and Card declined. Each turns on a different group. No branching inside the flows, no test accounts set up just so.

Only while it runs

A group answers only while a scenario that uses it runs. Your normal traffic, and every other scenario, still talk to the real backend. Recording more traffic tomorrow doesn't change today's contract.

Partial on purpose

Requests no rule matches go to the real network. Keep only the calls that make the run unstable, a feed, a balance, a stream, and leave login real if you want to.

The same contract on CI

To take a contract to CI, export the regression bundle (Home › Regression builder › Export bundle). It carries the mock groups its scenarios use. Commit it and add --with-mocks:

devscenario run e2e/regression-shop.zip --with-mocks

--with-mocks starts the same matcher the app uses, stream scripts included, and points the in-app interceptor at it for the run: on Android devices and on iOS simulators. With one scenario in the bundle it gets the same answers as on a developer's laptop. With several, a group that any of them uses is on for the whole run.

Let an agent write the edge cases

With the AI Assistant plugin on, AI agents get mock tools too. Point one at the folder with your app's DTOs and API services, and ask for "every error case of the payment API". It can add up to 100 rules into a group per call, HTTP or SSE and WebSocket scripts, and bind that group to a scenario. It asks before it creates anything you didn't ask for.

What it doesn't do (yet)

  • It doesn't check what the app sends. A rule can match on the request body, but a flow can't assert "the app sent this request". If no rule matches, the request goes through.
  • No response delays on HTTP rules. Stream steps have delays; a plain HTTP rule answers right away.
  • Physical iPhones on CI. --with-mocks reaches the interceptor on Android devices and iOS simulators, not on a physical iPhone.
  • Flutter streams. The Flutter package records and mocks HTTP. It doesn't capture or mock SSE and WebSockets yet.

Try it on the flow that fails on Mondays

Every team has one: the scenario that's green all week and red after staging data resets. Save one good run of it with Record group, untick what it doesn't depend on, and turn the group on for that scenario. If it still fails, you've found an app bug, not a data problem.

Download Dev Scenario Set up the interceptor