Skip to content
DocuccinoDocuccinoDocuccino

Contract testing

Your test suite already calls real endpoints with real data. Docuccino can hold every one of those exchanges to the document it generates — so a response that drifts from its own documentation fails a test instead of reaching a client.

The assertions read the UIR artifact rather than an OpenAPI export, and that is the whole point. UIR carries x-docuccino.provenance, so a failure does not stop at “total should be a number” — it tells you the shape came from integration:eloquent, out of app/Models/Invoice.php:31. That is almost always where the fix is.

Setup

Section titled “Setup”

  1. Export the document as UIR. Add a uir target beside whatever you already write, in docuccino.yaml:

    documents:
      default:
        export:
          targets:
            - { format: 'openapi-3.2', path: 'docs/openapi.json' }
            - { format: 'uir', path: 'docs/api.uir.json' }
    Terminal window
    php artisan docuccino:export

    The assertions find that file on their own — they read the document’s own uir target, so there is nothing else to configure. Commit the artifact; it is deterministic, and the freshness and breaking-change assertions below both compare against it.

  2. Add the trait to your base test case.

    tests/TestCase.php
    use Docuccino\Laravel\Testing\AssertsApiContract;
    

    abstract class TestCase extends BaseTestCase
    {
        use AssertsApiContract;
    }

  3. Optionally, register the response macros so the assertions chain off the call you already make:

    tests/TestCase.php
    protected function setUp(): void
    {
        parent::setUp();
    

        \Docuccino\Laravel\Testing\ApiContract::registerMacros();
    }

Assert an exchange

Section titled “Assert an exchange”
it('lists invoices', function () {
    $this->getJson('/api/invoices?page=2')
        ->assertOk()
        ->assertValidExchange();
});

Three assertions cover the exchange, as methods on your test case or as chained macros:

Assertion Checks
assertValidRequest($response) The path, query, header and cookie parameters, and the request body, against what the operation documents.
assertValidResponse($response) The status, the media type, the response headers, and the body against the schema documented for that pair.
assertValidExchange($response) Both halves.

Each returns the response, so they chain with Laravel’s own assertions in any order.

What they hold you to:

  • The status must be documented. A 500 nobody wrote down fails, naming the statuses the operation does document.
  • The media type must be documented for that status, application/problem+json and vendor +json types included.
  • A body JSON Schema cannot check passes with a note rather than passing blank — a text/csv download, or a media type documented with no schema. The note is raised on your runner’s warning channel, against the test that produced it, so a pass that proved less than it looks like says so (pest --display-warnings, phpunit --display-warnings). It never fails a test on its own.
  • Form request bodies are checked, uploads included. An endpoint whose body Docuccino publishes as multipart/form-data or application/x-www-form-urlencoded — anything with a file or image validation rule lands in the first — is held to that schema like any other: required parts, types, enums, and a part the document names no property for. $this->post('/api/invoices', [...]) is all it takes. Your framework parses a form body out of the request before a test can see the bytes, so the check reads the fields it was parsed into, and every member is read as the type the document documents, the way a query value is — except for a comma, which in a form body is a character and not a list separator. A file part is checked as present and as the string a binary part is; its contents are not read.
  • The fields are read as they arrived, not as your application left them. TrimStrings and ConvertEmptyStringsToNull are in Laravel’s default global stack and rewrite the request in place, and every $request->merge() in your middleware and controllers adds fields no client sent — so the parameter bag an assertion can reach is not the message. The trait records each request at the front of the middleware stack, ahead of all of it. Where nothing recorded one — a suite that reaches the assertions without the trait, or a TestResponse built by hand — the body is not checked and the note says so, rather than reading whatever is in the bag. ApiContract::captureRequestBodies() turns the recording on for a test case that cannot take the trait.
  • A request carrying a file is read as the multipart request it is. Laravel’s test client labels post() application/x-www-form-urlencoded even when you hand it an UploadedFile, and no client can send a file that way — so a request with file parts is checked against the multipart/form-data body your document publishes. postJson() is the other way round: it lifts the UploadedFile out and really does send JSON, so it is checked as JSON and told the contract documents multipart. Use post() for an endpoint documented as multipart.
  • A body that got both the bytes and the type wrong hears about both. A missing required body and a media type the contract documents no entry for are two independent mistakes, and they are reported together rather than one at a time.
  • A streamed response is checked against what it streamed. response()->stream(…) and streamJson(…) hand PHP no body string at all; the assertions run the stream and check the bytes it wrote, so a streamed endpoint is held to its documented schema like any other. Only where you asked about the response: assertValidRequest() never runs the callback, so an endpoint that streams forever, or whose closure consumes a queue or deletes a file, does neither on a request-only assertion.
  • A status documented with no content at all is different, and it fails. Omitting content in OpenAPI asserts the response has no body, so a 404 documented that way against a 404 that really sends bytes is a document under-describing your API, and the failure names both halves: the status the contract documents no body for, and how many bytes came back. The commonest way to get there is a renderer of your own that Docuccino could see run and could not read, which withholds the framework’s shape rather than asserting it; the failing test is the push to make that arm readable, or to state the shape with #[Response(status: …, type: …)]. The inferred-handler.too-dynamic warning on your build predicts this test, so a red suite here should never be the first you hear of it.
  • Query and header values are read as the type the contract documents. ?page=2 is checked as the integer 2; ?page=first is checked as the string it is, so the failure names the real problem instead of silently becoming 0. A comma list (?sort=total,-created_at) becomes the array Docuccino documents it as.
  • postJson($uri, []) sends the empty body it looks like. PHP has one array where JSON has two containers, so json_encode([]) writes [], and there is no argument you could pass those helpers that would write {}. Where the operation documents a body an empty object satisfies, that [] is read as the empty object; where it documents a list, it stays the list it looks like, minItems and all. Nothing else moves: a populated array against an object body still fails, and so does an empty one against an object whose properties are required. The same reading covers a webhook payload you hand over as a PHP array — JSON text you pass as a string already said which container it is, and is taken at its word. Your responses are never re-read this way, because [] in a response body is what your clients really receive.
  • The response headers the document publishes are checked, not just the body. A Location on a 201, the Retry-After and X-RateLimit-* on a 429 — every one of them is read as the type documented beside it, the same way a query parameter is. A header marked required that the response never sent fails; an optional one it never sent does not, because the contract said it might not be there. Docuccino marks one required where the framework leaves no branch in which it is absent — a redirect’s Location, and all four throttle headers — and you can say the same about your own with #[ResponseHeader(required: true)]. Header names are matched case-insensitively, and a header the response sent more than once (Set-Cookie) is held to the schema once per value it sent.
  • A response header the check cannot read passes with a note, the same way a text/csv body does: a header documented with no schema, or documented with content instead of one. A Content-Type entry in a headers map is ignored entirely — OpenAPI says it is, because content is what describes the media type.

Assert a webhook you send

Section titled “Assert a webhook you send”

Everything above is the inbound half: requests your API answers, which your suite already makes. The other half is the payload your application delivers — and no HTTP test in your suite goes anywhere near it, so it is the half of the document that drifts silently.

assertValidWebhook() closes it. Give it the name the webhook is published under and the payload you are about to dispatch, and it is held to the schema the document publishes for that name:

use App\Webhooks\InvoicePaid;


it('delivers a payload subscribers can rely on', function () {
    $this->assertValidWebhook('invoice.paid', new InvoicePaid(
        invoiceId: 42,
        amountInCents: 1250,
        paidAt: '2026-03-01T09:00:00Z',
    ));
});

Pass the payload in whatever form your code holds it at the moment it dispatches — the event object, a Data object, an array, a Jsonable, or JSON text you already encoded. It becomes the bytes your subscriber would receive, and those are what the schema judges. An object that says what its own JSON is (toJson(), jsonSerialize()) is asked rather than encoded around.

What it holds you to:

  • The name must be documented. A webhook nobody wrote down fails, naming the ones the contract does publish.
  • The payload must satisfy the documented body, including every enum in it. A sender that starts delivering a status outside the published set breaks every client generated from the document, and this is the only assertion that sees it.
  • A webhook the document publishes no body for fails. Nothing is in the way there — the document simply says nothing about what you deliver, so there is no contract for the payload to be held to, and a pass would claim one had been checked. It is the outbound twin of a status nobody wrote down.
  • A body JSON Schema cannot check passes with a note rather than passing blank — a text/csv delivery, or a media type documented with no schema — and the note reaches the same warning channel the exchange assertions use.

The assertion reads the same artifact as everything else on this page. Assert against the UIR export, or an OpenAPI 3.1 or 3.2 one: 3.0 defines no webhooks member, so a 3.0 artifact dropped every webhook it had, and the assertion says exactly that rather than reporting your webhook as undocumented.

Report the responses your suite never proved

Section titled “Report the responses your suite never proved”

Every response assertion records the documented response it matched — the operation’s stable x-docuccino.id and the status that came back. Never a path string, so a renamed route reads as still covered rather than as one endpoint vanishing and another appearing.

Responses, not endpoints. A documented 422 is a promise of its own: it is what a consumer writes a catch against, and what their generated client types an error as. A suite that only ever asserts the happy path has touched every endpoint and proved none of those promises — so a number counting endpoints calls that full coverage, which is exactly the too-generous reading a coverage gate exists to prevent. The report prints both numbers and gates on responses and deliveries.

A webhook you document counts too. A passing assertValidWebhook() records the delivery it checked, and every documented webhook carries one row of its own — so a document whose outbound half nothing asserts can never read as complete. The row is the delivery, never the webhook’s own responses: those are what the receiver answers, and nothing in a sending application’s suite can exercise one.

Coverage is a question about the whole suite, and no test can see the whole suite: a parallel worker holds its own share, a shard holds its own machine’s, and neither can know when the others have finished. So it is not an assertion. Each process writes down what it exercised, and a command merges those logs afterwards and gates — the same shape line coverage has, where workers write and the runner merges once they are done.

  1. Turn the recorder on where you set the assertions up:

    // tests/TestCase.php or tests/Pest.php, beside registerMacros()
    \Docuccino\Laravel\Testing\ApiContract::recordCoverage();

    Each process appends what it reached to a file of its own under coverage.log — storage/docuccino/coverage unless you say otherwise. Point it somewhere your repository ignores; the logs are per-run build output.

  2. Merge and gate after the run:

    Terminal window
    php artisan docuccino:coverage --reset   # last run's logs are not this run's
    vendor/bin/pest --parallel
    php artisan docuccino:coverage --min=85

    Or as one composer script, the way a line-coverage gate is usually written:

    "scripts": {
        "test:contract": [
            "@php artisan docuccino:coverage --reset",
            "pest --parallel",
            "@php artisan docuccino:coverage --min=85"
        ]
    }
Coverage — default
──────────────────
/app/storage/docuccino/coverage
8 log files, 29 entries


Docuccino contract coverage: 29 of 41 documented responses and webhook deliveries exercised (70.73%, floor 85%).
21 of 23 documented operations were reached at all — the floor is measured against responses and deliveries, not operations.


Never exercised:
  GET    /api/invoices                 422            op:v1:k9wd2mrb7ks9tvzq
  GET    /api/invoices/{invoice}       404, default   op:v1:h4dqx2mrb7ks9tvz
  POST   /api/invoices/{invoice}/void  201, 409, 422  op:v1:p6nw3jc8ygf5s0ea
  POST   webhooks.invoice.paid         delivery       op:v1:m2xq8bd4nf6ha1cy


Cover them, or — if this is the honest measured floor for now — move the floor to 70 and ratchet it up from there.

The middle column is the statuses that operation documents and your suite never produced, in the document’s own order. An operation appears here as soon as one of its responses is unproved, so an endpoint whose happy path is covered and whose errors are not is listed with its errors named — that is the gap the number is about.

Every documented response counts as one, including a 204 documented with no body and a default error response: both are promises a client can be handed. An operation that documents no responses at all has no status to name, so it reads (no responses documented) and counts once — as “was it ever reached”. A webhook counts once too, under delivery.

The one thing that counts nowhere is a response key no status can resolve to — 4xx in lower case where OpenAPI spells the range 4XX, or a word like ok. Nothing can ever produce a response that matches it, so counting it would put --min=100 permanently out of reach. The report names each one under the numbers rather than dropping it quietly, because a short denominator with no explanation is worse than the key was. An operation whose response keys are all unreachable reads (no response a status can name) rather than (no responses documented) — it documented responses; none of them can be met.

Logs accumulate until something clears them, which is what the --reset in the recipe is for: skip it and the report unions this run with every earlier one, reading more generous than the truth. If the logs it merged were written far enough apart to be more than one run, the report says so above the numbers.

Treat the floor the way you treat a line-coverage floor: set it to what you measure today, and raise it as you close gaps. A floor you cannot meet is a floor somebody deletes. --min=100 is “every documented response answered, and every documented webhook delivered”.

Sharded CI

Section titled “Sharded CI”

--path is repeatable, because the merge is the only place several shards’ logs ever meet. Each shard uploads its directory; a final job downloads them all and names every one of them before gating:

# each shard
- run: vendor/bin/pest --shard=${{ matrix.shard }}/4
- uses: actions/upload-artifact@v4
  with:
    name: contract-coverage-${{ matrix.shard }}
    path: storage/docuccino/coverage
    if-no-files-found: error   # a shard that logged nothing must fail here, not vanish


# the gate, after the matrix
- uses: actions/download-artifact@v4
  with: { pattern: contract-coverage-*, path: coverage-logs }
- run: |
    php artisan docuccino:coverage --min=85 \
      --path=coverage-logs/contract-coverage-1 \
      --path=coverage-logs/contract-coverage-2 \
      --path=coverage-logs/contract-coverage-3 \
      --path=coverage-logs/contract-coverage-4

Name every shard’s directory, one --path each. Subdirectories are walked, so a single --path=coverage-logs reads all four when all four arrive — but if shard 3’s artifact never arrives, the tree simply lacks a subdirectory, and a directory nobody named is a directory nothing can report as missing. The merge would be “complete” over three shards. Named individually, an absent shard is an absent directory, the command refuses, and the gate stays shut. if-no-files-found: error closes the other half: without it, upload-artifact warns and uploads nothing when a shard wrote no logs at all.

Log filenames are unique per writing process, so two shards on one machine — or two workers sharing a token — never overwrite each other.

Catch an example that lies

Section titled “Catch an example that lies”

Inference cannot be wrong about a shape it derived. A hand-written #[Example] can say anything at all — and it is the part of your documentation a reader copies.

it('publishes examples that are actually valid', function () {
    $this->assertValidExamples();
});

Every example in the document goes through the same validator the response assertions use: response content, request bodies, parameters, named examples maps, and examples nested anywhere inside a component schema.

The count is the examples the validator actually judged. One it refused to judge — a schema no validator will read — is not a passing example, so it is kept out of that number and named separately at the foot of the failure, alongside what the validator objected to.

1 of 34 documented examples does not match the schema beside it.


  components/schemas/Invoice
    at /components/schemas/Invoice/properties/total/example
      the example
        The data (string) must match the type: number
        schema   /components/schemas/Invoice/properties/total
        from     integration:eloquent (integration) — app/Models/Invoice.php:31

Record your suite’s responses

Section titled “Record your suite’s responses”

The bodies your assertions just checked are also the most honest examples your document could carry. Turn the recorder on where you set the assertions up:

// tests/TestCase.php or tests/Pest.php, beside registerMacros()
\Docuccino\Laravel\Testing\ApiContract::record();

Then say which responses are worth publishing, one assertion at a time. recordAs: names the scenario the test set up, and naming it is what asks for it:

$this->getJson('/api/carts/1')->assertOk()->assertValidExchange();                       // checked
$this->getJson('/api/carts/1')->assertOk()->assertValidExchange(recordAs: 'empty-cart'); // checked and published

Recording is deliberate because checking and publishing want opposite things. Check every exchange you can — that is how you find a contract defect. Publish one response per operation, chosen by you — that is documentation. An assertion that names no scenario checks the response and records none of it, so a suite full of assertValidExchange() calls never fills your document with whatever a factory happened to generate.

Recording writes committed files that the build reads — it publishes nothing on its own, and runs none of your endpoints at build time. Where the files live, how the best of several tests sharing a name is chosen, what gets redacted on the way out and how a recorded example ranks against one you wrote are all on Example payloads.

Recording survives --parallel too, and differently: a recording is one operation’s, and the best body for it is a total order on the bodies themselves — so workers take turns on a lock rather than writing files nobody has to reconcile, as coverage does.

Gate breaking changes

Section titled “Gate breaking changes”

assertNoBreakingChanges() runs the same semantic, id-based diff as docuccino:diff, against the artifact you committed — so the assertion and the command can never disagree about what “breaking” means.

it('does not break the published contract', function () {
    $this->assertNoBreakingChanges();
});

Pass a git ref to compare against a branch or a tag instead of your working tree, exactly as the command’s --against does:

$this->assertNoBreakingChanges('origin/main');

Prose never fails this. A schema’s title, description, example, examples, externalDocs and $comment are reported as schema.annotation-changed and are never breaking, so a rewritten description cannot fail a build. default, readOnly, writeOnly and deprecated are not in that group.

A recorded example is published beside its schema rather than inside one, so it is not compared at all — a suite that records its own payloads has never been able to fail this assertion because the last run sent different ids. Full rule, including what else the diff does not see, under docuccino:diff.

The failure renders the changeset, then says where each broken node came from:

The current document makes 1 breaking change to the committed contract.


1 change (1 breaking)


BREAKING
  ~ [schema] components.schemas.Invoice.properties.total.type  (schema.type-changed)
      type: number -> string


  Where those changes came from:
    components.schemas.Invoice.properties.total.type
      integration:eloquent (integration) — app/Models/Invoice.php:31 in App\Models\Invoice::$total

Keep the committed artifact honest

Section titled “Keep the committed artifact honest”

A committed document is only useful while it matches the code. assertDocumentUpToDate() compares a fresh build against every file your export targets write — byte for byte, which is a fair test precisely because Docuccino’s output is deterministic:

it('has an up-to-date API document', function () {
    $this->assertDocumentUpToDate();
});

Emit options that do not change the contract are not staleness: an artifact exported at a different --provenance level, or with --drop-ids, still passes. When the bytes really have moved, the failure names the file and shows what changed:

docs/api.uir.json is out of date.


  What changed since it was written:


    2 changes (0 breaking)


    NON-BREAKING
      + [parameter] GET /api/invoices  query.status  (parameter.added)
      ~ [operation] GET /api/invoices  (operation.summary-changed)


  Where those changes came from:
    GET /api/invoices  query.status
      integration:query-builder (integration) — app/Queries/InvoiceQuery.php:18


  Regenerate it: php artisan docuccino:export

Pointing at a different document

Section titled “Pointing at a different document”

Multi-document apps pick one; anything else names a path directly. Both belong in your test bootstrap:

use Docuccino\Laravel\Testing\ApiContract;


ApiContract::forDocument('admin');          // assert against documents.admin
ApiContract::using('docs/api.uir.json');    // or name the artifact yourself

ApiContract::assertions() hands you the same assertions as an object, for a test case you cannot add the trait to.

Moving between documents mid-suite is what a per-version check is made of: pin an API version on the request, point the assertions at that version’s document, and require the response to validate against it. See API versioning.

Watch every exchange

Section titled “Watch every exchange”

Every assertion hands the request, the response and the matched operation to any observer you register — the seam anything built on top of contract testing needs:

use Docuccino\Laravel\Testing\ApiContract;
use Docuccino\Laravel\Testing\Contracts\ContractObserver;
use Docuccino\Laravel\Testing\ObservedExchange;


final class RecordExchanges implements ContractObserver
{
    public function observed(ObservedExchange $exchange): void
    {
        // $exchange->operationId(), ->method(), ->pathTemplate(), ->status(), ->body()
        // ->request (Illuminate\Http\Request), ->response (TestResponse), ->result
    }
}


ApiContract::observe(new RecordExchanges);

Observers are notified before the assertion fails, so they see failing exchanges as well as passing ones. Docuccino’s own coverage recorder is one of these, and so is the response recorder.

Commandsdocuccino:export and docuccino:diff, the commands these assertions mirror.
The UIR formatWhat provenance is, and why the assertions read UIR rather than OpenAPI.