OpenRTB testing: validate the contract before blaming the bidder.

Engineering field workbook: Bid Request And Response Contract Test

Purpose. This exercise turns the guide into a reproducible investigation. The scenario is synthetic and is provided for learning, not as evidence of a production incident. Begin with a banner bid request with one impression, a currency restriction, a floor, and a tmax deadline. The question to answer is not merely whether a component is present, but whether the relevant system behavior can be demonstrated with a traceable set of observations. Record the environment, configuration version, test date, and which participant owns each piece of evidence before drawing a conclusion.

Scenario: identify the first divergence

Suppose the bidder responds with a price below the declared floor and an unexpected currency. Write down the expected behavior in one sentence and the observed behavior in another. Avoid diagnosing the entire system from a final dashboard number: many upstream failures collapse into the same visible symptom. Start at the earliest boundary that can be independently verified, then follow the transaction toward the user-visible outcome. Treat a missing log entry as an unknown until logging coverage is confirmed. A negative result is meaningful only if the test could have observed a positive one.

Evidence package to collect

For this investigation, validate the request assumptions, inspect bid.price and bid.cur, then check the exchange decision and no-bid behavior. Preserve raw samples and a normalized summary separately. Redact identifiers that could expose real users, partner credentials, proprietary bid details, or account-specific configuration. Label every timestamp with its time zone and specify whether it is client time, server receipt time, or reporting time. Keep an explicit column for missing information. This makes the exercise repeatable and prevents an apparent mismatch from being explained away by undocumented assumptions.

Controlled test procedure

First, freeze the test conditions: choose a known placement or request fixture, record configuration hashes, and ensure that unrelated releases are not running. Next, build fixtures for floor equality, below-floor bids, unsupported currency, expired responses, and missing creative attributes. Repeat the baseline at least twice to check whether the observation is stable. Change one variable at a time and record both the intended change and the actual request payload produced by the system. If a result changes, repeat the original baseline before attributing causality. Where production experiments could affect revenue or privacy, use synthetic fixtures or a non-billable test environment instead.

How to interpret conflicting signals

One useful distinction is between configured, requested, accepted, and counted. A setting can exist without appearing in a network request; a request can reach a partner without being accepted; and an accepted auction event can fail to become a billable impression. In this topic, the DSP returning HTTP 200 does not mean the exchange accepted the bid. Build a small decision table that identifies the system responsible for each transition and the observable proof needed to confirm it. When two dashboards disagree, compare their event definitions before comparing totals.

Negative controls and edge cases

Create one known-good fixture and at least three deliberate failure fixtures: a missing required input, a syntactically acceptable but semantically wrong value, and a delayed or partially unavailable dependency. For each fixture, specify the expected validation message, whether the system should fail open or fail closed, and whether any downstream report should change. Record what happens if the same request is retried, if a partner returns no response, and if the response arrives after the decision deadline. These checks reveal whether a diagnostic method is robust or only succeeds for a happy-path demonstration.

Decision record and next action

Conclude the investigation with four fields: confirmed observation, leading hypothesis, disconfirming evidence, and next smallest experiment. A useful outcome can be “insufficient evidence” when a partner-controlled log is unavailable. partner-specific deal logic and exchange policies may override simplified examples. If the issue cannot be reproduced, document what was attempted and what instrumentation would make the next occurrence diagnosable. Do not claim a performance improvement until the outcome metric and comparison population are defined in advance.

Practical review questions

1. Which specific observation proves that the transaction reached the stage being investigated?
2. What alternative explanation would produce the same symptom without the suspected defect?
3. Which one-variable test could distinguish those explanations?
4. What data must be withheld or sanitized before sharing the example?
5. How would the team know the fix worked without relying on a single dashboard screenshot?

Takeaway: A high-quality technical investigation should be repeatable by another engineer. Keep the fixture, expected output, observed output, and limits of inference together so that a reader can test the claim rather than merely accept it.