AdTech QA checklist
Build reusable functional and integration coverage.
Test lab / Yield testing
Publisher Floor Price Lab Scenario: Isolating the Threshold
A floor test should vary one economic input while keeping inventory, campaign eligibility, geography, format, and timing stable.
Transparency: This is a synthetic, reproducible lab scenario created for education. It is not presented as a client result, Google result, DSP result, or production measurement.
Scenario setup
This is an illustrative lab scenario. Use a controlled test publisher and a campaign whose eligibility is known. Choose three floor conditions: below the expected bid, near the expected bid, and above the expected bid.
Evidence to collect
For every condition record request count, bidder response count, bid values, wins, impression events, currency, and the exact floor sent to the bidder or applied by the exchange. Use the same reporting window and timezone.
Interpreting the result
If bids remain present but wins fall, inspect auction ranking, targeting, and downstream filtering. If bids disappear, determine whether the bidder received the changed floor and whether the DSP applies its own eligibility rules. A lower win rate alone does not prove floor rejection.
Currency check
If the exchange and DSP use different currencies, normalize the values before comparing them. A numerical threshold without its currency and unit is incomplete test evidence.
Regression case
Automate or document a matrix covering low, boundary, and high floor values. Include a no-bid case and a successful-win case. The test should prove that only the intended floor changed.
Case study: A floor increase reduces eligible bids
A publisher raises the auction floor from USD 0.80 to USD 1.60 for a single 300×250 placement. The synthetic bidder is configured to value the same impression at USD 1.20. The test is about bid eligibility, not whether a higher floor automatically increases revenue.
Evidence ledger: what to capture
Log the effective floor after all rules, currency, placement ID, bidder ID, bid CPM, auction timeout, and rejection reason. Record whether the floor was communicated in bidfloor and whether the bid is rejected by the exchange or wrapper.
Controlled experiment and expected observations
Run an A/B fixture with identical site, device, geography, consent, and demand configuration. In the baseline, set the effective floor to USD 0.80; in the treatment, set it to USD 1.60. Repeat with a USD 2.00 bid and compare outcomes.
Reading the result without overclaiming
If the USD 1.20 bid clears only the baseline while USD 2.00 clears both, floor eligibility is a credible explanation. If both tests fail, investigate bidder eligibility and timeouts before blaming the floor.
Correction, retest, and monitoring
Roll back the rule for the affected inventory or adjust floor segmentation. Verify that currency conversion and deal-specific overrides are handled correctly; compare fill, clearing price, and revenue per thousand opportunities rather than CPM alone.
Important edge case
A deal floor, bidder-specific adjustment, or stale floor configuration can override the displayed UI value. Confirm the effective auction value, not merely the rule saved in the dashboard.
Hands-on verification worksheet
- Write a falsifiable hypothesis for this scenario: a floor increase reduces eligible bids. State what observation would disprove it.
- Collect the minimum evidence before changing configuration. Log the effective floor after all rules, currency, placement ID, bidder ID, bid CPM, auction timeout, and rejection reason. Record whether the floor was communicated in bidfloor and whether the bid is rejected by the exchange or wrapper.
- Create an isolated negative test and a positive control. Run an A/B fixture with identical site, device, geography, consent, and demand configuration. In the baseline, set the effective floor to USD 0.80; in the treatment, set it to USD 1.60. Repeat with a USD 2.00 bid and compare outcomes.
- Classify the outcome using a precise event boundary. If the USD 1.20 bid clears only the baseline while USD 2.00 clears both, floor eligibility is a credible explanation. If both tests fail, investigate bidder eligibility and timeouts before blaming the floor.
- After any correction, run the same fixture and retain before/after evidence. Roll back the rule for the affected inventory or adjust floor segmentation. Verify that currency conversion and deal-specific overrides are handled correctly; compare fill, clearing price, and revenue per thousand opportunities rather than CPM alone.
- Include a note about alternative explanations. A deal floor, bidder-specific adjustment, or stale floor configuration can override the displayed UI value. Confirm the effective auction value, not merely the rule saved in the dashboard.
Practical investigation notes and worked methods
Model the difference between price and yield
Raising a floor can increase the minimum accepted bid while reducing the number of bids that qualify. Those effects pull revenue in opposite directions. In a synthetic cohort of 10,000 comparable opportunities, suppose a low-floor variant produces 7,000 monetized impressions at a $1.20 average CPM, while a high-floor variant produces 4,500 at a $1.60 average CPM. The estimated gross revenue is $8.40 versus $7.20 respectively, using impressions multiplied by CPM divided by 1,000. These numbers illustrate the arithmetic, not a universal market response. A higher CPM alone is not evidence of higher yield.
Design a fair floor experiment
Split comparable inventory into control and treatment groups using a stable assignment rule. Keep device, geography, format, traffic source, time of day and demand eligibility balanced. Avoid comparing Monday morning with Saturday night, or premium video with standard display. Record the exact floor rule applied to each request, including currency and whether the rule is hard or soft. If the auction includes multiple demand sources, verify that the floor is applied consistently at the relevant decision point. A change in fill may reflect demand composition rather than floor policy alone.
Inspect rejection reasons and bid distributions
Collect the distribution of incoming bids before and after the change, including how many fall below the threshold. Segment the data by bidder, placement and format. If the exchange exposes floor-related rejection reasons, compare them with actual submitted prices. A floor configured in one currency and interpreted in another can produce dramatic losses that resemble weak demand. Check whether the floor is applied to gross bid price or a net value after fees. Preserve the definition used by each system; otherwise, the analysis can appear precise while comparing incompatible numbers.
Define guardrails beyond revenue
Revenue is important but insufficient as a sole experiment metric. Track fill rate, timeout rate, latency, viewability where measured, and the distribution of empty slots. A treatment that raises short-term revenue but substantially worsens user experience may not be acceptable. Decide before the test what constitutes a rollback condition and how much traffic the experiment can safely expose. For small cohorts, avoid declaring a winner from a single noisy day. Present uncertainty and confidence intervals when sample sizes and statistical assumptions permit.
Interpret a negative result constructively
If the high-floor group underperforms, isolate whether the loss comes from fewer eligible bids, a changed winning-price distribution, a technical misconfiguration or a traffic mix imbalance. Restore the baseline rule before trying a different threshold. Consider segmented floors only when the data shows meaningful differences by inventory cohort. Publish a sanitized experiment worksheet with the cohort definitions, calculation formula, exclusion rules and decision criteria. That makes the lesson reusable even when the final result is to leave the floor unchanged.