AdTech QA checklist
Build reusable functional and integration coverage.
Test lab / Video testing
VAST Lab Scenario: From Valid XML to Failed Playback
Video ad debugging should separate tag retrieval, wrapper resolution, media selection, player compatibility, and tracking.
Transparency: This is a synthetic, reproducible lab scenario created for education. It is not presented as a client result, Google result, DSP result, or production measurement.
Scenario setup
Use a test VAST tag and a player with a known supported media type. Record the tag URL, response status, wrapper depth, media URLs, MIME types, and player version.
Expected path
The expected path is tag retrieval → XML parsing → wrapper resolution if applicable → media selection → playback → impression and quartile events. Each stage should have an observable result.
Observed failure
Suppose the XML is valid and the media URL is reachable, but playback does not start. Inspect MIME type, codec support, media dimensions, player restrictions, and timeout behavior before changing the VAST document.
Tracking check
If playback succeeds but impression or quartile events are missing, inspect tracking URLs and event timing separately. Do not treat a playback failure and a tracking failure as the same defect.
Regression case
Keep one successful creative and negative fixtures for unreachable media, malformed XML, unsupported media, slow wrapper resolution, and missing tracking. Test them after player or VAST integration changes.
Case study: A video ad returns VAST but does not play
A synthetic video player retrieves a VAST document, yet playback stalls before the first frame. The lab separates XML retrieval, wrapper resolution, media compatibility, player policy, and measurement events.
Evidence ledger: what to capture
Capture VAST response status, XML validity, wrapper depth, MediaFile MIME type, bitrate, codec, HTTPS URLs, player error code, browser console, and tracking-event timestamps.
Controlled experiment and expected observations
Test the inline VAST response directly, then test each wrapper hop. Compare a known-playable MP4 media file with the failing asset using the same player and browser. Record whether the failure happens before load, on media selection, or at start.
Reading the result without overclaiming
A successful VAST HTTP response is not proof that the media can decode or autoplay. A wrapper loop or inaccessible MediaFile can fail even when XML syntax is valid.
Correction, retest, and monitoring
Correct broken media URLs, supported formats, wrapper redirects, or player configuration. Retest on mobile and desktop with muted autoplay policies and actual user-gesture behavior accounted for.
Important edge case
Do not equate a VAST impression beacon with a completed view. Start, quartile, error, and completion trackers have different firing conditions.
Hands-on verification worksheet
- Write a falsifiable hypothesis for this scenario: a video ad returns vast but does not play. State what observation would disprove it.
- Collect the minimum evidence before changing configuration. Capture VAST response status, XML validity, wrapper depth, MediaFile MIME type, bitrate, codec, HTTPS URLs, player error code, browser console, and tracking-event timestamps.
- Create an isolated negative test and a positive control. Test the inline VAST response directly, then test each wrapper hop. Compare a known-playable MP4 media file with the failing asset using the same player and browser. Record whether the failure happens before load, on media selection, or at start.
- Classify the outcome using a precise event boundary. A successful VAST HTTP response is not proof that the media can decode or autoplay. A wrapper loop or inaccessible MediaFile can fail even when XML syntax is valid.
- After any correction, run the same fixture and retain before/after evidence. Correct broken media URLs, supported formats, wrapper redirects, or player configuration. Retest on mobile and desktop with muted autoplay policies and actual user-gesture behavior accounted for.
- Include a note about alternative explanations. Do not equate a VAST impression beacon with a completed view. Start, quartile, error, and completion trackers have different firing conditions.
Practical investigation notes and worked methods
A controlled VAST playback matrix
A video ad can pass XML validation yet fail before the first frame. Design the lab as a matrix, not a single successful response. Test an inline VAST document with a directly reachable MP4, a wrapper that resolves to the same inline creative, a wrapper that times out, and a creative whose MIME type is unsupported by the selected player. Hold player version, viewport, browser and network conditions constant. Record the HTTP status and duration of every wrapper and media request. A failure that only appears in the wrapper case points to a different layer than a failure shared by inline and wrapped creatives.
Separate the ad decision from media delivery
Capture the point at which the ad server returns a VAST document, the point at which the player parses it, the point at which it selects a media file, and the point at which playback actually begins. These are four distinct events. A successful ad response establishes only that a document was returned; it does not prove that the player found a compatible rendition or that the browser accepted autoplay. When an impression event is missing, first ask whether the player ever reached the condition that triggers it. Do not infer a billing defect from a single missing client-side event.
Synthetic fixture and expected observations
For a reproducible exercise, configure fixture A with a reachable MP4 and a valid duration, fixture B with a media URL returning HTTP 404, fixture C with a wrapper pointing to an unreachable endpoint, and fixture D with a media type the test player cannot decode. Expected results are player-dependent, so write down the player’s documented error mapping before running the test. A should progress to playback in a compatible environment. B should reveal a media retrieval failure. C should reveal a wrapper-resolution failure. D should be rejected during media selection or decoding. These fixtures are educational, not measurements from a real publisher.
Investigate tracking without double counting
For each fixture, capture start, first-quartile, midpoint, third-quartile and completion signals only if the player actually reaches those milestones. Distinguish a missing event from a duplicate event, a blocked beacon and a beacon that reaches the server but is rejected. Inspect the browser network waterfall for redirects, status codes, request cancellation and timing. A completion beacon is not evidence of a completed video if the implementation fires it on teardown. Conversely, an absent beacon does not necessarily mean playback failed. Compare player telemetry with the tracking endpoint response and the server’s ingestion logs.
A practical regression release gate
After a fix, repeat the exact failing fixture and at least one previously passing fixture. Record player version, VAST version, creative type, wrapper depth, browser, operating system and network shaping profile. Require the team to state what changed: parsing, media selection, network access, autoplay handling or tracking. Store sanitized VAST XML and expected event sequences in version control. Never include live advertiser tokens or user identifiers in fixtures. The release is acceptable when the original defect no longer reproduces and the baseline case has not regressed.