Diff Batches
A diff batch compares two capture batches to detect visual regressions. One batch is the reference (the known-good baseline) and the other is the candidate (the new deployment to review).
Triggering a diff batch
You can start a diff batch in two ways:
- Manual — from the project's Diff Batches page, click New diff batch, then select an environment, a candidate capture batch, and a reference capture batch. You can filter batches by time period to find the right ones.
- Automated — configure automated diffing in an environment's Automation settings to have deploydiff launch a diff batch automatically after each successful capture.
Lifecycle
| Status | Meaning |
|---|---|
| Pending | Created, not yet started |
| Running | At least one page is being diffed |
| Done | All page comparisons have finished |
| Error | The batch failed to launch |
| Cancelled | Every page comparison was skipped, so there was nothing to diff |
Verdict
Once complete, a diff batch has a verdict that summarises the result across all its page comparisons:
| Verdict | Meaning |
|---|---|
| Pass | Every diffed page stayed within the configured thresholds |
| Warn | Edge cases only (new/removed pages), per the environment's verdict config |
| Fail | A diffed page exceeded a threshold (percentage or pixel count), or a critical error occurred |
| Skipped | All comparisons were skipped (e.g. all reference screenshots errored) |
Verdict rollup priority: Fail > Warn > Pass > Skipped.
Page comparisons
Each page in the batch is compared individually. A comparison can result in:
- A diff image showing exactly what changed, along with the percentage and raw count of pixels that differ.
- A verdict. A page that is actually diffed is either Pass or Fail, decided by the thresholds in the project's diff config, which a page can override. The Warn and Skipped verdicts apply to the edge and skip scenarios below (new/removed page, capture or diff errors), each mapped to a verdict by the environment's verdict config.
Some comparisons are skipped rather than diffed when the page situation makes a pixel comparison impossible:
| Reason | Meaning |
|---|---|
| New page | Page exists in candidate but not in reference |
| Removed page | Page exists in reference but not in candidate |
| Reference error | Reference screenshot failed to capture |
| Candidate error | Candidate screenshot failed to capture |
| Both errored | Both screenshots failed to capture |
The verdict assigned to each skipped scenario is controlled by the environment's verdict config.
When a page comparison fails
A comparison that could not be computed shows an error code. Its verdict is the one the environment's verdict config assigns to a diff processing error.
| Code | What happened |
|---|---|
PNG_WIDTH_MISMATCH | The two screenshots do not have the same width, usually because the page's viewport width changed between the two captures. |
PNG_TOO_SMALL | One of the screenshots is smaller than 100 × 100 pixels. |
| Any other code | An internal error on our side. Launch a new diff batch to try again. |
A comparison that fails is never counted in your usage, whatever its code.