Upload accessibility scans from CI
Last updated
Run axe-core or Lighthouse in CI and post the results to Metrickle. Each scanned page’s violations then show beside the friction on that page, so a failing button sits next to the people it’s stopping. A violation on an element people rage-click or abandon lands on that element’s friction hotspot, and on a case whose step is that page and whose failure names that element, so it goes into the issue you file from the case. For how to read the results, see accessibility breakdowns.
Get the scan hook URL
Open the app’s Integrations page and choose Set up scan hook under Accessibility scans. Only owners and admins, or people with the apps.keys permission, can see it. The URL looks like this:
https://in.metrickle.com/v1/a11y-scans/ah_…
Anyone with the URL can replace the app’s scan results, so keep it in your CI secrets. If it leaks, choose Replace URL. The old URL stops working straight away.
Post results
With the axe CLI, against your staging or preview site after each deploy:
npx @axe-core/cli https://staging.quillmere.example/ https://staging.quillmere.example/checkout \
--save axe-results.json --dir .
curl -X POST "$METRICKLE_SCAN_HOOK?version=$GIT_SHA&environment=staging" \
-H "content-type: application/json" --data-binary @axe-results.json
From Playwright tests with @axe-core/playwright, once per page:
import AxeBuilder from "@axe-core/playwright";
const results = await new AxeBuilder({ page }).analyze();
await fetch(process.env.METRICKLE_SCAN_HOOK!, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify(results),
});
With Lighthouse, post the JSON report. Lighthouse CI (lhci collect) saves one report per run in .lighthouseci/; post them as an array:
npx lighthouse https://staging.quillmere.example/checkout --only-categories=accessibility \
--output json --output-path lighthouse.json --chrome-flags="--headless"
curl -X POST "$METRICKLE_SCAN_HOOK?version=$GIT_SHA" \
-H "content-type: application/json" --data-binary @lighthouse.json
# Lighthouse CI: every report from the last collect, in one request
jq -s . .lighthouseci/lhr-*.json | curl -X POST "$METRICKLE_SCAN_HOOK" \
-H "content-type: application/json" --data-binary @-
Metrickle reads only the report’s accessibility category. Lighthouse runs axe-core for it, so each failed audit is an axe rule with its impact and WCAG criteria. Passed, not applicable and manual audits aren’t violations, and other categories (performance, SEO) are ignored. An audit that fails without naming an element, like a missing page title, is kept as a violation on the page.
What Metrickle accepts
- One axe-core results object, an array of them (as the axe CLI saves), or
{ "results": [...] }. - One Lighthouse report (JSON with
lighthouseVersionandaudits), an array of them, or{ "results": [...] }. Reports and axe results can be mixed. - Each axe result needs
url, or apathstarting with/. Lighthouse reports usefinalDisplayedUrl, thenfinalUrl, thenrequestedUrl. Metrickle files violations under the page’s path. versionandenvironment, each up to 64 characters, are optional. Send them in the query string or as top-level fields of an object body. They’re shown with the scan.
Limits
| Limit | Per scan |
|---|---|
| Request size | 10 MB. Split larger scans by page |
| Pages | 500 |
| Elements per rule on a page | 25 |
| Violations | 5,000 |
How results are kept
Each page keeps only its latest scan. A page in a new upload replaces that page’s earlier violations, so a clean run clears it. Pages that aren’t in the upload keep their last results.
Several runs of one page in the same upload, for example at different screen widths, are combined: each rule and element is listed once.
Scans in visitors' browsers
CI reaches the pages you script it to visit. Signed-in pages, pages behind a search or a basket, and pages nobody thought to list usually go unscanned. For those, the web SDK (the script tag, and the npm package from its next release) can check pages in your visitors' browsers. It’s off until you turn on Check pages in visitors' browsers under the app’s Settings → Research, and it’s web only.
How it works:
- On a share of page views (5% by default, set beside the switch), once the page has loaded, settled for 3 seconds and the browser is idle, the SDK asks Metrickle whether the page is due a check. A page CI has scanned at the same path never is: CI’s results always win. Otherwise a page is checked at most once a day, across all visitors.
- Only when it is due does the SDK load the checker:
m-a11y.jsnext tom.js(axe-core 4.11, about 165 KB gzipped), or a separate chunk of the npm package.m.jsitself doesn’t grow by more than a few hundred bytes. The checker runs axe-core’s WCAG 2.0, 2.1 and 2.2 level A and AA rules on the page, in idle time, without frames and without fetching stylesheets. - The SDK sends the violations to
POST https://in.metrickle.com/v1/a11y-scanand they’re kept like a CI scan of that page, marked as from visitors' browsers.
What it sends, and what it doesn’t:
- Sent: the page path, each failing rule (its id, impact, WCAG tags, help text and link) and, for each failing element, its opening tag with only
id,class,name,typeandrole, and the selector friction uses (button#pay.btn.primary). - Never sent: page text, input values, labels, alt text, links, other attributes, axe’s failure summaries, the query string, or any visitor, session or user id. Metrickle rebuilds each element’s tag from the same short list on arrival, whatever is sent.
- Paths with ids in them are filed under
:id: a check of/orders/8812or/u/ada@quillmere.exampleis filed as/orders/:idor/u/:id. Any path segment that’s all digits, has a digit and is 6 or more characters long, or contains@counts as an id. Readable slugs, such as/u/quillmere-ada, are kept as they are, so leave this off if your paths name people. - Left out: elements inside
data-mk-block. Elements insidedata-mk-maskordata-privateare sent as their tag,typeandroleonly, without id, class, name or selector. - It doesn’t run once a visitor has opted out (Do Not Track, Global Privacy Control or
optOut()), when the browser has Save-Data on, under browser automation, or in browsers without idle callbacks (requestIdleCallback, which Safari lacks). In cookieless mode it stores nothing in the browser; otherwise it notes each checked page in local storage for a day (mk_ax:<path>).
An app keeps browser checks for at most 1,000 pages. Pages CI scans don’t count towards that.
Responses
| Status | Body | Means |
|---|---|---|
201 | { "ok": true, "scan": {...} } | Stored |
400 | { "error": "invalid_json" } | The body isn’t JSON |
400 | { "error": "not_axe_results", "message": "…" } | The body isn’t axe-core results or a Lighthouse report, or no result had a page address. message says which |
404 | { "error": "not_found" } | The URL is wrong or was replaced, or the app is archived |
413 | { "error": "payload_too_large" } | The body is over 10 MB |
429 | { "error": "rate_limited" } | Too many requests. Try again shortly |
The scan object has id, tool (axe-core or lighthouse and the version that ran), version, environment, pages, violations and at (Unix milliseconds).
Read results from an assistant
An assistant connected over MCP can read the latest scans with get_a11y_scans and the violations on a page with get_a11y_violations. get_case includes the violations on the case’s page and element.