Metrickle

Accessibility breakdowns

Last updated

Most analytics count people using accessibility settings in with everyone else, so a checkout that people using keyboard navigation can’t finish looks like a small dip. Metrickle records which settings each visitor is using and compares them with people using no accessibility settings. You see who converts less, where they drop out, and what it costs.

Settings come from the device or browser. Nobody is asked, and Metrickle doesn’t try to detect disability. Plenty of people use large text because their phone is small, and plenty of disabled people use no setting at all. Settings tell you how someone is using your product, which is what a fix needs.

What’s detected

The SDKs detect settings automatically. What each platform can see differs, because browsers and operating systems expose different things.

SettingLabel in the dashboardWebiOS and Android
KeyboardKeyboard navigationFive Tab presses, once Tab presses outnumber pointer pressesA hardware keyboard, or Switch Control (iOS) or Switch Access (Android)
ZoomZoomed inThe page is magnified with pinch zoomNot detected
Large textLarge textThe browser’s default font size is above 16pxThe system text size is above the default
High contrastHigh contrastprefers-contrast: moreIncrease Contrast or high-contrast text
Forced coloursForced coloursforced-colors: active, such as Windows contrast themesNot applicable
Reduced motionReduced motionprefers-reduced-motion: reduceReduce Motion, or animations turned off
Screen readerScreen readerNot detectedVoiceOver or TalkBack is running

Browsers deliberately hide whether a screen reader is running, so on the web Metrickle reports sustained keyboard navigation instead. That covers many people using a screen reader, and many people who don’t use one.

Flutter and React Native read the same system settings as iOS and Android, with two differences on React Native: it doesn’t detect keyboard use, and you pass AccessibilityInfo and the font scale to the SDK when you start it.

Depending on the platform, the SDKs also report Reduced transparency, Inverted colours, Greyscale and Bold text. A visitor can have several settings at once, and a setting turned on part-way through a visit is picked up from then on.

On the web, keyboard navigation is remembered for the rest of the browser tab’s session. In cookieless mode nothing is stored, so it’s detected again on each full page load. To stop collecting settings on the web, turn off the a11y option when you start the SDK. See the web SDK guide.

Where you see it

Overview

On the app’s Overview, the Technology card has an A11y tab. It lists visitors by setting, including No accessibility settings, with each one’s conversion rate if you’ve set up goals. Select a setting to filter the whole page by it.

Friction

The Friction page has an Accessibility card. For each setting it shows sessions, the friction rate and the conversion rate, with the difference in points from sessions with no accessibility settings. Select a setting to filter by it. See friction.

Filters everywhere

Once you’ve filtered by a setting, funnels, journeys, retention and replays all show that group only. This is the quickest way to answer “where do people using keyboard navigation drop out of checkout?”

Protected tasks

Every protected task compares each setting with people using no accessibility settings after each release. By default a setting needs attempts from 15 different people before it’s judged. If it’s more than 10 points behind, or its slowest 10% take 1.5 times as long, and the gap is clear of noise, the verdict is Gap found, naming the setting and the step where those people fall behind. Until it’s clear, it’s an Early signal.

When a protected task breaks for a setting, the case is about those people: what they hit, on which element, and the recordings that show it.

A feedback report marked as an accessibility barrier takes back a completion from the same person within seven days. It counts against the setting that was on when the report was sent.

Research

  • Studies show each task’s success by accessibility setting.
  • Surveys can be shown only to people using particular settings.
  • Feedback reports and Replays list the settings in use.

See surveys and feedback and heatmaps and replay.

Portfolio

Portfolio answers “Can everyone finish your protected tasks?” across all your apps, with each protected task’s latest verdict.

Accessibility scans from CI

Automated checkers find problems in the code, but can’t say who they affect. Metrickle puts the two together. Post axe-core results or Lighthouse reports from your CI to the app’s scan hook, and each violation shows next to the friction on the same page or element.

To set it up:

  1. Open your app and select Integrations.
  2. Under Accessibility scans, select Set up scan hook.
  3. Copy the URL into your CI secrets. Anyone with it can replace this app’s scan results.
  4. After each deploy to staging or preview, post the results from the axe CLI, from Playwright tests using @axe-core/playwright, or from Lighthouse. The dialog shows examples.

Each page keeps only its latest scan, so a clean run clears it. View pages lists scanned pages with their critical, serious and total violations, worst first.

On the Friction page, a hotspot with a violation on the same element shows the finding, its impact and the WCAG criterion. Pages show their violation count, or “Not scanned”. A case shows the violations on its step’s page and element under Accessibility scan, and an issue filed from a case or a hotspot lists them too.

Plans

Every plan includes the accessibility split. See pricing for what each plan includes.

Scans in visitors' browsers

CI only scans the pages you tell it to. Signed-in pages and pages deep in a flow are often missed. For web apps, Metrickle can check those pages in your visitors' browsers instead:

  1. Open your app and select Settings.
  2. Under Research, turn on Check pages in visitors' browsers, choose the share of page views that may check their page, and save.

A chosen page view checks its page against WCAG A and AA once the page has settled and the browser is idle, at most once a day per page. Only the failing rules and each element’s tag, id, class, name, type and role are sent: never page text, what people type, or who was visiting. Pages your CI scans keep CI’s results. View pages under Integrations → Accessibility scans shows where each page’s scan came from: CI or Visitors' browsers. See scans in visitors' browsers for the details.