Accessibility Testing with axe and a Keyboard: What Each One Catches

Introduction

"Is this page accessible?" is not one question, and no single tool answers it. An automated scan checks rules that a machine can decide. A keyboard pass checks whether you can actually operate the page. The accessibility tree shows what a screen reader would be handed. They overlap, but each one sees things the others miss.

This guide runs all of them against the same page, with one deliberate bug, and compares the answers. Every command and output below comes from runs on 2026-09-16 with Playwright 1.63.0 and axe-core 4.13.0.

The page under test

The target is this site's own practice shop, which is free, ad-free, and built for exactly this. Its checkout page asks for a first name, a last name, and a postal code.

The shop has ten variants, each with one seeded bug. Variant v10 breaks the postal code field: the <label> still says "Postal code" and still points at postalCode, but the input's id was changed to postal. Nothing moves, nothing looks different, and the label text is still on screen.

Open the correct shop with ?variant=clean and the broken one with ?variant=v10. Everything below was run against both.

Test only sites you own or have permission to test. That is why this guide uses our own practice shop rather than someone else's.

Set up the scan

Playwright's accessibility testing docs recommend @axe-core/playwright, which injects the axe engine into a page you already drove to the right state:

npm install --save-dev @axe-core/playwright

The scan itself is a normal test:

// tests/a11y.spec.ts
import AxeBuilder from "@axe-core/playwright";
import { expect, test } from "@playwright/test";
 
test("checkout has no serious or critical WCAG 2.1 AA violations", async ({ page }) => {
  await page.goto("/practice/shop/checkout");
  const { violations } = await new AxeBuilder({ page })
    .withTags(["wcag2a", "wcag2aa", "wcag21a", "wcag21aa"])
    .analyze();
 
  const serious = violations
    .filter((v) => v.impact === "serious" || v.impact === "critical")
    .map((v) => `${v.id}: ${v.help} (${v.nodes.length}× e.g. ${v.nodes[0]?.target.join(" ")})`);
  expect(serious).toEqual([]);
});

Two details matter more than they look.

withTags decides what "accessible" means here. Without it, axe runs every rule it has, including best practices that no standard requires. Naming the WCAG tags you are held to keeps the result honest and the failures actionable.

Filtering by impact is a choice, not a rule. axe labels each violation minor, moderate, serious, or critical. Failing only on serious and critical is a reasonable starting point for an existing product, but it does mean moderate problems pass silently. Decide that deliberately.

Scan the correct shop first

Run the scan against the clean shop before touching the buggy one, so you know what a pass looks like:

{"variant":"clean","axeVersion":"4.13.0","violations":[]}

Zero violations. It is worth being precise about what that does and does not mean. It means no rule that axe can decide automatically was broken on this page in this state. It does not mean the page is accessible. Playwright's own documentation says automated tests "can detect some common accessibility problems", and that many problems "can only be discovered through manual testing". We come back to that at the end, with a measured example from this very page.

Scan the broken variant

The same scan against v10:

rule id:  label
impact:   critical
help:     Form elements must have labels
tags:     wcag2a, wcag412
target:   #postal
 
Fix any of the following:
  Element does not have an implicit (wrapped) <label>
  Element does not have an explicit <label>
  aria-label attribute does not exist or is empty
  aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
  Element has no title attribute
  Element has no placeholder attribute
  Element's default semantics were not overridden with role="none" or role="presentation"

Read a violation in four parts:

The rule id (label) is the stable name. Look it up: Deque's page for this rule is titled "Form elements must have labels".
The impact (critical) is axe's severity for the rule, not a measurement of your users' pain.
The WCAG tag (wcag412) points at the success criterion, here 4.1.2 Name, Role, Value. That is the sentence you quote in a bug report.
The target (#postal) is the element. This is the part people skip, and it is the part that tells you where to look.

The failure summary is a list of every way the element could have got a name, each of which failed. That is the fix list: give it a real <label for="postal">, or an aria-label, or wrap it.

The same bug in the accessibility tree

A scan tells you a rule failed. The accessibility tree shows you what a screen reader is actually handed. Playwright can print it:

const form = page.locator("form").first();
console.log(await form.ariaSnapshot());

The correct shop:

- text: Postal code
- textbox "Postal code"

Variant v10:

- text: Postal code
- textbox

The words "Postal code" are still on the page, as plain text. The textbox next to them has no name at all. A sighted user sees a labelled field; a screen reader announces an unlabelled edit box. That one line is the clearest possible explanation of the bug, and it belongs in the bug report.

Your locators are already an accessibility check

If your tests find fields the way a person describes them, they fail when the description breaks. Counting matches on both variants:

clean:  First name 1, Last name 1, Postal code 1
v10:    First name 1, Last name 1, Postal code 0

A test written like this already catches the bug, with no accessibility tooling at all:

test("checkout asks for a postal code", async ({ page }) => {
  await page.goto("/practice/shop/checkout");
  await expect(page.getByLabel("Postal code")).toBeVisible();
});

Whereas a test written against page.locator("#postal") or a test id would keep passing, because those do not care whether the field has a name. This is the cheapest accessibility win available: prefer getByLabel and getByRole in ordinary functional tests, and naming bugs start failing your suite for free.

Now try the keyboard

Keyboard access is the check people reach for first, and here it is instructive. Tabbing from the top of the checkout page, on the correct shop:

Practice Shop link → Cart link → Log out button → First name → Last name → Postal code → Back to cart link → Continue button

On v10, the order is identical. All three inputs are reachable. Each one draws the same focus outline, solid 2px rgb(110, 29, 41), on both variants.

So a keyboard pass over v10 finds nothing. The field is reachable, focusable, visibly focused, and typing into it works. Nothing about moving through the form reveals that it has no name, because a name is not something you can see or feel with a keyboard.

That is the lesson worth keeping: the technique that finds a bug depends on the kind of bug it is. A missing name is invisible to the keyboard and obvious to a scan. A keyboard trap, or a focus order that jumps around the page, is invisible to a scan and obvious the moment you press Tab. Neither replaces the other.

What passed every check and was still wrong

Go back to the correct shop, the one with zero axe violations, and submit the form empty:

- textbox "First name"
- alert: "Error: First name is required"

The error is exposed as an alert, so it is announced. Good. Now look at the field it is about:

firstName input, after a failed submit:
  aria-invalid:      null
  aria-describedby:  null
  aria-required:     null
error element id:    null

Nothing marks the field as invalid. Nothing connects the message to the field, and the error element has no id for anything to point at. Focus stays on the Continue button. A screen reader user hears "Error: First name is required", then has to go find which box that was about, and nothing tells them they have arrived at it.

axe reported zero violations on this page while all of that was true. Not because axe is bad, but because "the error is not associated with its field" needs judgement about which field the message refers to, and that is not a decision a scanner can make reliably. This is what the documentation means by problems that only manual testing finds.

Exercise: find the bug three ways

On the practice lab, switch the shop to v10 and reach the checkout details page.

Find the bug with an axe scan. Write down the rule id, the impact, the WCAG tag, and the target.
Find it again without any accessibility tool, using only a locator that describes the field the way a person would.
Try to find it with the keyboard alone. Tab through the form, note the order, and write down what you can and cannot tell.
Then go back to the correct shop and find one problem that the scan does not report at all.

Answer key

1. The scan. Rule label, impact critical, tag wcag412 (WCAG 4.1.2 Name, Role, Value), target #postal. The failure summary lists every naming route that was missing: no wrapped label, no explicit label, no aria-label, no aria-labelledby, no title, no placeholder.

2. Without accessibility tooling. page.getByLabel("Postal code") matches once on the correct shop and zero times on v10. Any test that locates the field by its label or role fails; a test using #postal or a test id does not notice.

3. The keyboard. Nothing to find. Tab order, reachability, and the focus outline are identical on both variants. This is the expected answer, not a trick: the bug is a missing accessible name, and the keyboard never asks for one.

4. On the correct shop. Several possible answers, all measured above: a failed submit leaves the field without aria-invalid, the error is not linked to the field by aria-describedby, the error element has no id, and focus is not moved to the first invalid field. The page still passes the axe scan with zero violations.

What to do with this

Put an axe scan in CI on the handful of pages that matter, scoped with withTags to the standard you are held to, and treat new serious or critical violations as failures.
Prefer getByLabel and getByRole in ordinary tests. It costs nothing and turns naming regressions into test failures.
Keep a short manual pass for the things scans cannot decide: tab order, focus after an action, error association, and whether the flow is usable at all.
When you excluded something to get the suite green, write down why. exclude() skips every rule on those elements, not just the noisy one, so an exclusion hides more than it looks like it does. disableRules() is narrower when one rule is the problem.

Conclusion

The bug in this guide was one changed id. A scan named it in milliseconds and told you the exact element and criterion. A locator that described the field like a person would failed on it without any tooling. A careful keyboard pass missed it completely, and a page with zero violations still left a screen reader user hunting for the field an error was about. Use the tools together, and know which question each one answers.