Nightwatch E2E Accessibility Testing

A masochistic experiment of including accessibility tests into end-to-end testing via Nightwatch.js, testing against @ted_drake's a11y-control-test. Tests include HTML validation,
keyboard tab tests, and accessibility test using Deque's aXe and Tenon by @karlgroves.

Slides

Slides are available at: https://unfetteredthoughts.net/slides/a11yTools/index.html

Installation

Install Node.js and then:

$ git clone https://github.com/gerardkcohen/nightwatch-a11y-testing.git
$ cd nightwatch-a11y-testing
$ npm install

Run tests

Launch Nightwatch, using any of the available Command-line Options.

Available WebDriver Environments

The following WebDriver environments are supported:

• Firefox (via GeckoDriver) - default
• Chrome (via ChromeDriver) - chrome
• Safari < v10 (via Selenium HQ) - safari

You can specify any particular WebDriver environment by using the --env option.

$ nightwatch // all environments
$ nightwatch --env default // firefox
$ nightwatch --env chrome
$ nightwatch --env safari

Available Groups

You can specify the group using the --group option. There are two main Nightwatch groups, one using aXe and the other using Tenon. The are further broken up by page. Groups are based on the tests directory structure.

nightwath-a11y-testing/
  |
tests/
  ├── aXe
  |   ├── about
  |   └── apply
  |   └── products
  └── Tenon
      ├── about
      └── apply
      └── products

$ nightwatch // all groups
$ nightwatch --group aXe
$ nightwatch --group aXe/about
$ nightwatch --group aXe/apply
$ nightwatch --group aXe/products
$ nightwatch --group Tenon
$ nightwatch --group Tenon/about
$ nightwatch --group Tenon/apply
$ nightwatch --group Tenon/products

Available Tags

Since the a11y-control-test project has both good and bad examples, these tests are further
split into 2 tags, simply good and bad. You can specify the group using the --tag option.

$ nightwatch --tag good
$ nightwatch --tag bad

Environments, Groups, and Tags

Of course, Nightwatch gives you the ability to use any combination of environments, groups, and tags.

$ nightwatch --env chrome --group aXe --tag good // test all good tests with aXe using Chrome
$ nightwatch --env safari --group Tenon/apply --tag bad // test the bad apply page with Tenon using Safari

Additional Information

I am not endorsing any of the products used here, or even the idea of this level of testing. Automated accessibility testing is really nice, but it should be a supplement to testing with actual assistive technologies, preferably with actual assistive technology users. I am thankful for the providers of the tools I used for this example. Please check the RESOURCES for more information.

History

• 05/17/2016 - Update npm dependencies and slides
• 09/10/2016 - Update npm dependencies, add visual/ CSS bookmarklet tests