Thesis project regarding WCAG 2.1 and the European Accessibility Act
Background – As my thesis project for my bachelor’s degree at Stockholm University, I conducted a comparative study regarding WCAG 2.1 and the European Accessibility act which will become law starting June 2025.
Problem – It is estimated that 15 % of the world’s population has some form of disability that affects their capability to assess information and use services. With that being said, there is a lack of empirical studies validating whether conforming to the EU Directive or WCAG 2.1 ensures accessibility for users with disabilities.
Method – Qualitative and quantitative data was generated through usability tests with both a test group of participants with varying disabilities, as well as a control group. The accessibility issues encountered were then compared to WCAG 2.1 and Directive 2019/882 to discern which errors would be solved through compliance, and what issues would not be solved.
Result – Out of the 48 found accessibility issues, 27 issues could be addressed by following the WCAG 2.1 guideline, and 26 of the issues could be addressed by following the Directive 2019/882. Blind users, visually impaired users, and users with cognitive impairments were overrepresented when it came to encountered issues left unaddressed by the guidelines.
What is accessibility?
Accessible design is a design process in which the needs of people with disabilities are specifically considered. The need for accessible design is prevalent, but the way those needs express themselves can look very different. It is estimated that 15 % of the world’s population has some form of disability that affects their capability to assess information and use services, and those impairments can take different forms: either situational, temporary or permanent (World Health Organization, 2011). To gain a greater understanding of the topic of accessibility on the web, it’s important to understand what types of accessibility issues exist and how they present themselves.
Accessibility Directives
WCAG 2.1
The industry standard for web accessibility is WCAG (Web Content Accessibility Guidelines). The guidelines are divided into three levels: A (lowest), AA (mid range), and AAA (highest) where AA is the recommended level to strive for. The current version of WCAG is 2.1 and it is based on four principles:
– Perceivable: Content must be detectable, users must be able to recognize the information that is being presented to them.
– Understandable: Users must be able to process the information in front of them and also be able to understand how to use and navigate the website.
– Robust: Robust states that content must be robust enough that it can be interpreted reliably by a wide variety of user agents, including assistive technologies.
EU Accessibility Act
The European Parliament passed the European Accessibility Act (EAA) in 2019, requiring accessibility compliance for ICT services. All e-commerce websites, except a few exceptions, are covered by this law. Compliance is required for new services from June 2025 and there is a grace period of a couple of years for existing services that need to be updated.
The research question was ...
What is the effectiveness of WCAG 2.1 or Directive 2019/882’s ability to prevent accessibility problems on e-commerce websites, and what issues are not prevented?
The participants
To be able to answer the research question, usability testing was conducted with 20 participants, 10 in the control group and 10 in the test group.
- 9 men and 11 women participated
- Age group was between 20-58 years old.
- The test group consisted of: 2 Blind, 1 colorblind, 1 mobility impaired, 1 dyslexic and 5 visual impaired users.
- User performed test on mobile only except two users who needed to use computers because of their disability.
- Tests were performed both remotely and onsite in Lookback.
- All tests were performed in Swedish.
Method
To be able to identify what errors were accessibility issues and what were general usability issues, all usability errors found through the usability tests were divided into three categories:
– Issues encountered by control group
– Issues encountered by both group
– Issues encountered only by the test group
If the usability issue was encountered by only the control group or both the control group and the test group, it was categorized as a general usability issue. However, if the issue only was found by the test group and it derived because of the participant’s disability, it was categorized as an accessibility issue.
The task
Two Swedish telecom websites were tested, we call them Website A and Website B. The participants got one task that they performed on both websites:
“Imagine that you are a person who often travels to Norway. Now you are going to buy a new mobile subscription. Buy a mobile subscription at [Website A/Website B] where you have at least 10 gigabytes of mobile data in Norway.”
The results
The test resulted in 48 found accessibility issues across the two websites. 26 issues were found on website A and 22 issues were found on website B. Out of the issues found, a majority of them were encountered by the blind users in the test group. They encountered 67 % of all accessibility issues found, which really shows the reality of what blind individuals have to face in their day to day experiences with inaccessibility on the web.
Unsurprisingly, the approval rating of the websites were significantly lower by the disabled participants compared to the non disabled participants. When asked to rate the websites from 1-10, the disabled gave the websites an average score of 5 and 6,5 respectively, whereas the non disabled participants gave the same websites a 7 and an 8,5.
Out of these 48 accessibility issues found, 27 issues could be addressed by following the WCAG 2.1 guideline, and 26 of the issues could be addressed by following the Directive 2019/882.
This means that by following the accessibility guidelines, 42 % of website A’s issues and 73 % of the issues found on website B could be solved.
Blind users encountered 32 issues in total where 10 were left unaddressed by the guidelines, and visually impaired users encountered 11 issues in total where six of them were left unaddressed. All issues categorized as cognitive accessibility issues were left unaddressed. This shows that the existing guidelines are a good step forward, but does not solve all issues encountered by disabled users. The results may be used to argue whether Directive 2019/882 and WCAG 2.1 are appropriate guidelines to determine accessibility on the web, which may have societal consequences.
Examples of found accessibility issues
Below you can see five examples of found accessibility issues.
