Eurovision Song Contest (ESC) is an international song competition known for its musical extravaganza and inclusivity. The 2023 event was viewed live by 162 mln people with largest audiences tuning in from the UK, Germany, Italy, and Spain.
Over the last few years, the countries hosting Eurovision have tried to make it not only inclusive but also more accessible both for offline and online viewers. So, does Eurovision truly live up to its vision of being a celebration for all?
In this article, we’ll focus primarily on the online aspect of Eurovision, namely its official and fan-made apps. We’ve performed accessibility testing to see if those apps are adapted to users with disabilities and hopefully help the Eurovision crew deliver smooth experiences for everyone. Stay tuned for the results!
Why accessibility testing needs more attention
The Eurovision Song Contest has made strides to ensure a more accessible experience for all audiences, but there’s still room for improvement. It’s great that we have sign language interpreters for live broadcasts, but what about other digital products associated with Eurovision, like its mobile app? The app can play a crucial role in audience engagement and should also be tested for accessibility. Especially considering that accessibility testing is often overlooked in the development process.
Accessibility testing is a type of software testing that helps determine whether a web solution, desktop, or mobile app can be used by people with disabilities. It includes checking the app’s compatibility with assistive technology software and compliance with global and country-specific accessibility standards like WCAG (Web Content Accessibility Guidelines), EAA (European Accessibility Act), or ADA (Americans with Disabilities Act).
As a company providing accessibility testing services, we can help ensure no one is excluded from enjoying a digital experience. Imagine the joy of a Eurovision fan with low vision who can finally feel the energy of the crowd through descriptive audio. Or the disappointment of a viewer with hearing impairments missing out on the live commentary and song details because there are no closed captions.
So, is the Eurovision Song Contest 2024 ready to welcome fans with disabilities? Let’s find out.
Accessibility bugs in ESC apps
The official Eurovision apps for iOS and Android allow users to keep up with the latest news, learn more about the artists, and participate in fan polls. On the semi-final, final, and grand finale days, users will be able to watch the competition live. Most importantly, one can vote for their favorite singer and cheer from home sending emojis as songs are performed.
As for unofficial apps like ESC Party and My Eurovision Scoreboard, they focus more on the entire Eurovision history, providing a collection of performances dating back to 1956. They also allow creating scorecards together with family and friends and sharing them on social media, along with other handy features.
All Eurovision apps we’ve tested work with TalkBack (screen reader for Android) and VoiceOver (screen reader for iOS) enabled. However, the experience is far from flawless and requires further testing and polishing. Here’s what can be improved right now.
Font size issues
Android and iOS device settings allow users to adjust the font size and weight. Something as simple as a larger font size can make a big difference for people with visual impairments or dyslexia.
Whether you’re testing Android apps or busy with iOS app testing, checking how the increased font size and weight affect usability is a must. Here are a few examples why.
App: Eurovision Song Contest for iOS, version 6.1.10
❌ Actual result: Turning on display & text size accessibility features has no effect on the app’s font size and weight.
✅ Expected result: The app’s font is larger and bolder as per the device settings.
Bugs with font size persist in fan-made apps, regardless of the OS.
App: ESC Party for Android, version 3.6.1
❌ Actual result: When the user increases the font size and weight in device settings, it breaks the app’s design and results in overlapping content.
✅ Expected result: Using a larger and bolder font doesn’t break the app’s layout.
Here is one more example.
App: ESC Party for iOS, version 3.6.1
❌ Actual result: When the user chooses bold text and enlarges it to the maximum, it completely breaks the app’s layout.
✅ Expected result: The app’s layout should remain visually consistent and functional even when the user increases font size and weight to the largest settings.
Here is another third-party Eurovision app that has layout issues after enlarging the font.
App: My Eurovision Scoreboard for Android, version 14.0
❌ Actual result: When the user increases the font size and weight, it affects the app’s layout, resulting in cramped content.
✅ Expected result: The app’s design should be responsive, maintaining visual clarity and functionality across all font size and weight options.
Scroll issues
Scrolling allows us to explore and navigate vast amounts of information. However, scrolling takes on a different meaning for users who rely on screen readers like VoiceOver (iOS) and TalkBack (Android). They use specific gestures, like three-finger swipes for VoiceOver and two-finger swipes for TalkBack, to navigate through content.
Improperly implemented scrolling can lead to frustration and difficulty navigating the app. People with disabilities should be able to access all the app’s features and engage with its content effortlessly. Here is what happens when the scrolling function is not adequately tested using assistive technology.
App: Eurovision Song Contest for iOS, version 6.1.10
❌ Actual result: The three-finger swipe gesture for scrolling the participant page is unresponsive; instead, the entire text is narrated.
✅ Expected result: The three-finger swipe gesture allows the user to scroll the participant page.
Here is another screen where scrolling is hindered.
App: Eurovision Song Contest for iOS, version 6.1.10
❌ Actual result: The three-finger swipe gesture for scrolling the list of participants is unresponsive; instead, random elements are voiced.
✅ Expected result: The three-finger swipe gesture works as intended and allows the user to scroll the page.
The same issue is reproduced in an unofficial Eurovision app.
App: ESC Party for iOS, version 3.6.1
❌ Actual result: When VoiceOver is turned on, the user can’t scroll through the winner page with the three-finger swipe.
✅ Expected result: The user can scroll the winner page with the three-finger swipe.
Scrolling issues may occur not only when using screen readers but also when enabling other accessibility features, such as zoom.
App: ESC Party for iOS, version 3.6.1
❌ Actual result: When ‘Zoom’ enabled in accessibility settings, the user can’t scroll through zoomed content with the three-finger swipe.
✅ Expected result: The user can scroll through zoomed content with the three-finger swipe.
Unlabeled content
Screen readers rely on content labels assigned to app elements like buttons or images to understand their purpose and convey that information to the user. Unlabeled content becomes invisible to the screen reader, leaving the user with no clue about what it is or how to interact with it.
If an app developer forgot to add a content label, screen readers might announce the element as “unlabeled button” or “unlabeled image,” which is not very helpful. This doesn’t tell the user what the button does or what the image depicts. Unlabeled content creates significant barriers for users with visual impairments who rely on screen readers to interact with digital content.
There are two ways to check if an app’s interface is missing content labels. The first is to test the app manually using a built-in screen reader, and the second is to perform automated testing with tools like Espresso, Robolectric, or XCUITest. Here are bugs we discovered manually.
App: Eurovision Song Contest for Android, version 6.2.0
❌ Actual result: When using TalkBack, the selected content is voiced as unlabeled and nothing happens with two taps.
✅ Expected result: There’s a description for each page in pinned feed and it’s possible to open selected pages.
Here is a similar issue but in a different app.
App: ESC Party for iOS, version 3.6.1
❌ Actual result: The name of the button for adding a profile photo is not announced.
✅ Expected result: Every button name should be voiced so that it’s clear what action to take.
Here is one more example of how a lack of content labels for buttons affects the user experience.
App: Eurovision Song Contest for iOS, version 6.1.10
❌ Actual result: When tapping the social media icons, VoiceOver doesn’t announce their names and keeps reading other content on the screen.
✅ Expected result: The icons are voiced when tapped and it’s possible to go to the selected social media page.
Voiceover issues
People with visual impairment heavily rely on audio guidance to access and enjoy content. A fully accessible app, compatible with screen readers, fosters a sense of autonomy and allows users to fully immerse themselves in the digital experience, whether catching the latest Eurovision drama or exploring any other event.
However, when some information is voiced and the other is skipped, it creates an information gap and confuses users about what’s happening on the screen. Here are a few hands-on examples proving the need in comprehensive accessibility testing.
App: Eurovision Song Contest for Android, version 6.2.0
❌ Actual result: There is no sensible voiceover when the user taps on the cookie banner.
✅ Expected result: The text on the cookie banner is voiced properly when selected.
Here is another voiceover issue in a different app.
App: ESC Party for iOS, version 3.6.1
❌ Actual result: While VoiceOver announces the menu itself, it doesn’t voice the selected text.
✅ Expected result: When the user taps a word within the menu, it should be voiced.
Unsynchronized audio
Both Android and iOS offer built-in accessibility features like ‘Select to Speak’ and ‘Speak Selection’ respectively. These features are helpful for people with low vision who may have some remaining sight but find it difficult to read because of tiny font size, glare, or visual distortions.
Users with dyslexia or learning disabilities may also use these features for auditory reinforcement alongside the visual text, which helps with understanding and focus.
However, for these features to be truly effective, the voiced text should always coincide with the visual cues highlighting the content being spoken. Otherwise, it only detracts from the experience, like in the example below.
App: Eurovision Song Contest for Android, version 6.2.0
❌ Actual result: The green border of the text in focus does not correspond to the text being voiced.
✅ Expected result: The visual indication of the currently selected element fully matches the audio track.
Video usability
For an app devoted to Eurovision, video usability should be made a priority. People with disabilities deserve the same opportunities to experience the richness of video content as everyone else. Ensuring screen readers can control playback features like play, pause, rewind, and fast forward is crucial.
You don’t want a person to be stuck listening to the entire video, unable to pause if they need a break or rewind to catch a missed detail. This lack of control creates a frustrating and isolating experience.
App: Eurovision Song Contest for iOS, version 6.1.10
❌ Actual result: Once the video is started, the progress bar disappears and it’s impossible to stop or rewind the video.
✅ Expected result: It should be possible to bring up the progress bar.
Other bugs
Accessibility issues manifest in different ways. Sometimes, it’s a pure functional bug, like a button not working. Other times, it’s VoiceOver announcing things weirdly. You never know what bugs you’ll encounter unless you navigate the app from A to Z with a screen reader and other accessibility features enabled.
Usability and accessibility are often intertwined. For instance, the frustration of being unable to complete an action, such as creating a scorecard, is not exclusive to users with disabilities. It’s equally felt by those without. Similarly, enhancing color contrast not only helps users with low vision but also enhances the visual appeal of the interface for all users.
Here are examples of functional and security issues directly affecting the app’s usability and accessibility.
App: ESC Party for Android, version 3.6.1
❌ Actual result: The ‘Choose a contest to score’ button doesn’t work when TalkBack is switched on.
✅ Expected result: The user can choose and add a contest to their scorecard with TalkBack on.
Here is another example of how a functional bug can limit access to certain features and content.
App: My Eurovision Scoreboard for Android, version 14.0
❌ Actual result: When the user tries to go to the Facebook page with TalkBack on, the app returns an error.
✅ Expected result: It is possible to go to the Facebook page with TalkBack enabled.
People with disabilities deserve the right to privacy but software bugs can undermine it.
App: ESC Party for Android, version 3.6.1
❌ Actual result: The username remains visible after logging out.
✅ Expected result: The sign-in screen displays no username after logging out.
Lack of localization testing can lead to confusing and frustrating experiences, especially for users unfamiliar with the other language, as in the example below.
App: My Eurovision Scoreboard for Android, version 14.0
❌ Actual result: There is Russian text in the English version of the app.
✅ Expected result: All screens and modal windows should be displayed and voiced in the preferred language; in this case, it’s English.
Summing up
Accessibility testing is a quite demanding process. The bugs we showcase here are just the tip of the iceberg. We tested only some of the most common accessibility features, whereas mobile devices offer many more options for users with disabilities to interact with apps.
We hope that this article will help bring attention to the needs of people with disabilities and help deliver more accessible and inclusive digital experiences in the future. Eurovision is a large-scale and vibrant event that brings together nations through a captivating blend of music, performance, and a healthy dose of friendly competition. All people, regardless of their ability, should be able to experience its magic, celebrate, and join the conversation.
If you’re unsure where to begin with accessibility testing, consider our Bug Crawl program. It offers a free, no-obligation exploratory review of your app, complete with a bug report outlining any accessibility concerns we find.
Improve your app’s accessibility!
Our testers can perform free exploratory testing through our Bug Crawl program. Sign up to receive a detailed bug report identifying any accessibility issues we find.
