Testing is a vital phase in software development and an absolute must for flawless software aligned with customer needs. Testing software manually can consume a significant amount of time and effort when done by individuals lacking experience and the necessary skills in this area.
But what about machines? This is precisely what we’ll be discussing in this article — automation testing with Selenium.
Selenium is one of the most popular and widely used open-source frameworks for web automation that accelerates testing, minimizing human errors and ensuring consistent test results. However, similar to any framework or tool, it comes with its set of advantages and disadvantages. The following article addresses the main limitations and challenges in automation testing with Selenium, along with potential solutions to tackle them.
Main Challenges in Selenium
So, without further ado, let’s review the most common challenges in automation testing:
1. Popups and Alerts Handling
In web automation, an alert is a message box that contains information or notifications for the user. It may require additional actions, such as entering data, or ask for permission to proceed with the next steps. Interacting with a web application involves dealing with different types of popups and alerts. Among them, we can define the following groups:
- Browser level notifications
- Web-based alerts (JavaScript alerts)
- OS-level popups
Browser level notifications
To this group, we can include examples of alerts such as:
- “Know your location”
- “Get access to camera/microphone”
- “Show notifications”
Solutions for browser level notifications
Since these alerts are related to the browser level, their handling varies depending on the browser. For instance, in the case of the Chrome browser, we can utilize the ChromeOptions class, which includes specific methods for configuring ChromeDriver capabilities. Below, you’ll find code snippets for handling “Geolocation” and “Notifications” alerts.
public class BrowserAlert {
protected WebDriver webDriver;
@BeforeMethod (alwaysRun = true)
public void setUp(){
WebDriverManager.chromedriver().setup();
// create object of ChromeOptions class
ChromeOptions options = new ChromeOptions();
// disable notifications
options.addArguments("--disable-notifications");
// disable geolocation
options.addArguments("--disable-geolocation");
webDriver = new ChromeDriver(options);
}
}
public class BrowserAlert {
protected WebDriver webDriver;
@BeforeMethod (alwaysRun = true)
public void setUp(){
WebDriverManager.chromedriver().setup();
// create Map to store preferences
Map prefs = new HashMap<>();
// set appropriate value: 1-allow; 2-block; 0-default state
// notifications
prefs.put("profile.default_content_setting_values.notifications", 1);
// geolocation
prefs.put("profile.default_content_setting_values.geolocation", 1);
// create object of ChromeOptions class
ChromeOptions options = new ChromeOptions();
// set Experimental Option - prefs
options.setExperimentalOption("prefs", prefs);
// initialize ChromeDriver with predefined settings
webDriver = new ChromeDriver(options);
}
} Web-based alerts
There are the following alert types in Selenium:
- Simple Alert: This is used to display information to the user. A simple alert contains a message with an [OK] button.
- Prompt Alert: This alert contains a text field within its box and is used when the user needs to input some data into it.
- Confirmation Alert: This alert comes with the option to either accept or dismiss it (using [OK] or [Cancel] buttons).
Solutions for web-based alerts
Alerts, as mentioned above, can be managed through the Alert interface, which is available in Selenium WebDriver. This interface offers appropriate methods for interacting with alerts:
- void accept()
- void dismiss()
- String getText()
- void sendKeys(String text)
Below are a few code snippets for handling alerts:
Accept Alert:
wait.until(ExpectedConditions.alertIsPresent());
Alert alert = webDriver.switchTo().alert();
// press the [OK] button
alert.accept();Dismiss Alert:
wait.until(ExpectedConditions.alertIsPresent());
Alert alert = webDriver.switchTo().alert();
// press the [Cancel] button
alert.dismiss();Get text from Alert:
wait.until(ExpectedConditions.alertIsPresent());
Alert alert = webDriver.switchTo().alert();
// get Alert text
String alertText = alert.getText();Enter data into text field in alert:
wait.until(ExpectedConditions.alertIsPresent());
Alert alert = webDriver.switchTo().alert();
// enter text into Alert
alert.sendKeys("Sample Alert text");
alert.accept();OS-level popups
During the automation process of web applications, there are cases when we need to interact with desktop popups (e.g. upload/download popups, etc.). However, it’s impossible to handle such OS-level popups using Selenium WebDriver, as it only interacts with browsers.
Solutions for OS-level popups
In such situations we need to investigate and integrate third-party tools (like AutoIT, WinAppDriver etc), which provide possibilities to automate windows-based applications.
2. Handling Dynamic Content
Many web applications and websites contain dynamic web elements, which are characterized by the following features:
- Elements appear on the page after a certain amount of time
- Elements become visible on the page after specific user actions
- Different elements are available for different users, according to their requirements
- Elements have dynamic attributes (e.g., IDs)
Automating web applications with dynamic content can become challenging in automation testing with Selenium. This is because locators for web elements can change frequently, leading to potential reasons for test failures.
Solutions for Dynamic Content Handling
Dynamic web elements can be managed using the following options:
1. Using Waits in Selenium WebDriver (Implicit, Explicit, and Fluent)
- Implicit wait. This instructs the WebDriver to wait for a certain period of time before throwing an exception. Implicit waits can be declared in the following format:
webDriver.manage().timeouts().implicitlyWait(Duration.ofSeconds(30));- Explicit wait. This involves instructing the WebDriver to wait for specific conditions to be met or for a maximum time to be exceeded before throwing an exception. Several types of Expected Conditions (as examples), which can be used in Explicit Wait, are listed below:
- alertIsPresent()
- elementToBeClickable()
- elementToBeSelected()
- frameToBeAvailableAndSwitchToIt()
- presenceOfElementLocated()
- visibilityOfElementLocated()
For declaring Explicit Wait we need to create an object of the WebDriverWait class:
WebDriverWait wait = new WebDriverWait(webDriver, Duration.ofSeconds(30));
wait.until(ExpectedConditions.elementToBeClickable(By.id("element_id")));- Fluent wait. This defines the maximum period of time to wait for an appropriate condition and the frequency with which to check this condition. Users may also configure the wait to ignore certain exceptions while searching for an element on the page.
Wait wait = new FluentWait<>(webDriver)
.withTimeout(Duration.ofSeconds(30))
.pollingEvery(Duration.ofSeconds(20))
.ignoring(NoSuchElementException.class);
wait.until(ExpectedConditions.alertIsPresent());
} 2. Using dynamic XPath / CSS selectors
In the case of dynamic content, we may not be able to identify web elements using locators such as id, name, tagName, className, etc. In such situations, custom XPath or CSS selectors should be used.
3. Timeout Handling and Synchronizing Events
This challenge has become one of the most common and widespread issues in automation testing. Synchronization issues occur when we need to perform operations with web elements that are not yet present on the page or are not in an appropriate state (not clickable, not enabled, not visible, etc.). In most cases, these issues result in test script failures. Therefore, it’s crucial to address these challenges to enhance the stability and accuracy of test scripts.
Solutions for Timeout Handling
To address the above-mentioned issues, the following methods are available:
- Using waits (Implicit, Explicit, Fluent). These are accessible within Selenium WebDriver (as described in the previous paragraph)
- Using timeout methods, such as pageLoadTimeout(), scriptTimeout(), etc. The values for these timeouts are configured at the WebDriver level
webDriver.manage().timeouts().pageLoadTimeout(Duration.ofSeconds(50));
webDriver.manage().timeouts().scriptTimeout(Duration.ofSeconds(60));4. Limited Reporting
Reporting plays a crucial role in the entire test automation process. Detailed reports assist QA automation engineers in analyzing results and obtaining accurate information about the test run, including execution time, the number of passed/failed test cases, and errors for failed tests (including screenshots). However, due to limitations, Selenium cannot generate comprehensive reports. This is why the generation and maintenance of reports have also become common challenges in automation using Selenium.
Solutions for Limited Reporting
There are available solutions for generating reports that can interact with Selenium and provide opportunities to obtain detailed results of test execution. Generally, we can categorize reporting tools into two groups:
- Built-in reporting is provided by the appropriate test automation framework (based on the programming language). For example, in Java, we can utilize frameworks like JUnit or TestNG, which include default features for generating HTML format reports. Examples of such reports are displayed below:
TestNG:
JUnit:
- External tools (libraries) are not included by default in the test framework and require additional setup and integration with the automation project. Some examples of such tools include Allure and Extent Reports.
Allure:
Extent Report:
Of course, each reporting tool has its own specifications, advantages, and limitations. Therefore, you need to analyze all the pros and cons of the existing tools and select the most suitable and effective one for your project.
5. Limited Captcha Handling
In the era of rapid progress in digital technologies, data security continues to be one of the most critical tasks in software development. Hackers often use bots to gain access to users’ confidential information, resulting in the loss of essential data and its unauthorized use. One measure that can aid in combating bots is Captcha.
This technology is employed to determine whether the user is a human or a bot. Captchas are dynamic components by nature, implying that they can only be successfully carried out through human analysis. Captcha handling is also one of the challenges in Selenium, which can result in test failures during the automation process.
Solutions for Captcha handling
Although handling Captchas in Selenium is indeed a tricky task and not considered best practice, there are a few ways to manage Captchas:
- Disabling Captcha in the test environment
- Adding waits in the test script and manually proceed with Captcha handling
However, it’s important to remember that not all test scenarios can be automated. Some scenarios are better suited for manual verification. Tests that involve Captcha interactions fall into such scenarios.
Mobile and Desktop Apps Testing
Of course, Selenium is a widely used framework for automating web-based application testing. It supports interaction with the most popular browsers (Chrome, Edge, Firefox, Safari) on various desktop operating systems (Windows, Mac OS, Linux). However, when it comes to automating mobile or desktop application testing, Selenium cannot provide us with such capabilities.
Solutions for mobile and desktop apps testing
As an alternative solution for automating mobile apps, there is Appium. It is an open-source framework used to automate native, hybrid, and mobile web applications on both Android and iOS platforms.
For desktop apps, there are several tools available, including WinAppDriver, Sikulix, and Ranorex Studio. Each of these tools has its own limitations, so it’s important to conduct a thorough analysis of the tool before starting to use it.
Scalability
Scalability is considered one of the major challenges in automation testing. One of the key goals of web automation is to enable the execution of as many tests as possible on various browsers (with different versions) and operating systems. This approach helps increase test coverage within a minimal timeframe.
Selenium WebDriver itself provides the capability to execute tests sequentially. However, when dealing with a large number of tests, this approach can be time-consuming and may not be as efficient as desired.
Solutions for Scalability
Among the scalable approaches in automation testing, we can identify Selenium Grid. This tool enables you to execute tests on multiple combinations of browsers and operating systems. However, it’s important to note that Selenium Grid does not permit running tests on a specific browser and operating system simultaneously. As an alternative, you can utilize a cloud-based tool that offers the ability to run tests on multiple browsers and operating systems concurrently.
Summing Up
In the web automation process, we frequently encounter various challenges and limitations, given that technologies are constantly changing, developing, and evolving. Each tool and framework used for automation purposes comes with its own set of advantages and disadvantages. QA automation engineers need to be aware of these factors and consider them when beginning to utilize these tools.
Selenium is not an exception. It is indeed a powerful framework for web automation, but it also has limitations that need to be addressed. This article has described the major challenges in automation testing using Selenium. Despite these limitations, there are ways to resolve or manage them and maximize the benefits of Selenium. It’s important to remember that no tool is perfect! We must consider both the pros and cons of each tool.
Whether it’s manual or automated testing (both are important stages of the QA process), it should be conducted by a highly qualified team.
See how we helped Kazidomi
speed up feature delivery with test automation and become #1 startup in Belgium
