ChitChat

Challenge

ChitChat turned to QAwerk when they were in the early stages of development, crafting the core architecture and user stories. They hired us to establish and lead a comprehensive QA process from the ground up. This included both manual and automated testing to ensure a seamless and secure user experience.

The innovative nature of the ChitChat app presented unique testing challenges:

• In-depth payment testing: The in-chat payment functionality required rigorous testing. This demanded access to a diverse set of test bank cards and multiple test phone numbers to simulate various payment scenarios.
• Market specifics and device fragmentation: Unlike typical mobile app users in Europe or America, the target audience in Zambia primarily uses different phone models. We addressed this by studying usage data and assembling a comprehensive test phone set that mirrored the most popular devices in Zambia. The list of devices with different configurations was extensive.
• Enhanced security standards: Since the app deals with financial transactions, we were expected to exceed standard quality expectations. We needed to implement exceptionally deep and comprehensive test strategies to guarantee the app’s security and reliability when handling users’ money.

Let’s dive deeper into how we navigated these challenges.

Solution

We actively participated in the development process, analyzing each user story as it became available and identifying potential gaps needing clarification. Based on the user stories, we created robust test scenarios that covered every possible user interaction.

We conducted thorough manual testing and built automated testing frameworks for both the backend and frontend aspects of the app. Our test plan included the following testing types:

• Functional Testing. We executed a comprehensive suite of tests designed to verify the app’s performance against the functional requirements outlined in the user stories. This ensured that all features were implemented correctly and functioned as intended, meeting the defined user needs.
• Integration Testing. Since the app interacts with various third-party services like Primer and Onfido, seamless integration was crucial. We performed integration testing to ensure smooth data flow and functionality between the app and these external services.
• Compatibility Testing. We tested the app on a wide range of device models, operating systems, and screen resolutions. In total, ChitChat was tested on 24 iOS and Android devices.
• Automated Testing. We focused on automating both frontend and backend tests, leveraging the power of TypeScript for enhanced code clarity and type checking.

Also, we continuously monitor user feedback, translating it into clear bug reports with detailed steps to reproduce the bugs and sharing them with developers.

Test Automation

Our success with automated testing hinged on the choice of efficient tools.

Mocha, a JavaScript testing framework, provided the structure for writing our tests. For frontend testing, we adopted Appium, a popular framework for automating interactions with mobile apps. We utilized WebdriverIO, a powerful tool that acts as an Appium client, to streamline the process. This allowed us to write automated tests covering various aspects of the app, from user login and chat functionalities to core banking features.

BrowserStack, a cloud-based platform, offered a critical advantage – the ability to run our automated tests on a vast array of real devices with different configurations.

The automated tests for the backend, namely the app’s APIs, were written in Jest, a testing framework that utilizes TypeScript. TypeScript’s strength lies in its ability to check variable and function types, enhancing code clarity and preventing errors during development.

However, the tools alone, no matter how fast and reliable they are, can’t guarantee that everything will go smoothly. One major hurdle we faced was ensuring test stability across a wide variety of mobile devices. This was overcome by leveraging BrowserStack’s extensive collection of real devices, allowing us to test on actual hardware with various configurations.

Another challenge emerged due to the app’s numerous integrations with third-party applications. These integrations added complexity, especially for automated testing. Tests for these functionalities were often written using webhooks (a communication method between apps), but their instability required frequent code adjustments.

Finally, parallel execution of tests, while offering efficiency gains, occasionally led to instability issues. In some cases, tests might pass twice but fail on the third run. To address this, we strategically divided the tests into three groups based on functionality. Tests from two groups were run simultaneously for maximum speed, while the group containing the unstable tests ran sequentially. This approach effectively resolved the instability and ensured consistent test results.

The implemented testing strategy achieved a ratio of approximately 70% automated tests to 30% manual tests. This emphasis on automation yielded significant benefits. Automated tests sped up test execution, enabling us to cover a broader range of functionalities more efficiently. This led to faster bug identification and resolution, ultimately contributing to a more secure and stable application.