ClickHouse

Challenge

ClickHouse struggled to balance between increasing automated test coverage and conducting manual testing for each build. To maintain their rapid release cycle without compromising quality, they turned to QAwerk. Our testing expertise and flexible resources allowed us to alleviate their internal team’s workload and enhance overall quality assurance efforts.

Here are the main peculiarities of this project:

• Weekly Release Cadence. ClickHouse aimed to achieve a weekly release cycle to quickly deliver new features and improvements to their users. A fast-paced release schedule, however, can increase the risk of introducing bugs.
• Stability and Performance. Our testing approach needed to be comprehensive enough to ensure ClickHouse can maintain a high level of stability and performance.
• High-Stakes Product. As a cloud infrastructure provider, ClickHouse is critical to their customers’ operations. Bugs or errors can lead to reputational damage, financial losses, and potential legal liabilities. Our QA services aim to mitigate these risks and ensure the product’s reliability.

Solution

To tackle the complexities of ClickHouse’s database management system, QAwerk implemented a custom testing strategy. Our team of seasoned testers put their skills to work, making sure the ClickHouse platform met the highest standards of quality.

Our QA services comprise:

• Exploratory Testing: This is a non-scripted type of testing that allowed us to freely roam ClickHouse Cloud and get to know the ins and outs of the platform. Using our experience with cloud-based tools, we mimicked how real users might interact with ClickHouse to identify potential usability or performance issues. Exploratory testing is also valuable at later stages of software testing as it can uncover new test scenarios and increase the product’s overall test coverage.
• Functional Testing: We conducted thorough functional tests to ensure ClickHouse’s features and functions worked as designed. These tests covered key scenarios like account creation, login, database management, query execution, and data visualization. By systematically validating each feature, we verified the platform’s compliance with requirements and ability to deliver the intended functionality.
• Regression Testing: This testing is instrumental in maintaining the platform’s stability and reliability. To guarantee that new features or changes did not introduce unintended side effects, aka regressions, we re-execute a subset of existing tests. This ensures that the platform’s core functionality remains intact.
• Regular Documentation Updates: We maintain up-to-date test documentation to ensure that our testing efforts remain aligned with the project’s evolving requirements. This includes creating new test cases, updating existing ones, and documenting any changes to the testing process.
• UX Improvements: Beyond identifying technical problems, our team actively contributes to enhancing the user experience, pinpointing things that could make it hard for users to navigate the platform. We suggest ways to make the design better, the process easier, and the platform more user-friendly overall.

We tested the web app on Windows (Chrome) and Mac (Safari) to ensure it worked seamlessly across different platforms. Our efficient and comprehensive testing approach enabled ClickHouse to maintain a rapid release cycle without compromising quality.