To remain at the forefront of innovation, VirtaMed not only partners with reputable hospitals, medical societies, and medical device companies, but heavily invests in quality assurance.
VirtaMed turned to QAwerk to strengthen their on-site QA team with independent software testers. They looked for someone proficient in game testing because their medical training software is built on Unity, and the sessions themselves, to some extent, have game-like mechanics.
VirtaMed leverages Unity’s 3D digital twin technology and integrated haptic feedback to provide hyperrealistic surgical training.
VirtaMed also needed someone flexible enough to cope with the following challenges:
- Remote Testing. Our task was to test the simulation software remotely, using keyboard combinations and controllers instead of the physical instruments. We needed to get the hang of manipulating instruments to perform a smooth testing of the virtual simulator.
- Medical Terms. We were also expected to master a decent number of medical terms since this is crucial for understanding the training itself, reporting bugs, and day-to-day communication.
- Product Novelty. We’ve tested numerous gamified e-learning solutions, but nothing like VirtaMed’s surgical simulators. Therefore, the onboarding period required more effort in terms of diving into the industry specifics, understanding our client’s quality goals, and optimizing our testing workflows.
The project itself is quite large, so we also needed to devise a feasible testing strategy to have a clear picture of cases that passed QA, cases causing most bugs, and bug fixes in need of retesting, among other aspects.
Our QA team was engaged in testing software for LaparoS™ General Surgery and LaparoS™ Gynecology simulators. To achieve comprehensive, in-depth testing of the simulation software, we conducted the following:
- Functional Testing. We check if the simulator behaves as intended at every step of the procedure under test.
- Non-Functional Testing. Here we test haptics and instrument sensitivity. It’s done simultaneously with going through testing scenarios.
- UI Testing. Our QA engineers make sure the right information is displayed before and after performing simulations. We pay attention to warnings and errors popping up during the VR training.
- Regression Testing. We conduct full-fledged testing of the product before every release to see if the new build is stable and can be rolled out safely.
- Smoke Testing. If there are changes to a decoupled functionality or a minor bug fix, we quickly go through the main features to ensure nothing else has been affected by this change.
- Negative Testing. Occasionally, we deliberately deviate from happy paths and run negative test cases to detect hidden bugs, reduce the probability of crashes caused by improper user input, and increase the reliability and stability of the software.
To bring order into our testing process, we created test plans and test execution issues. The latter allowed us to instantly see what aspects have been checked and at what point issues usually arise.
VirtaMed also invited us to visit their facilities and exchange knowledge in person. We traveled to Zurich and tested their software on-site, this time using the actual VR simulators. The latter was beneficial for understanding the difference between testing simulators on-site and remotely.
VirtaMed’s simulators help improve medical education worldwide, so we also hope to contribute to localization testing in the near future.
According to our observations, complex and intricate cases are most likely to contain issues. Also, things like realistic surgical physics interactions between organs and tools are quite difficult to implement software-wise, so they’re prone to bugs too.
Actual result: The cystic duct falls down into the body. It’s impossible to pass the task.
Expected result: The collision is between the vessel and anatomy.
Actual result: The gallbladder penetrates the anatomy.
Expected result: The collision is between the gallbladder and anatomy.
Actual result: The grasper is a bit "sticky" to the adhesions.
Expected result: The adhesion does not stick to the tools.
With our continuous QA support, VirtaMed can consistently release software updates while maintaining the quality bar high.
Together with QAwerk, VirtaMed managed to significantly increase test coverage. As an independent, third-party advisor, we explore their software from a different angle, which helps improve existing products and develop new solutions for online training.
VirtaMed’s ongoing investment in quality assurance has only strengthened their reputation as a world leader in data-driven education. VirtaMed, along with big names like Microsoft, has recently become a consortium member of the PROFICIENCY project that is meant to further advance simulator-based training in surgery and received CHF 12 mln funding from the Swiss Innovation Agency.
With this type of limitless virtual accessibility, VirtaMed hopes to help accelerate the time a learner needs to gain competency across the educational and training scenarios.
VirtaMed and Memic today announced a partnership to develop a new virtual reality simulator program to support surgeon skills training for the Hominis® Surgical System
We have a specialty/niche product that needs a significant time for onboarding normally. QAwerk was willing and able to do the onboarding and work on the rather complex testing setup. There were some initial hiccups with the technical setup, largely owing to our specialty configuration, but the testing team was patient and willing to work with us on tackling the issues. The flexibility alongside the good technical work has become a staple of this product's testing.
Thanks to QAwerk, the client can efficiently cover the app's complete regression testing, which is a time-consuming task. The team manages the project well using Jira and Confluence and has regular sync-ups. Moreover, they make valuable suggestions to improve the product's testing.
Looking for QA engineers experienced in VR testing?Let’s talk
QAwerk Team Comments
This project truly challenged me because the medical field is quite complex. It took me some time to memorize all the needed medical terms. Also, I learned the basics of performing some surgical procedures on a real simulator while visiting the VirtaMed team in Switzerland. Every test case, be it General Surgery or Gynecology, is a type of surgical operation, so it requires meticulous and in-depth testing. Grateful to VirtaMed teammates for being so communicative and organized; it makes our cooperation fruitful and easy.
I'm thrilled to work on the VirtaMed project cause, besides using and improving my testing skills, I get to learn a thing or two about surgery. The testing itself is performed in line with the step-by-step test execution document. To a certain degree, it resembles testing a space flight simulator: all actions are in 3D, and you first need to master the skills of controlling a virtual operator.
Related in Blog
If you are a developer or a tester, the word agile is nothing new to you. But if you are new in it and wonder what agile testing is and what it means in practice, you won’t have any questions left after reading this article. What is Agile Testing? Agile testing is an inse...Read More
Other Case Studies
Achieved app stability and speeded up software releases by 20% with overnight testing and automation
Helped increase regression testing speed by 50% and ensure the platform runs 24/7
Helped Norway #1 study portal improve 8 content-heavy websites used by 110 mln students annually