Bug Crawl Digest #4: What Mobile App QA Testing Misses

Mobile app QA testing usually asks whether a feature works. What it misses is everything around that: a lost signal, an interruption, a customer who has already used their free trial. Those conditions produced nearly every bug we found across 14 products last month.

Such situations are perfectly ordinary. Most apps run into all of them within days of launch. Even so, a test plan usually covers these cases last, and plenty of teams never get that far. Finding those bugs is what our mobile application testing engineers do every week.

Here are the four failures worth your attention, and the product that earned our pick for bug of the month.

Apps covered in this digest:

The other seven products we crawled in August are in the full bug crawl archive.

Ads That Lock You Out When the Signal Goes

Apps: Isle Survival: Land Builder, Extreme Makeover: Home Edition (both iOS)
Severity: Critical
Type: Advertising and connectivity

Both games save a video advert onto the phone in advance so it plays smoothly later. That part worked, but everything wrapped around it assumed the internet would still be there.

Our tester put Isle Survival into Airplane Mode, tapped Watch, and sat through the whole video. The store screen meant to follow couldn’t load without a connection, and no Close button appeared either. There was no way back into the game. A player who watched an advert in good faith ends up trapped behind it.

Extreme Makeover: Home Edition broke the same way with a different ending. Its saved advert also played offline, then returning to the game produced a black screen where nothing responded.

Players read both as the game failing rather than the network, and that difference costs money. Somebody who blames the product uninstalls it, while a dropped signal usually gets forgiven.

What to check on your side: Test every advert that can be saved in advance with the connection switched off, and keep testing past the video itself. The way out must never depend on a screen that can only appear when the internet is working. Where a reward genuinely can’t be granted offline, hide the offer rather than letting a player earn something you can’t hand over.

How this gets caught: Exploratory testing on real hardware, with somebody deliberately cutting the connection at awkward moments. Automated suites run on a healthy network and never meet this. Our game testing work treats adverts as their own area, not a feature that happens to sit inside the game.

Bug Crawl Digest #4: What Mobile App QA Testing Misses

Apps That Forget Where You Left Off

Apps: SUN AI: Audiobook & Podcasts (Android), Grounds: Fitness App for Women (iOS)
Severity: Critical to Major
Type: Interruptions and resuming

Every phone puts apps to sleep. A call arrives, somebody checks a message, and your product sits quietly in the background for half a minute. What it does when the person comes back is a feature, whether or not anyone designed it as one.

SUN AI handled the interruption worst. Our tester switched to something else, returned a moment later, and found a blank white screen that responded to nothing at all. The only fix was closing it completely and starting over.

Grounds failed more gently. Our tester started a workout, opened the details page for one exercise, switched away briefly, and came back to the overview list instead.

Neither product realized it had been interrupted, and that’s the common thread. It also explains why this slips through so often: a written test never gets a phone call, so the app is never asked to survive one.

What to check on your side: Switch away from every screen that holds something in progress, not just the home screen. Wait a few minutes with other apps open, long enough for the phone to start clearing yours from memory. Then check three things on return: the screen is where you left it, any number that was counting is right, and any audio or video is playing or paused exactly as the person chose.

How this gets caught: Regression testing that builds the interruption into existing test cases instead of treating it as a separate exercise. These failures gather on screens holding something in progress, so that’s where the effort belongs.

When the Timer and the App Disagree

Apps: Grounds: Fitness App for Women, January: AI Health Tracker (both iOS)
Severity: Critical to Major
Type: Timing and progress tracking

A number ticking upward promises that something is being measured. Two products broke that promise in August, and neither gave anyone a way to notice.

January: AI Health Tracker produced the more unsettling version. Our tester started a voice recording, an alarm went off, and they carried on talking straight through it. The counter kept running the whole time, which said the recording was fine, but not one word spoken during that alarm reached the finished transcript. The app was reporting work it had quietly stopped doing.

Grounds had the heavier problem. Halfway through a session, our tester rearranged the order of the workout. The tick showing which exercise was already finished stayed behind at its old spot instead of moving along with it. So a move nobody had performed appeared complete, while the one genuinely done looked untouched.

That app also kept its workout clock running while our tester sat rearranging the list, so time spent organizing got recorded as time spent training.

For products whose entire value is an honest record of what you did, this is the worst possible outcome. A wrong number does more damage than no number, because people act on it.

What to check on your side: Interrupt anything that tracks time or progress. Open another screen, switch to a different app, set off an alarm, take a call. Then compare what gets reported against what genuinely happened, and make sure a timer pauses the moment the activity does. If the phone can quietly take the microphone away, your app has to notice and say so.

How this gets caught: Functional testing built around interruptions rather than clean runs. It takes somebody willing to deliberately ruin their own attempt, which is rarely how a test plan gets written.

Free Trials the Product Can't Keep Track Of

Apps: OLY: Personal Fitness Coach (iOS)
Severity: Major
Type: Subscriptions and billing

This is the only bug we found in August that actually took money from a customer.

OLY showed our tester a paywall with a button reading Start my 7-Day Free Trial, and they tapped it. However, they had already used their introductory offer, so no trial was left to give them. Confirming the purchase charged them for a full year instead. The screen promised one thing and the payment delivered another. Our previous digest found the same gap in in-app purchase testing, where the money went through and nothing arrived.

That’s precisely the pattern American regulators are watching. The FTC has kept pursuing subscription cases under its existing powers while it reworks the Negative Option Rule, and one complaint centers on a free-trial button that led to a charge. A bug here looks exactly like the practice being prosecuted. Neither the customer who paid nor the regulator who hears about it can tell an accident from a deliberate trick.

The bug itself is simple enough: OLY’s paywall was built for a first-time visitor and never asked who it was talking to.

What to check on your side: Test every subscription screen as a returning customer, not only as a new one. Check what your paywall offers somebody who has already had a trial, and confirm the charge that follows matches the offer on screen. Treat the promise and the payment as two separate facts to compare.

How this gets caught: Testing the whole life of an account, using the stores’ practice payment systems and accounts at different stages of their history. Our App Store compliance testing covers the platform rules these screens answer to alongside the commercial ones.

Mobile App QA Testing: What to Add to Your Checklist

Every item below comes from a bug we found in August. Pass the list to whoever tests your releases.

Every saved advert: play it with the connection off, then confirm there’s still a way back into the app.
Every screen with something in progress: switch away for several minutes, then confirm the screen, the numbers and the audio all come back correctly.
Every counter: interrupt it with a call, an alarm or another screen, then compare what it reports against what happened.
Every paywall: view it as somebody who has already used a trial, and check the charge matches the offer.
Every loading screen: launch it offline and confirm it explains itself instead of waiting forever.

Bug of the Month

Our pick is Weglot, a translation platform. Businesses connect it to their site so visitors can read the content in their own language. Delivering that translated page is the product’s core job, and both bugs we found meant the app fails at it.

The first showed up on a test website our tester built with the Weglot plugin connected. The page design broke as soon as they switched from the original language to another. This level of failure was deemed critical as it defeated the app’s main promise by damaging the site’s structure.

The second is about hand-written corrections. Weglot translates a site automatically, and the owner can manually fix anything the software got wrong. Our tester did exactly that, then told the platform to stop using machine output so only the hand-written version would show. None of those corrections appeared on the website.

Honorable mention goes to Isle Survival. A player sits through an advert, cannot get back into the game, and has to close it. Plenty will just delete it.

Nobody wants to hear about bugs like these from a customer. Tell us about your app and we’ll find them first.

FAQ

What is mobile app QA testing?

Mobile app QA testing means checking that software behaves properly on real devices before customers use it. It covers whether features function, and also how the product copes when it gets interrupted, loses signal, or meets somebody who has used it before. Those unplanned conditions cause the more serious problems, because they rarely appear in a scripted test run.

Why do so many app bugs only show up offline?

Because most testing runs on a good connection, so nobody ever watches what an app does without one. Products get built to succeed, and they usually handle a request that fails outright, but few teams decide what happens in between. A screen waits forever, a button does nothing, or a saved advert plays while the rest of the app cannot load.

Can automated tests catch offline and interruption bugs?

They catch some, not most. Automation is good at confirming a feature still works and at spotting differences between builds. It struggles with interrupted sessions, missing connections, and anything a person has to judge by eye. Those need somebody holding a real phone, which is why exploratory testing stays part of any serious plan.