How to Fix Flaky Tests

What Makes a Test Flaky?

A flaky test is one that sometimes passes and sometimes fails without any change to the code it is testing. Flaky tests erode confidence in your test suite because developers learn to ignore test failures, assuming they are caused by flakiness rather than real bugs. This defeats the purpose of automated testing.

Flakiness typically stems from non-deterministic behavior in the test itself or its environment. Timing-dependent assertions, shared mutable state between tests, reliance on external services, and race conditions in asynchronous code are the most common culprits behind intermittent test failures.

Common Causes of Flaky Tests

Timing issues are the leading cause of flaky tests, especially in E2E and integration tests. Tests that rely on hard-coded sleep durations or race against asynchronous operations will occasionally fail when the system is slower than expected. Auto-waiting mechanisms and explicit condition checks are the solution.

Shared state between tests is another frequent cause. When tests share a database, file system, or in-memory state, the order in which tests run affects their results. A test that passes in isolation may fail when run after another test that modifies the shared state in an unexpected way.

Strategies for Identifying Flaky Tests

The most effective way to identify flaky tests is to run your test suite multiple times in succession and flag any tests that produce inconsistent results. Many CI platforms support automatic retries and can report tests that passed on retry, making it easy to spot intermittent failures.

Quarantining known flaky tests into a separate suite prevents them from blocking your CI pipeline while you investigate. Track flaky tests in your issue tracker with the same priority as bugs, because a flaky test that is never fixed is worse than having no test at all.

Fixing Timing and Async Issues

Replace hard-coded sleep statements with explicit waits that check for a specific condition before proceeding. Modern testing frameworks provide built-in mechanisms for this: Playwright's auto-waiting, Cypress's retry-ability, and sarvaTest's smart selectors all handle timing automatically.

For asynchronous operations in unit tests, use async and await or the framework's built-in async support rather than callbacks or manual promise handling. Ensure your test waits for the operation to complete before making assertions, and set appropriate timeouts for operations that may take longer under load.

Eliminating Shared State and Test Order Dependencies

Each test should set up its own test data and clean up after itself. Use before and after hooks to reset the database, clear caches, and restore environment variables to a known state. This ensures tests are independent and produce the same result regardless of execution order.

If setting up fresh data for every test is too slow, use transactions that are rolled back after each test. This approach is common for database-dependent tests and provides isolation without the overhead of full database recreation. Most testing frameworks and ORMs support this pattern natively.

Preventing Flakiness in New Tests

Code review is your first line of defense against new flaky tests. Reviewers should look for hard-coded waits, shared state, date or time dependencies, and reliance on external services. Establishing team conventions for test isolation and timing reduces the chance of flakiness being introduced.

Run new tests in a loop as part of your review process, executing them ten or twenty times to check for intermittent failures before merging. This practice catches most timing-related flakiness early. Some CI platforms support this automatically through a quarantine or burn-in mechanism for new tests.