Fixing flaky parallel tests
I'm currently working on a large Laravel project with about 1.900 tests. Unfortunately, running those tests in sequence takes about ten minutes, and nobody has time for that. Fortunately, Laravel comes with a parallel test runner (https://github.com/paratestphp/paratest), which makes use of all available cores on your machine and cuts the time down to about 2:30 minutes. After I've written about 1.850 tests, running the whole suite sometimes resulted in tests failing. When running those failing tests, they're passing. I was stunned, not knowing where to start to fix this annoyance. Consulting the PHPUnit docs (https://phpunit.readthedocs.io), I came across the processIsolation
attribute, which resides in the phpunit.xml
configuration file. The documentation reads
This attribute configures whether each test should be run in a separate PHP process for increased isolation.
<?xml version="1.0" encoding="UTF-8"?>
<phpunit
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
backupGlobals="false"
backupStaticAttributes="false"
bootstrap="vendor/autoload.php"
colors="true"
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
- processIsolation="false"
+ processIsolation="true"
This simple change seems to fix my problem; now, all tests pass all the time.
Update: Although setting processIsolation
to true
fixes tests flaking out, the run time is about ten minutes and therefore utterly useless. I found out that running ./vendor/bin/paratest
instead php artisan test --parallel
really fixes my problems. I'm currently on the brink of submitting an issue to the Laravel repository, but it isn't effortless to create a proof of concept to attach to the submission.