Fixing flaky parallel tests
I'm currently working on a large Laravel project with about 1.900 tests. Unfortunately, running those tests in sequence takes about ten minutes, and nobody has time for that. Fortunately, Laravel comes with a parallel test runner (https://github.com/paratestphp/paratest), which makes use of all available cores on your machine and cuts the time down to about 2:30 minutes. After I've written about 1.850 tests, running the whole suite sometimes resulted in tests failing. When running those failing tests, they're passing. I was stunned, not knowing where to start to fix this annoyance. Consulting the PHPUnit docs (https://phpunit.readthedocs.io), I came across the
processIsolation attribute, which resides in the
phpunit.xml configuration file. The documentation reads
This attribute configures whether each test should be run in a separate PHP process for increased isolation.
<?xml version="1.0" encoding="UTF-8"?> <phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" backupGlobals="false" backupStaticAttributes="false" bootstrap="vendor/autoload.php" colors="true" convertErrorsToExceptions="true" convertNoticesToExceptions="true" convertWarningsToExceptions="true" - processIsolation="false" + processIsolation="true"
This simple change seems to fix my problem; now, all tests pass all the time.
Update: Although setting
true fixes tests flaking out, the run time is about ten minutes and therefore utterly useless. I found out that running
php artisan test --parallel really fixes my problems. I'm currently on the brink of submitting an issue to the Laravel repository, but it isn't effortless to create a proof of concept to attach to the submission.