bryancall opened a new pull request, #12867:
URL: https://github.com/apache/trafficserver/pull/12867
## Description
Adds a parallel test runner (`autest-parallel.py`) that distributes autests
across multiple worker processes for significantly faster execution. On a
16-core machine, test suite completion drops from ~80 minutes (sequential) to
~6 minutes.
The existing `cmake --build -t autest` workflow is completely untouched.
This is an additive tool for developers who want faster local test runs.
## Changes
### New files
- `tests/autest-parallel.py` - Parallel test runner with:
- Port offset isolation per worker (AUTEST_PORT_OFFSET) to prevent port
conflicts
- Timing-based load balancing (LPT algorithm) using historical test
durations
- Serial test support for tests that cannot run concurrently
- Live progress line with ETA, failure counts, and worker status
- Verbose mode (-v) with real-time test output streaming
- Per-test timing collection (--collect-timings) for load balancing
optimization
- `tests/serial_tests.txt` - List of tests requiring serial execution
- `tests/README.md` - Added parallel testing documentation
### Modified files
- `tests/gold_tests/autest-site/ports.py` - Added AUTEST_PORT_OFFSET
environment variable support for dynamic port range isolation (backward
compatible, defaults to 0)
- `tests/gold_tests/basic/config.test.py` - Switched from hardcoded ports to
dynamic port selection
- `tests/gold_tests/basic/copy_config.test.py` - Same, removed
select_ports=False
## Usage
```bash
cd tests
python3 autest-parallel.py -j 16 \
--ats-bin <install>/bin \
--build-root <build-dir> \
--sandbox /tmp/autest-parallel
```
## Testing
- Tested with 16 parallel workers on a Ryzen 9 16-core machine
- 401 tests: 370 passed, 39 skipped (missing optional deps), 6 pre-existing
failures
- All pre-existing failures also fail when run sequentially on master
- The ports.py change is backward compatible (offset defaults to 0, no
effect on existing test runs)
- config.test.py and copy_config.test.py changes use dynamic ports which are
more robust than hardcoded values
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
