baibaichen opened a new issue, #11559:
URL: https://github.com/apache/incubator-gluten/issues/11559
## Problem description
When running `mvn test-compile` without any code changes, the build still
takes a long time (~150s) because:
1. **`maven-antrun-plugin:build-info`** runs in `generate-resources` phase,
regenerating build info (including git revision) on every build, which triggers
unnecessary recompilation of dependent modules.
2. **`protobuf-maven-plugin`** (version 0.5.1) lacks incremental compilation
support, regenerating proto files even when sources are unchanged.
### Benchmarks
**Build Command:**
```bash
./build/mvn -P
java-17,spark-4.0,scala-2.13,backends-velox,hadoop-3.3,spark-ut \
-Piceberg,iceberg-test,delta,paimon test-compile
```
| Metric | Before | After | Improvement |
|--------|--------|-------|-------------|
| Total Build Time | 150.5s | 69.7s | **-53.7%** |
| Gluten Substrait | 52.09s | 8.67s | -83.4% |
| Gluten Core | 24.62s | 6.83s | -72.3% |
| Gluten Backends Velox | 35.13s | 16.93s | -51.8% |
### Proposed Changes
1. Move `build-info` and `build-info-with-backends` execution phase from
`generate-resources` to `prepare-package` (so it only runs during
`package`/`install`/`deploy`, not during `test-compile`)
2. Upgrade `protobuf-maven-plugin` from 0.5.1 to 0.6.1 and enable
`<checkStaleness>true</checkStaleness>` for incremental compilation
### Impact
- **Developer Experience**: Faster local iteration during development
- **CI/CD**: No impact - release builds use `mvn install` which still
triggers build-info generation
- **Tested with all profiles**: java-17, spark-4.0, scala-2.13,
backends-velox, hadoop-3.3, spark-ut, iceberg, delta, paimon
This issue was written with the assistance of GitHub Copilot.
## System information
```
Gluten Version: 1.6.0-SNAPSHOT
Commit: d793744abd2d04d1e9561bcae70263f84cefc86e
CMake Version: 3.30.1
System: Linux-6.6.87.2-microsoft-standard-WSL2
Arch: x86_64
CPU Name: AMD EPYC 7763 64-Core Processor
```
## CMake log
N/A - This is a Maven build optimization, not a CMake issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
