yew1eb opened a new pull request, #3787:
URL: https://github.com/apache/celeborn/pull/3787
### What changes were proposed in this pull request?
Exclude `netty-codec-http3`, `netty-codec-classes-quic`, and
`netty-codec-native-quic` from the `netty-all` dependency in the root pom's
dependencyManagement.
### Why are the changes needed?
Since the Netty `4.1.118` → `4.2.10.Final` upgrade, `netty-all` pulls in the
QUIC/HTTP3 stack that 4.2 merged into the mainline:
```
netty-all
└─ netty-codec-http3 (compile)
└─ netty-codec-classes-quic (compile)
└─ netty-codec-native-quic (runtime, 5 classifiers)
```
`netty-codec-native-quic` ships a `libnetty_quiche42_*` native library per
platform classifier (`linux-x86_64`, `linux-aarch_64`, `osx-x86_64`,
`osx-aarch_64`, `windows-x86_64`) — **~29MB uncompressed / ~15MB compressed**
in total, plus 156 QUIC classes + 112 HTTP3 classes.
Because every `*-shaded` module uses `maven-shade-plugin` with
`<include>io.netty:*</include>`, the wildcard drags all of the above into every
shaded client jar. As a result `celeborn-client-spark-3-shaded` grew from
**15MB** (0.6.3, Netty 4.1.118) to **27MB** (1.0.0-SNAPSHOT, Netty 4.2.10). The
same bloat applies to all 11 shaded modules (spark-3/4, flink 1.18~2.3, mr,
tez) and to the runtime classpath of common/client/master/service/worker.
Celeborn only uses Netty TCP RPC; a full-repo grep confirms **zero**
references to `codec.quic` / `codec.http3` / `quiche` in source. These are pure
dead weight.
Excluding them at the `netty-all` declaration (inherited by all modules via
dependencyManagement) drops the shaded jars back to ~15MB with no functional
impact.
### Does this PR introduce _any_ user-facing change?
No. Only build-time artifact size: shaded client jars drop back to ~15MB.
### How was this patch tested?
- Built `celeborn-client-spark-3-shaded` (spark-3.5 profile) and
`celeborn-client-flink-1.20-shaded` (flink-1.20 profile). Each jar shrank from
27MB to 15MB, with zero `*quiche*` native libs and zero `handler/codec/quic/**`
/ `handler/codec/http3/**` classes.
- Verified epoll/kqueue native libs (renamed by antrun) remain present.
- `mvn dependency:tree` on `common`: quic dependency nodes dropped from 7 →
0 after the exclusion, confirming it takes effect at the dependency resolution
layer for all inheriting modules.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
