Andrey Yarovoy created HDDS-16985:
-------------------------------------
Summary: OM gRPC server / dist do not use Netty native epoll
Key: HDDS-16985
URL: https://issues.apache.org/jira/browse/HDDS-16985
Project: Apache Ozone
Issue Type: Improvement
Components: OM
Reporter: Andrey Yarovoy
## Problem
The OM gRPC server (used by S3 Gateway through `GrpcOmTransport`) always uses
Netty's NIO transport, even on Linux where native epoll is available. The
shipped tarball also cannot load unshaded native epoll at all, so the S3
Gateway's OM gRPC client falls back to NIO without any warning.
## Current state
1. **`GrpcOzoneManagerServer` hard-codes NIO:**
- boss and worker groups are `new NioEventLoopGroup(...)`
-`NettyServerBuilder.channelType(NioServerSocketChannel.class)`
`XceiverServerGrpc` on the datanode already switches on `Epoll.isAvailable()`
and picks `EpollEventLoopGroup` / `EpollServerSocketChannel` when it can. The
OM server has no such switch.
2. **The dist is missing the native library for unshaded Netty.**
`share/ozone/lib` contains only `netty-transport-native-epoll-<ver>.jar` with
no classifier, and that jar has no `.so` in it. The `linux-x86_64` and
`linux-aarch_64` classifier jars are not packaged. As a result,
`io.netty.channel.epoll.Epoll.isAvailable()` returns `false` at runtime, which
affects:
-`GrpcOzoneManagerServer`: it would stay on NIO even after fix 1 unless this is
also fixed.
-`GrpcOmTransport` (S3 Gateway → OM client): it does not set a transport
explicitly, so gRPC chooses epoll only when it is available. Today it always
ends up on NIO.
The shaded gRPC/Netty from ratis-thirdparty is not affected, because
`ratis-thirdparty-misc` bundles the shaded epoll `.so`. So the datanode, SCM HA
and Ratis gRPC paths already get epoll on Linux.
3. **Minor:** the datanode `StreamingServer` / `StreamingClient`
(`container/stream`) hard-code `NioEventLoopGroup(100)` and
`Nio[Server]SocketChannel`. Today they are only called from tests and freon
`StreamingGenerator`.
## Proposed change
- In `GrpcOzoneManagerServer`, select `EpollEventLoopGroup` +
`EpollServerSocketChannel` when `Epoll.isAvailable()` is true and keep NIO as
the fallback, mirroring `XceiverServerGrpc`. Log the selected channel type at
startup.
- Package `io.netty:netty-transport-native-epoll` with the `linux-x86_64` and
`linux-aarch_64` classifiers in the dist, and update `jar-report.txt` and
`LICENSE.txt`.
- Optional: apply the same switch to `StreamingServer` / `StreamingClient`.
No protocol, config or compatibility impact. Platforms without epoll keep using
NIO.
## Testing
- Existing OM gRPC / S3 Gateway unit and integration tests, which use the NIO
fallback on macOS.
- Linux compose cluster: confirm the OM startup log shows the epoll channel
type, and that jstack shows epoll event-loop threads in OM and S3 Gateway.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
