[
https://issues.apache.org/jira/browse/HDDS-16985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Yarovoy reassigned HDDS-16985:
-------------------------------------
Assignee: Andrey Yarovoy
> OM gRPC server / dist do not use Netty native epoll
> ---------------------------------------------------
>
> Key: HDDS-16985
> URL: https://issues.apache.org/jira/browse/HDDS-16985
> Project: Apache Ozone
> Issue Type: Improvement
> Components: OM
> Reporter: Andrey Yarovoy
> Assignee: Andrey Yarovoy
> Priority: Major
>
> ## Problem
> The OM gRPC server (used by S3 Gateway through `GrpcOmTransport`) always uses
> Netty's NIO transport, even on Linux where native epoll is available. The
> shipped tarball also cannot load unshaded native epoll at all, so the S3
> Gateway's OM gRPC client falls back to NIO without any warning.
> ## Current state
> 1. **`GrpcOzoneManagerServer` hard-codes NIO:**
> - boss and worker groups are `new NioEventLoopGroup(...)`
> -`NettyServerBuilder.channelType(NioServerSocketChannel.class)`
> `XceiverServerGrpc` on the datanode already switches on `Epoll.isAvailable()`
> and picks `EpollEventLoopGroup` / `EpollServerSocketChannel` when it can. The
> OM server has no such switch.
> 2. **The dist is missing the native library for unshaded Netty.**
> `share/ozone/lib` contains only `netty-transport-native-epoll-<ver>.jar` with
> no classifier, and that jar has no `.so` in it. The `linux-x86_64` and
> `linux-aarch_64` classifier jars are not packaged. As a result,
> `io.netty.channel.epoll.Epoll.isAvailable()` returns `false` at runtime,
> which affects:
> -`GrpcOzoneManagerServer`: it would stay on NIO even after fix 1 unless this
> is also fixed.
> -`GrpcOmTransport` (S3 Gateway → OM client): it does not set a transport
> explicitly, so gRPC chooses epoll only when it is available. Today it always
> ends up on NIO.
> The shaded gRPC/Netty from ratis-thirdparty is not affected, because
> `ratis-thirdparty-misc` bundles the shaded epoll `.so`. So the datanode, SCM
> HA and Ratis gRPC paths already get epoll on Linux.
> 3. **Minor:** the datanode `StreamingServer` / `StreamingClient`
> (`container/stream`) hard-code `NioEventLoopGroup(100)` and
> `Nio[Server]SocketChannel`. Today they are only called from tests and freon
> `StreamingGenerator`.
> ## Proposed change
> - In `GrpcOzoneManagerServer`, select `EpollEventLoopGroup` +
> `EpollServerSocketChannel` when `Epoll.isAvailable()` is true and keep NIO as
> the fallback, mirroring `XceiverServerGrpc`. Log the selected channel type at
> startup.
> - Package `io.netty:netty-transport-native-epoll` with the `linux-x86_64` and
> `linux-aarch_64` classifiers in the dist, and update `jar-report.txt` and
> `LICENSE.txt`.
> - Optional: apply the same switch to `StreamingServer` / `StreamingClient`.
> No protocol, config or compatibility impact. Platforms without epoll keep
> using NIO.
> ## Testing
> - Existing OM gRPC / S3 Gateway unit and integration tests, which use the NIO
> fallback on macOS.
> - Linux compose cluster: confirm the OM startup log shows the epoll channel
> type, and that jstack shows epoll event-loop threads in OM and S3 Gateway.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
