yandrey321 opened a new pull request, #11726:
URL: https://github.com/apache/ozone/pull/11726
## What changes were proposed in this pull request?
**Problem.** A cold Ozone client initializes Netty, and with gRPC TLS also
netty-tcnative (BoringSSL), on the main
thread at its first datanode connection. That's about 115 ms in a fresh JVM:
- about half is Netty itself: `PlatformDependent`, the native transport, the
event loop and the pooled buffer
allocator;
- the rest is netty-tcnative: loading the library, and the fixed checks in
`OpenSsl`'s static initializer.
None of it can be configured away. But nothing else happens while it runs,
although the client knows long before
that it will need it: its first datanode connection comes more than half a
second after it's created, after the
Kerberos login and the OM calls. Short-lived client JVMs pay this every
time: YARN containers, CLI commands.
**Implementation idea.** When an `XceiverClientCreator` (and so an
`XceiverClientManager`) is created, it starts a
daemon thread, at most once per JVM. The thread allocates and releases one
buffer from Netty's pooled allocator, and
with security and gRPC TLS enabled it calls `OpenSsl.isAvailable()`. The
connection path stays as it is: it finds
these initialized, or initializes them itself.
**Key decisions.**
- **Once per JVM, best effort.** A failure in the thread is logged at DEBUG
and ignored; the first connection then
does the work, as before.
- **TLS warm-up only when security and gRPC TLS are both on.** It doesn't
depend on `hdds.grpc.tls.provider`, because
`XceiverClientGrpc` builds its context with `GrpcSslContexts.forClient()`,
which uses OpenSSL whenever it's
available.
- **New setting `ozone.client.netty.warmup.enabled`,** default `true`, to
turn it off. The warm-up needs a spare CPU
core to help; in a container capped at one core it competes with the main
thread, and gains less.
**Side effects.**
- **One more short-lived daemon thread per JVM**
(`ozone-client-netty-warm-up`).
- **Earlier initialization:** Netty, and netty-tcnative with TLS, are
initialized when the client is created, even
in a process that never connects to a datanode.
Generated-by: Claude Code (Claude Opus 5.5)
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-17357
## How was this patch tested?
Unit and integration tests
**Measurement:** cold `ofs://` client on the secure `ozonesecure` compose
cluster (Kerberos, gRPC TLS; create the
client, getFileStatus, read 64 KB, close), JDK 17 (17.0.20.1), same build,
10 interleaved rounds per arm:
| | warm-up off | warm-up on (default) |
|---|---|---|
| total | 1,293.5 ms (1,272–1,322 ms) | **1,225.5 ms (1,199–1,244 ms), −68
ms** |
| first read | 436.5 ms | 365.5 ms (−71 ms) |
| configuration + client creation | 712 ms | 717 ms |
The ranges don't overlap. The warm-up thread doesn't slow the main thread
while it runs (the machine has spare
cores).
- **TestDFSIO on secure YARN** (YARN images with JDK 17) against the same
cluster: 3 NodeManagers with
`LinuxContainerExecutor`, 4 files, write and read, 6 interleaved rounds
per arm, 24 of 24 jobs successful. In the
"off" arm, `ozone.client.netty.warmup.enabled=false` was set on the job,
and in the NodeManagers' `ozone-site.xml`,
which the localizers read.
| JVM | warm-up on vs off |
|---|---|
| localizer: localization time (main band of 65 localizations at about
1.5–1.6 s) | −66 ms median, −79 ms mean (standard error about 0.013 s) |
| AppMaster start-up | −103 ms median, −91 ms mean |
| map start-up | −22 ms median, −42 ms mean |
Every cold client on a job's critical path gains: about 0.27 s per job.
The maps gain least, as a map reaches its
first datanode connection soon after creating its client, which leaves
less time to overlap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
