[
https://issues.apache.org/jira/browse/HDDS-17719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Yarovoy updated HDDS-17719:
----------------------------------
Labels: ozone-hbase-perf (was: )
> Avoid Token.encodeToUrlString() on every datanode read request
> --------------------------------------------------------------
>
> Key: HDDS-17719
> URL: https://issues.apache.org/jira/browse/HDDS-17719
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Client
> Reporter: Andrey Yarovoy
> Assignee: Andrey Yarovoy
> Priority: Major
> Labels: ozone-hbase-perf
>
> *Description*
> Every datanode command built by `ContainerProtocolCalls` re-encodes the block
> token:
> {code:java}
> if (token != null) {
> builder.setEncodedToken(token.encodeToUrlString());
> }
> {code}
> This applies to `getBlock`, `getBlockFromDatanode`, `readChunk`, `listBlock`,
> `getCommittedBlockLength`, `finalizeBlock`, `writeSmallFile`,
> `readSmallFile` and `buildReadBlockCommandProto` (9 sites), and to
> `BlockInputStream`'s short-circuit GetBlock.
> `Token.encodeToUrlString` serializes the token `Writable` into a
> `DataOutputBuffer` and Base64-URL-encodes it (commons-codec): an allocation,
> a copy
> and an encode per request. The token for a block does not change between
> requests until it is refreshed.
> The write path already avoids this: `BlockOutputStream` and
> `BlockDataStreamOutput` compute `tokenString` once in the constructor.
> *Evidence (HBase YCSB on EC, RegionServer async-profiler JFR, 6×60s per arm)*
> `Token.encodeToUrlString` costs **0.3% (stream read ON) and 0.5% (OFF)** of
> RS CPU, about 3% of the classic read path.
> `org/apache/commons/codec/binary/Base64.encode` appears among the RS self
> frames.
> *Proposed fix*
> 1. Add overloads of the read-side `ContainerProtocolCalls` methods
> (`readChunk`, `getBlock`, `buildReadBlockCommandProto`) that take the
> already-encoded token `String`. Keep the `Token` overloads delegating, so
> other callers are unaffected.
> 2. In `BlockInputStream`, `ChunkInputStream`/`ChunkReader` and
> `StreamBlockInputStream`, keep the encoded string alongside the `tokenRef` and
> recompute it only when the token instance changes. That happens on
> `refreshBlockInfo` / token-expiry retry, when the stream gets a new token
> from OM. A small holder pairing the token with its encoding (compared by
> identity) keeps this to one field.
> {code:java}
> private static final class EncodedToken {
> private final Token<OzoneBlockTokenIdentifier> token;
> private final String encoded;
> ...
> static EncodedToken of(Token<OzoneBlockTokenIdentifier> t) throws IOException
> {
> return t == null ? null : new EncodedToken(t, t.encodeToUrlString());
> }
> }
> {code}
> *Compatibility / side effects*
> None. The same bytes go on the wire; this only caches the string per token
> instance. No public API removed.
> *Tests*
> - Unit: across N reads of one block, `encodeToUrlString` is called once per
> token instance (Mockito spy). After a token refresh (simulated
> `BLOCK_TOKEN_VERIFICATION_FAILED` retry), the new token's encoding is sent.
> - Existing secure read tests: `TestSecureOzoneRpcClient`, block-token expiry
> tests.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]