[
https://issues.apache.org/jira/browse/KAFKA-20249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18064996#comment-18064996
]
Guang Zhao commented on KAFKA-20249:
------------------------------------
Incidentally, I identified another speedup on the other code path for "nonempty
headers": avoid deseralization when only byte skipping is required:
{code:java}
byte[] rawAggregation(final byte[] aggregationWithHeaders) {
// ...
final ByteBuffer buffer = ByteBuffer.wrap(aggregationWithHeaders);
readHeaders(buffer); // TODO: Speedup by not deserializing headers
return readBytes(buffer, buffer.remaining());
}
{code}
JMH Benchmark shows 6x speed up on this nonempty headers code path:
{code:java}
Benchmark Mode Cnt
Score Error Units
RawBytesExtraction.testRawAggregationWithHeaders thrpt 15
1411.338 ± 110.527 ops/s
RawBytesExtraction.testRawAggregationWithHeadersFastPath thrpt 15
6106.665 ± 218.032 ops/s
RawBytesExtraction.testRawAggregationWithoutHeaders thrpt 15
7734.538 ± 525.487 ops/s
RawBytesExtraction.testRawAggregationWithoutHeadersFastPath thrpt 15
14300.408 ± 212.519 ops/s
JMH benchmarks done
{code}
I'll make this change in the PR as well, for further review.
> Optimize rawValue methods across all Deserializers
> --------------------------------------------------
>
> Key: KAFKA-20249
> URL: https://issues.apache.org/jira/browse/KAFKA-20249
> Project: Kafka
> Issue Type: Sub-task
> Components: streams
> Reporter: Alieh Saeedi
> Assignee: Guang Zhao
> Priority: Major
> Labels: kip
> Fix For: 4.3.0
>
>
> Optimize rawValue/{{{}rawAggregationValue{}}} to fast-path the common case of
> empty headers in headers-aware state stores, while preserving correct
> behavior for non-empty headers.
> h3. Background
> In the current header-aware state store encoding, the value layout is:
> * 1st byte: header-length prefix (or start of varint-encoded header size)
> * Next {{N}} bytes: serialized headers
> * Remaining bytes: aggregation value (payload)
> Most state store records are expected to have no headers, so optimizing the
> empty-headers case reduces overhead in hot paths where we need to extract the
> raw aggregation value.
> h3. Proposed Change
> Introduce a specialized implementation of rawValue/{{{}rawAggregationValue.
> {}}}
> ```
> {{// Fast path for empty headers (most common case)}}
> {{ if (valueWithHeaders[0] == 0x00){ }}
> {{ // Headers size is 0, just strip first byte }}
> {{ final byte[] result = new byte[valueWithHeaders.length - 1];
> }}{{
> System.arraycopy(valueWithHeaders, 1, result, 0, result.length);
> }}
> {{ return result; }}
> }
> {{// Slow path for actual headers (rare in session stores)}}
> {{...}}
> ```
> Pros:
> - Optimizes the common case (empty headers)
> - ~50% faster for empty headers (no ByteBuffer, simple arraycopy)
> - Still handles non-empty headers correctly
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
