shangxinli commented on code in PR #19021:
URL: https://github.com/apache/hudi/pull/19021#discussion_r4304760988
##########
hudi-io/src/main/java/org/apache/hudi/io/hfile/HFileDataBlock.java:
##########
@@ -97,39 +100,57 @@ int seekTo(HFileCursor cursor, Key key, long
blockStartOffsetInFile) {
// Safe cast: a block is under 2GB, so an offset within it fits an int.
int relativeOffset = (int) (cursor.getOffset() - blockStartOffsetInFile);
int lastRelativeOffset = relativeOffset;
+ // The key-value cached at the starting position, if any. It is only
consulted to re-cache it
+ // in the cursor when the lookup lands "in range" on the very first
comparison; entries scanned
+ // past are compared directly against the backing buffer below (no
per-entry KeyValue/Key
+ // allocation), so this is emptied after the first iteration and the
cursor falls back to a
+ // deferred read.
Option<KeyValue> lastKeyValue = cursor.getKeyValue();
+ // The lookup key content is fixed across the scan; hoist it out of the
loop. Note the lookup
+ // key may be a UTF8StringKey, so use the polymorphic content accessors
(no 2-byte prefix).
+ byte[] lookupBytes = key.getBytes();
+ int lookupContentOffset = key.getContentOffset();
+ int lookupContentLength = key.getContentLength();
while (relativeOffset < uncompressedContentEndRelativeOffset) {
- // Full length is not known yet until parsing
- KeyValue kv = readKeyValue(relativeOffset);
- int comp = kv.getKey().compareTo(key);
+ // Compare the entry key against the lookup key directly on the buffer,
without materializing
+ // a KeyValue/Key for every scanned entry. Layout at `relativeOffset`:
[int keyLength]
+ // [int valueLength][short keyContentLength][key content]...; see
KeyValue and Key.
+ int keyContentLength = IOUtils.readShort(byteBuff, relativeOffset +
KEY_OFFSET);
Review Comment:
nit: this hardcodes the layout Key.getContentOffset and getContentLength
already own, so the two can drift silently - worth a static helper on Key
taking the buffer and offset.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
