[
https://issues.apache.org/jira/browse/HUDI-9325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sagar Sumit updated HUDI-9325:
------------------------------
Description:
When we do a seekTo for a key that does not exist, then the cursor gets set to
the first key corresponding to the block that holds the floor key. A subsequent
seekTo for a non-existent key that is lexicographically smaller than the next
key the cursor is pointing to throws follwing error:
{code:java}
Caused by: java.lang.IllegalStateException: The current lookup key is less than
the current position of the cursor, i.e., backward seekTo, which is not
supported and should be avoided.
key=UTF8StringKey{aZud4Oj9wxA=AZcHyzibzy8=bJfAFKIt2udY/eEKkDkFbg==}
cursor=HFilePosition{offset=10503431,
keyValue=Option{val=KeyValue{key=Key{aZud4Oj9wxA=AZcHyzibzy8=bLXWAR+d0ui0DMuh0uAXbw==}}}}
at
org.apache.hudi.io.hfile.HFileReaderImpl.seekTo(HFileReaderImpl.java:173)
at
org.apache.hudi.io.storage.HoodieNativeAvroHFileReader$RecordByKeyIterator.hasNext(HoodieNativeAvroHFileReader.java:404)
at
org.apache.hudi.common.util.collection.MappingIterator.hasNext(MappingIterator.java:39)
at java.util.Iterator.forEachRemaining(Iterator.java:115)
at
java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at
java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
at
org.apache.hudi.metadata.HoodieBackedTableMetadata.fetchBaseFileRecordsByKeys(HoodieBackedTableMetadata.java:438)
at
org.apache.hudi.metadata.HoodieBackedTableMetadata.readFromBaseAndMergeWithLogRecords(HoodieBackedTableMetadata.java:403)
at
org.apache.hudi.metadata.HoodieBackedTableMetadata.lookupKeysFromFileSlice(HoodieBackedTableMetadata.java:358)
at
org.apache.hudi.metadata.HoodieBackedTableMetadata.lambda$getRecordsByKeys$f9381e22$1(HoodieBackedTableMetadata.java:281)
at
org.apache.hudi.common.function.FunctionWrapper.lambda$throwingMapWrapper$0(FunctionWrapper.java:38)
... 45 more {code}
was:
When we do a seekTo for a key that does not exist, then the cursor gets set to
the first key corresponding to the block that holds the floor key. A subsequent
seekTo for a non-existent key throws follwing error:
{code:java}
Caused by: java.lang.IllegalStateException: The current lookup key is less than
the current position of the cursor, i.e., backward seekTo, which is not
supported and should be avoided.
key=UTF8StringKey{aZud4Oj9wxA=AZcHyzibzy8=bJfAFKIt2udY/eEKkDkFbg==}
cursor=HFilePosition{offset=10503431,
keyValue=Option{val=KeyValue{key=Key{aZud4Oj9wxA=AZcHyzibzy8=bLXWAR+d0ui0DMuh0uAXbw==}}}}
at
org.apache.hudi.io.hfile.HFileReaderImpl.seekTo(HFileReaderImpl.java:173)
at
org.apache.hudi.io.storage.HoodieNativeAvroHFileReader$RecordByKeyIterator.hasNext(HoodieNativeAvroHFileReader.java:404)
at
org.apache.hudi.common.util.collection.MappingIterator.hasNext(MappingIterator.java:39)
at java.util.Iterator.forEachRemaining(Iterator.java:115)
at
java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at
java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
at
org.apache.hudi.metadata.HoodieBackedTableMetadata.fetchBaseFileRecordsByKeys(HoodieBackedTableMetadata.java:438)
at
org.apache.hudi.metadata.HoodieBackedTableMetadata.readFromBaseAndMergeWithLogRecords(HoodieBackedTableMetadata.java:403)
at
org.apache.hudi.metadata.HoodieBackedTableMetadata.lookupKeysFromFileSlice(HoodieBackedTableMetadata.java:358)
at
org.apache.hudi.metadata.HoodieBackedTableMetadata.lambda$getRecordsByKeys$f9381e22$1(HoodieBackedTableMetadata.java:281)
at
org.apache.hudi.common.function.FunctionWrapper.lambda$throwingMapWrapper$0(FunctionWrapper.java:38)
... 45 more {code}
> Native HFileReader throws IllegalStateException for an edge case with back to
> back key misses
> ---------------------------------------------------------------------------------------------
>
> Key: HUDI-9325
> URL: https://issues.apache.org/jira/browse/HUDI-9325
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Sagar Sumit
> Priority: Major
>
> When we do a seekTo for a key that does not exist, then the cursor gets set
> to the first key corresponding to the block that holds the floor key. A
> subsequent seekTo for a non-existent key that is lexicographically smaller
> than the next key the cursor is pointing to throws follwing error:
> {code:java}
> Caused by: java.lang.IllegalStateException: The current lookup key is less
> than the current position of the cursor, i.e., backward seekTo, which is not
> supported and should be avoided.
> key=UTF8StringKey{aZud4Oj9wxA=AZcHyzibzy8=bJfAFKIt2udY/eEKkDkFbg==}
> cursor=HFilePosition{offset=10503431,
> keyValue=Option{val=KeyValue{key=Key{aZud4Oj9wxA=AZcHyzibzy8=bLXWAR+d0ui0DMuh0uAXbw==}}}}
> at
> org.apache.hudi.io.hfile.HFileReaderImpl.seekTo(HFileReaderImpl.java:173)
> at
> org.apache.hudi.io.storage.HoodieNativeAvroHFileReader$RecordByKeyIterator.hasNext(HoodieNativeAvroHFileReader.java:404)
> at
> org.apache.hudi.common.util.collection.MappingIterator.hasNext(MappingIterator.java:39)
> at java.util.Iterator.forEachRemaining(Iterator.java:115)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
> at
> org.apache.hudi.metadata.HoodieBackedTableMetadata.fetchBaseFileRecordsByKeys(HoodieBackedTableMetadata.java:438)
> at
> org.apache.hudi.metadata.HoodieBackedTableMetadata.readFromBaseAndMergeWithLogRecords(HoodieBackedTableMetadata.java:403)
> at
> org.apache.hudi.metadata.HoodieBackedTableMetadata.lookupKeysFromFileSlice(HoodieBackedTableMetadata.java:358)
> at
> org.apache.hudi.metadata.HoodieBackedTableMetadata.lambda$getRecordsByKeys$f9381e22$1(HoodieBackedTableMetadata.java:281)
> at
> org.apache.hudi.common.function.FunctionWrapper.lambda$throwingMapWrapper$0(FunctionWrapper.java:38)
> ... 45 more {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
