LuciferYang commented on code in PR #55932:
URL: https://github.com/apache/spark/pull/55932#discussion_r3868432762
##########
sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedDeltaLengthByteArrayReader.java:
##########
@@ -113,11 +119,12 @@ public ByteBuffer getBytes(int rowId) {
@Override
public void skipBinary(int total) {
+ long totalSkip = 0;
Review Comment:
A negative-length page is constructible without the writer getting in the
way: the layout is `concat(lengthWriter.getBytes(), data)`, so write the length
header yourself with `DeltaBinaryPackingValuesWriterForInteger`, which
`ParquetDeltaEncodingSuite` in this package already instantiates, and
`writeInteger(-6)` goes in unmodified. The `length < 0` branch is testable, and
nothing executes it today.
Worth covering because the property that matters is not that it throws but
that it throws before any skip, which means the check has to stay per value
inside the accumulation loop. Move it after `skipFully`, or turn it into
`totalSkip < 0`, and `[100, -100]` sums to 0, the stream stays put while
`currentRow` advances, and CI is still green.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
