iemejia commented on code in PR #55932:
URL: https://github.com/apache/spark/pull/55932#discussion_r3912742233
##########
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetDeltaLengthByteArrayEncodingSuite.scala:
##########
@@ -92,6 +95,61 @@ class ParquetDeltaLengthByteArrayEncodingSuite
}
}
+ test("skipBinary fails cleanly when the data region is truncated") {
+ // The length header is intact but the data region is short, so the
lengths decoded from
+ // the page declare more bytes than the stream actually holds. skipBinary
must surface a
+ // ParquetDecodingException instead of spinning forever on an exhausted
stream.
+ writeData(writer, values)
+ val fullBytes = writer.getBytes.toByteArray
+ val truncated = java.util.Arrays.copyOf(fullBytes, fullBytes.length - 3)
Review Comment:
Done — added `Arrays` to the existing `java.util` import and dropped the
fully-qualified `java.util.Arrays.copyOf`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
