LuciferYang commented on PR #55932: URL: https://github.com/apache/spark/pull/55932#issuecomment-5509291422
The user-facing-change section only covers DELTA_LENGTH_BYTE_ARRAY, but this PR also changes two behaviors it does not mention: PLAIN skipBinary switches from in.skip (return value ignored) to skipFully, so on a truncated page it used to skip too few bytes and keep decoding the remaining values from the wrong position, and now throws ParquetDecodingException, failing the query by default or skipping the rest of the file with ignoreCorruptFiles=true; and negative lengths in PLAIN and DELTA_BYTE_ARRAY, which used to silently rewind or no-op in PLAIN and fail with a different exception in DELTA_BYTE_ARRAY, now throw ParquetDecodingException in both. The code changes themselves are fine; please also cover them in the description so users upgrading can connect the new failure to this change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
