nastra opened a new pull request, #3626: URL: https://github.com/apache/parquet-java/pull/3626
<!-- Thanks for opening a pull request! If you're new to Parquet-Java, information on how to contribute can be found here: https://parquet.apache.org/docs/contribution-guidelines/contributing Please open a GitHub issue for this pull request: https://github.com/apache/parquet-java/issues/new/choose and format pull request title as below: GH-${GITHUB_ISSUE_ID}: ${SUMMARY} or simply use the title below if it is a minor issue: MINOR: ${SUMMARY} --> ### Rationale for this change When a Variant object is built with the same key appended more than once, `VariantBuilder.endObject()` deduplicates the entries and keeps the last-written value. However, the total `dataSize` of the object's value region was computed from each key's **first** occurrence rather than the value that was actually retained. When duplicate keys had values of different encoded sizes, this size mismatch corrupted the output: * If the retained value was larger (e.g. `{"a": 1, "a": "hello"}`), the buffer was under-reserved, the offset width could be undersized, and the final offset entry understated the real data length. Reading the field then threw `IllegalArgumentException: Invalid byte-array offset` or returned truncated data. * If the retained value was smaller, the object reserved stale trailing bytes and overstated its size. Closes https://github.com/apache/parquet-java/issues/3625 ### What changes are included in this PR? ### Are these changes tested? yes, added tests that reproduce the underlying issue ### Are there any user-facing changes? <!-- Please uncomment the line below and replace ${GITHUB_ISSUE_ID} with the actual Github issue id. --> <!-- Closes #${GITHUB_ISSUE_ID} --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
