arouel opened a new pull request, #3915: URL: https://github.com/apache/parquet-java/pull/3915
### Rationale for this change `BinaryStatistics.updateStats(Binary)` compares each value with the current minimum and the current maximum. `Binary.copy()` keeps the same instance when the backing bytes are not reused. Thus, the minimum or the maximum can be the same instance as a value that the writer sends again. For example, values that come from a dictionary repeat the same constant `Binary` instance. For these values, the comparisons always return "equal" and do not change the statistics. Each comparison reads the bytes of the two values, so the writer does work that it does not need. ### What changes are included in this PR? - In `BinaryStatistics.updateStats(Binary)`, return early when the value is the same instance as `min` or `max`. In this case, the comparator is not called. - Use an early return for the first value. This removes the nested `else` block. The subclasses `Float16Statistics` and `IEEE754Float16Statistics` override `updateStats(Binary)`. Thus, this change does not apply to them. ### Are these changes tested? Yes. The new tests in `TestStatistics` make sure that the statistics do not change: - `testBinaryMinMaxForRepeatedSingleConstantInstance`: The same constant instance repeats, and it is both the minimum and the maximum. - `testBinaryMinMaxKeepsBoundInstancesForConstantValues`: The bound instances repeat. A different instance with equal content does not replace a bound. Lower and higher values change the bounds. The existing test `testBinaryMinMaxForReusedBackingByteArray` continues to make sure that a reused byte array that changes between updates gives correct statistics. These tests also pass without the change, because the skipped comparisons do not change the result. The tests make sure that the behavior stays the same. ### Are there any user-facing changes? No. The statistics are the same as before. There are no API changes. Closes #3913 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
