zeroshade opened a new pull request, #1555: URL: https://github.com/apache/arrow-go/pull/1555
### Rationale for this change The ALP encoder rounds a scaled value to an integer by adding and subtracting a constant, 2^23 + 2^22 for `FLOAT` and 2^52 + 2^51 for `DOUBLE`. That is only correct below 2^22 and 2^51. Past those, the sum has no bit left for the units, and what comes back is a different integer. The integer then fails the round trip, and the value is stored as an exception. A float32 holds every whole number up to 2^24, so much of what a `FLOAT` column can hold exactly became exceptions. A column of whole numbers below 2^24 came out larger than `PLAIN`. Bits per value for a page of 131,072 values: | Data | Before | After | |---|---|---| | `FLOAT`, whole numbers below 2^24 | 41.3 | 24.1 | | `FLOAT`, 0 to 1,000,000 with one decimal | 41.9 | 33.8 | | `FLOAT`, whole numbers below 2^22 | 22.1 | 22.1 | | `FLOAT`, 0 to 10,000 with two decimals | 32.3 | 32.3 | | `DOUBLE`, whole numbers below 2^53 | 74.4 | 53.1 | The format leaves the rounding to the encoder, since every integer is checked against the decoding it specifies. Pages written before this change read as before, and so do the pages written after it. ### What changes are included in this PR? `alpScaling.round` uses `math.RoundToEven`. Rounding can no longer carry a value past the limits `encode` checks, which are whole numbers. The check that a `FLOAT` integer fits int32 could therefore never fail, and is removed with the fields it used. On an Apple M4 the existing encoding benchmarks take the same time as before, within noise. I have no amd64 timings. Without `GOAMD64=v2` or later, `math.RoundToEven` checks for SSE4.1 at run time before it uses the instruction. The scaling loops in #1553 reproduce the rounding as it is on `main`. Whichever of the two is merged second needs the same one-line change there. ### Are these changes tested? Yes. ### Are there any user-facing changes? `FLOAT` and `DOUBLE` columns with values past 2^22 and 2^51 once scaled compress better. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
