LuciferYang opened a new issue, #9776: URL: https://github.com/apache/paimon/issues/9776
### Bug description `HilbertIndexer.hilbertCurvePosBytes` pads the N-dimensional 63-bit hilbert index to a fixed 63 bytes. 63 bytes only holds 8 dimensions worth of index bits: with 9 or more order columns (accepted by the hilbert sorter — Flink/core only validate `size() > 1` — and the Spark Hilbert UDF), `paddingToNByte` drops entire low-order bytes of the big-endian index, so distinct points collapse into the same sort key and the space-filling order degrades. For 16 dimensions nearly a full byte per point is lost. ### Expected behavior The byte width should cover the full index: `63*N/8 + 1` bytes for N > 8 (the +1 covers BigInteger's sign byte when the top bit is set). Up to 8 dimensions keep the legacy 63-byte width so existing keys stay byte-stable — and even there the low byte is lost for the half of the space whose index has the top bit set (e.g. any NULL order column, which maps to `Long.MAX_VALUE`); that quirk is preserved for compatibility since hilbert keys are transient sort keys in all current consumers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
