Niels Pardon created CALCITE-7651:
-------------------------------------
Summary: Unify endianness semantics for bitwise shift (<<, >>) on
BINARY/VARBINARY
Key: CALCITE-7651
URL: https://issues.apache.org/jira/browse/CALCITE-7651
Project: Calcite
Issue Type: Task
Reporter: Niels Pardon
CALCITE-7109 added bitwise left shift ({{<<}} / {{LEFTSHIFT}}) over
{{BINARY}}/{{VARBINARY}}, and CALCITE-7639 adds the right-shift counterparts.
The binary implementation in {{SqlFunctions.leftShift(byte[], long)}} treats
the byte array as a *little-endian* bit string (index 0 = least-significant
byte) and normalizes the shift amount modulo {{8 × length}}.
This is inconsistent with CALCITE-7368, which made {{CAST(<integer> AS
BINARY)}} *big-endian*. As a result {{CAST(x AS BINARY(N)) << k}} != {{CAST((x
<< k) AS BINARY(N))}}, and e.g. {{RIGHTSHIFT(X'1234', 4)}} yields {{4103}}
(little-endian) rather than the big-endian {{0123}}.
The "PostgreSQL behavior" cited in the current code comments is inaccurate:
PostgreSQL does not define {{<<}}/{{>>}} on {{bytea}} — only on bit strings
({{bit}}/{{bit varying}}), where the semantics are big-endian,
length-preserving, zero-filled, and do *not* modulo-wrap the shift amount.
MySQL's {{<<}}/{{>>}} operate on the integer value of their operands, not
element-wise on bytes.
*Proposal:* decide a single documented semantics for binary shifts and apply it
to both {{<<}} and {{>>}}. Big-endian (matching CALCITE-7368 and PostgreSQL bit
strings) is the recommended choice, ideally so that {{CAST(x AS BINARY(N)) << k
= CAST((x << k) AS BINARY(N))}} holds for in-range shift amounts. Because
binary {{<<}} shipped in 1.41.0/1.42.0, changing its endianness is a behavioral
change that needs a compatibility note.
*Scope note:* binary {{>>}}/{{RIGHTSHIFT}} was descoped from CALCITE-7639
pending this decision — that PR ships only the integer/unsigned right shift and
rejects binary operands at validation. Binary left shift remains as released.
Related: CALCITE-7368, CALCITE-7109, CALCITE-7639, CALCITE-5087.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
