David Mollitor created SPARK-59004:
--------------------------------------
Summary: Use vectorized Arrays.compare for
ComparableByteArray.compareTo
Key: SPARK-59004
URL: https://issues.apache.org/jira/browse/SPARK-59004
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 4.1.0
Reporter: David Mollitor
{{ArrayWrappers.ComparableByteArray.compareTo}} (in {{{}common/kvstore{}}},
used to order {{byte[]}} keys in the KVStore that backs the Spark status/UI
store) implements a
hand-written scalar loop that compares bytes one at a time. This replaces that
loop
with {{{}java.util.Arrays.compare(byte[], byte[]){}}}.
{{Arrays.compare}} performs the identical comparison – signed, lexicographic,
comparing
the common prefix first and ordering the shorter array first on a tie – but it
is a
HotSpot intrinsic backed by a vectorized (SIMD) mismatch scan, so it is faster
while
preserving the exact ordering. It is also simpler than the hand-written loop.
*Semantics are unchanged.* The existing byte comparison is signed
({{{}array[i] - other.array[i]{}}} promotes each {{byte}} to a sign-extended
{{{}int{}}}), which
matches {{{}Arrays.compare{}}}'s {{Byte.compare}} semantics; the single-element
byte difference cannot overflow, so behavior is unchanged.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
