[
https://issues.apache.org/jira/browse/SPARK-59004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-59004:
-----------------------------------
Labels: pull-request-available (was: )
> Use vectorized Arrays.compare for ComparableByteArray.compareTo
> ---------------------------------------------------------------
>
> Key: SPARK-59004
> URL: https://issues.apache.org/jira/browse/SPARK-59004
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 4.1.0
> Reporter: David Mollitor
> Priority: Minor
> Labels: pull-request-available
>
> {{ArrayWrappers.ComparableByteArray.compareTo}} (in {{{}common/kvstore{}}},
> used to order {{byte[]}} keys in the KVStore that backs the Spark status/UI
> store) implements a
> hand-written scalar loop that compares bytes one at a time. This replaces
> that loop
> with {{{}java.util.Arrays.compare(byte[], byte[]){}}}.
> {{Arrays.compare}} performs the identical comparison – signed, lexicographic,
> comparing
> the common prefix first and ordering the shorter array first on a tie – but
> it is a
> HotSpot intrinsic backed by a vectorized (SIMD) mismatch scan, so it is
> faster while
> preserving the exact ordering. It is also simpler than the hand-written loop.
> *Semantics are unchanged.* The existing byte comparison is signed
> ({{{}array[i] - other.array[i]{}}} promotes each {{byte}} to a sign-extended
> {{{}int{}}}), which
> matches {{{}Arrays.compare{}}}'s {{Byte.compare}} semantics; the
> single-element byte difference cannot overflow, so behavior is unchanged.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
