David Mollitor created SPARK-59390:
--------------------------------------
Summary: Build padding in UTF8String.lpad/rpad with a shared
repeat-fill helper
Key: SPARK-59390
URL: https://issues.apache.org/jira/browse/SPARK-59390
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 4.1.0
Reporter: David Mollitor
h2. Summary
{{lpad}} and {{rpad}} build the padding by copying the pad string {{count}}
times in a loop -- one {{copyMemory}} per repetition, i.e. O(count) copy calls.
{{repeat()}} already solves the same "repeat these bytes N times" problem more
efficiently, with a single-byte {{Arrays.fill}} fast path and exponential
doubling. This factors that logic into a shared private helper
({{fillRepeated}}) used by {{repeat()}}, {{lpad}} and {{rpad}}.
h2. Details
* Single-byte pattern -- the common padding case (space, {{0}}, {{?}}) -- fills
the region with a single {{Arrays.fill}}.
* Multi-byte pattern -- seed one copy, then exponential doubling: O(log count)
copies instead of O(count).
* The total number of bytes copied is unchanged; only the number of copy calls
drops. {{repeat()}} now delegates to the shared helper, and its previous inline
doubling and single-byte branch are removed, leaving a single implementation.
* The benefit shows up when padding a short value to a large width with a short
pad (large {{count}}); ordinary small-width padding is unaffected. This is a
modest, targeted change.
h2. Does this PR introduce _any_ user-facing change?
No. Behavior is identical; only the way the padding bytes are written into the
result buffer changes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
