david-mollitor-db opened a new pull request, #58676:
URL: https://github.com/apache/spark/pull/58676
**This is a draft (WIP)** opened for early feedback; no JIRA is attached yet.
### What changes were proposed in this pull request?
`lpad`/`rpad` build the padding by copying `pad` `count` times in a loop —
one `copyMemory` per repetition, i.e. O(count) copy calls. `repeat()` already
solves the same "repeat these bytes N times" problem more efficiently
(exponential doubling, plus a single-byte `Arrays.fill` fast path).
This factors that logic into a shared private helper used by `repeat()`,
`lpad`, and `rpad`:
```java
private static int fillRepeated(byte[] data, int destPos, Object
patternBase, long patternOffset,
int patternNumBytes, int count) {
...
if (patternNumBytes == 1) { // common padding case: space,
'0', '?'
int end = destPos + count;
Arrays.fill(data, destPos, end, Platform.getByte(patternBase,
patternOffset));
return end;
}
// otherwise: seed one copy, then exponential doubling (O(log count)
copies)
...
}
```
- Single-byte pad → one `Arrays.fill`.
- Multi-byte pad → exponential doubling, O(log count) copies instead of
O(count).
- Total bytes copied is unchanged; only the number of copy calls drops.
`repeat()` now delegates to the same helper (its previous inline doubling and
single-byte branch are removed).
### Why are the changes needed?
Padding a short value to a large width with a short pad currently issues
O(count) copy calls, and the common single-character pad hits that loop rather
than a single `Arrays.fill`. Reusing `repeat()`'s strategy makes the
single-char case one fill and bounds the multi-byte case to O(log count)
copies, at no cost to normal small-width padding — and removes the duplicated
repeat logic.
### Does this PR introduce _any_ user-facing change?
No. Behavior is identical; this only changes how the padding bytes are
written into the result buffer.
### How was this patch tested?
Existing `UTF8StringSuite` passes (52/52), including `pad()` and `repeat()`.
Adds `padExponentialDoubling`, which exercises counts that trigger multiple
doubling steps and the `toCopy` clamp, plus multi-byte and single-byte pads,
and adds a single-byte `repeat` case.
### Was this patch authored or co-authored using generative AI tooling?
Yes. Generated-by: Claude Code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
