[
https://issues.apache.org/jira/browse/SPARK-59951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-59951:
-----------------------------------
Labels: pull-request-available (was: )
> Avoid allocation and boxing in UTF8String ASCII case conversion
> ---------------------------------------------------------------
>
> Key: SPARK-59951
> URL: https://issues.apache.org/jira/browse/SPARK-59951
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.2.0
> Reporter: David Mollitor
> Priority: Minor
> Labels: pull-request-available
>
> h3. What changes were proposed in this pull request?
> {{UTF8String.toLowerCase()}} / {{toUpperCase()}} dispatch to the ASCII fast
> paths {{toLowerCaseAscii()}} / {{toUpperCaseAscii()}} once {{isFullAscii()}}
> holds. This PR rewrites those fast paths:
> * Convert bytes with direct ASCII arithmetic ({{A}}\-{{Z}} <\-> {{a}}\-{{z}}
> by +/- 32) instead of routing each byte through an autoboxed
> {{Function<Character, Character>}} and
> {{Character.toLowerCase}}/{{toUpperCase}} (a full Unicode-table lookup). Over
> ASCII code points the two are identical.
> * Allocate the result array lazily, only when a byte actually changes case,
> bulk-copying the unchanged prefix with {{Platform.copyMemory}}. A string
> already in the target case returns {{this}} with no allocation.
> * Tag the converted result as full ASCII ({{IsFullAscii.FULL_ASCII}}), since
> ASCII case folding stays ASCII, so downstream operations skip re-scanning for
> ASCII-ness.
> * Remove the now-unused {{convertAscii}} helper and its
> {{java.util.function.Function}} import, and add an {{assert isFullAscii()}}
> precondition to both methods (callers already gate on {{isFullAscii()}}).
> Also adds {{UTF8StringBenchmark}} exercising {{toLowerCase}}/{{toUpperCase}}
> over all-lowercase, all-uppercase, and mixed-case ASCII inputs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
