[
https://issues.apache.org/jira/browse/SPARK-59556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-59556:
-----------------------------------
Labels: pull-request-available (was: )
> Compile a lookup-shaped CASE WHEN on a string key to a hash probe
> -----------------------------------------------------------------
>
> Key: SPARK-59556
> URL: https://issues.apache.org/jira/browse/SPARK-59556
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.1.0
> Reporter: David Mollitor
> Priority: Minor
> Labels: pull-request-available
>
> h2. Problem
> A {{CASE WHEN}} whose branches are all {{key = literal THEN constant}} on a
> single key is really a lookup table -- e.g. mapping status/country/category
> codes to labels, often generated by BI tools or dbt with tens to hundreds of
> arms. Today {{CaseWhen}} whole-stage codegen emits an if/else-if chain
> ({{CaseWhen.multiBranchesCodegen}}). For a string key that is up to N
> {{UTF8String}} comparisons per row, growing linearly with the branch count.
> {code:sql}
> CASE state_code WHEN 'AL' THEN 'Alabama' WHEN 'AK' THEN 'Alaska' ... ELSE
> 'Unknown' END
> {code}
> h2. Change
> A codegen-only recognizer in {{CaseWhen.doGenCode}}: when the CASE is a lookup
> on a binary-collation string key, it builds a constant key -> value table once
> on the driver and emits a boxing-free, constant-size inline hash probe (a
> power-of-two open-addressed {{int[]}} bucket table plus parallel key/value
> {{ArrayData}}) instead of the O(N) chain. The generated code is O(1) in size
> and O(1) per row. Non-lookup CASEs fall back to the existing chain unchanged.
> The interpreted {{eval}} path and the {{CaseWhen}} node itself are unchanged,
> so there is no plan-shape / golden-file impact -- purely a code-generation
> substitution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
