[
https://issues.apache.org/jira/browse/SPARK-54138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Tenedorio reassigned SPARK-54138:
----------------------------------------
Assignee: Christopher Boumalhab
> Enforce constant configuration parameter for Hll and Theta Sketches
> Aggregation
> -------------------------------------------------------------------------------
>
> Key: SPARK-54138
> URL: https://issues.apache.org/jira/browse/SPARK-54138
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.1.0
> Reporter: Christopher Boumalhab
> Assignee: Christopher Boumalhab
> Priority: Minor
> Labels: pull-request-available
>
> *Description:*
> Add validation to ensure that the sketch configuration parameter (e.g.,
> {{{}lgConfigK/lgNomEntries{}}}) in both {{HllSketchAgg}} and
> {{ThetaSketchAgg}} is constant. If the configuration expression
> ({{{}right{}}}) is not foldable, throw a
> {{QueryExecutionErrors.*MustBeConstantError(prettyName)}} to prevent runtime
> issues caused by dynamic or non-deterministic configurations.
> *Rationale:*
> The configuration parameter defines key precision and memory characteristics
> of HLL and Theta sketches. It must be known at analysis time to guarantee
> deterministic aggregation behavior and consistent accuracy. Allowing
> non-constant parameters could lead to unpredictable results and incorrect
> aggregations.
> *Acceptance Criteria:*
> * Queries fail with a clear error if the configuration parameter
> ({{{}right{}}}) is not foldable.
> * Constant parameters are accepted and validated with existing checks (e.g.,
> {{{}checkLgK{}}}).
> * Existing valid queries with constant configuration values continue to work
> without changes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
