Stefan Savic created SPARK-60214:
------------------------------------
Summary: Rejecting non-folable lgNomEntries in analysis phase-
#252798
Key: SPARK-60214
URL: https://issues.apache.org/jira/browse/SPARK-60214
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 4.4.0
Reporter: Stefan Savic
Previously, these HLL and Theta configuration operands were not rejected during
analysis. Aggregate functions could fail when evaluating their configuration
during
execution. Scalar {{theta_union}} could evaluate a row-varying {{lgNomEntries}}
on the
JVM; this PR now rejects that usage during analysis.
This change requires these arguments to be foldable and validates
them during analysis, matching the documentation, for:
* {{theta_union}} and its {{lgNomEntries}}
* {{theta_sketch_agg}} and its {{lgNomEntries}}
* {{theta_union_agg}} and its {{lgNomEntries}}
* {{hll_sketch_agg}} and its {{lgConfigK }}
* {{hll_union_agg}} and its {{allowDifferentLgConfigK}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
