Max Gekk created SPARK-57847:
--------------------------------
Summary: Support the TIME data type in
approx_count_distinct_for_intervals
Key: SPARK-57847
URL: https://issues.apache.org/jira/browse/SPARK-57847
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.3.0
Reporter: Max Gekk
This sub-task is part of the umbrella SPARK-57550 (extend support for the TIME
data type).
h2. Problem
{{ApproxCountDistinctForIntervals}}
(aggregate/ApproxCountDistinctForIntervals.scala) accepts
numeric/date/timestamp/interval in {{inputTypes}} (~L68-70) and its endpoint
validation (~L92-93), but not {{TimeType}}. This is not covered by SPARK-57557,
which extended {{ApproximatePercentile}} and the sketch quantile aggregates.
h2. Goal
Support {{TimeType}} as the value column (and TIME endpoints) in
{{approx_count_distinct_for_intervals}}.
h2. Scope
Add {{TimeType}} (or {{AnyTimeType}}) to the value + endpoints {{inputTypes}};
handle the nanos-of-day {{Long}} value in the counting path.
h2. Acceptance criteria
* The aggregate runs over a TIME column with TIME endpoints; results are
consistent with the numeric behavior.
h2. Testing
{{ApproxCountDistinctForIntervalsSuite}}.
h2. Dependencies
None - independent.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
