[ 
https://issues.apache.org/jira/browse/SPARK-61993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated SPARK-61993:
-----------------------------------
    Labels: pull-request-available  (was: )

> Resolve GAP FILL into a Generate with a GapFillGenerator
> --------------------------------------------------------
>
>                 Key: SPARK-61993
>                 URL: https://issues.apache.org/jira/browse/SPARK-61993
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>    Affects Versions: 4.4.0
>            Reporter: Dimitrije Zdravkovic
>            Priority: Major
>              Labels: pull-request-available
>
> Resolve the \{{GAP FILL}} relation operator in the analyzer, as the second 
> step of SPARK-61276.
> * The analyzer rule \{{ResolveGapFill}} now turns \{{UnresolvedGapFill}} into 
> a \{{Generate}} with a new \{{GapFillGenerator}}, and a \{{Project}} above it 
> that keeps the input column order. Only the partition columns pass through 
> the \{{Generate}}. Every other column becomes a new nullable column that 
> keeps the name, type, metadata and table name of its input column.
> * Add \{{GapFillResolution}}, which checks the clauses in a fixed order and 
> evaluates their constant values. The single-pass analyzer can use it too.
> * Add the analysis errors: 15 new error classes, such as 
> \{{GAP_FILL_TIME_TYPE_MISMATCH}} and \{{GAP_FILL_INVALID_RANGE}}, and 4 new 
> subclasses of \{{UNSUPPORTED_FEATURE}}: \{{GAP_FILL_FILL_METHOD_TYPE}}, 
> \{{GAP_FILL_PARTITION_KEY_TYPE}}, \{{GAP_FILL_TIMESTAMP_DAY_BIN_WIDTH}} and 
> \{{GAP_FILL_TIMESTAMP_NANOS}}.
> * \{{CheckAnalysis}} rejects a correlated input, a streaming input, and a 
> metadata column read after \{{GAP FILL}}.
> * \{{OptimizeOneRowRelationSubquery}} does not move a \{{GAP FILL}} onto the 
> rows of an outer query.
> There is no physical execution yet. Until a later sub-task adds it, the 
> planner rejects a resolved \{{GAP FILL}} with 
> \{{UNSUPPORTED_FEATURE.GAP_FILL}}. The conf 
> \{{spark.sql.gapFillRelationOperator.enabled}} stays internal and off by 
> default.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to