Jungtaek Lim created SPARK-50077:
------------------------------------
Summary: Introduce a new pattern object for LogicalRelation to
help avoiding default pattern (full parameters)
Key: SPARK-50077
URL: https://issues.apache.org/jira/browse/SPARK-50077
Project: Spark
Issue Type: Task
Components: SQL
Affects Versions: 4.0.0
Reporter: Jungtaek Lim
During my work (work-in-progress), I found it very difficult to add a param
into LogicalRelation, because a bunch of places use pattern matching with
LogicalRelation, and all of them are using default pattern with specifying all
params (despite wildcards).
I looked into the patterns and realized most of the patterns just need relation
and catalog table inside the LogicalRelation. output and isStreaming are used
infrequently. That said, introducing a new pattern object extracting
BaseRelation and CatalogTable from LogicalRelation would cover most of the
patterns we have for LogicalRelation, and this would help a lot to introduce a
new param into LogicalRelation in future.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
