Uros Stankovic created SPARK-48925:
--------------------------------------
Summary: Introduce interface to ensure extra strategy do planning
of scan plan (with additional filters and projections)
Key: SPARK-48925
URL: https://issues.apache.org/jira/browse/SPARK-48925
Project: Spark
Issue Type: Task
Components: Spark Core
Affects Versions: 4.0.0
Reporter: Uros Stankovic
If we have some plan that contains scan with filter (or project) as parent, it
can happen we want to do planning of scan using extra strategies instead of
DataSourceV2Strategy.
One of use-cases:
Snowflake and BigQuery connectors have their own strategies and we want to
prohibit DataSourceV2Strategy to do planning of scan node.
Despite the fact extra strategies have priority, it can happen that strategy
fails to plan
Filter->Relation, but it can plan only Relation without its parent. In that
case, DataSourceV2Strategy will jump in and it will be able to plan
Filter->Relation in just one pass.
So we want to have ability to prevent that for certain Scan classes
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
