Anton Okolnychyi created SPARK-53924:
----------------------------------------
Summary: Reload DSv2 tables in views created using plans
Key: SPARK-53924
URL: https://issues.apache.org/jira/browse/SPARK-53924
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.1.0
Reporter: Anton Okolnychyi
The current problem is that the view definition in the session catalog captures
the analyzed plan that references DataSourceV2Relation and Table. If a
connector doesn’t have an internal cache and produces a new Table object after
TableCatalog$load, Table referenced in the view will become orphan and there
will be no way to refresh it unless that Table instance auto refreshes on each
scan (super dangerous).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
