LuciferYang commented on PR #14297: URL: https://github.com/apache/gravitino/pull/14297#issuecomment-6574064069
Thanks for the review. 1. Yes, I looked at that first, but on the Spark versions the connector builds against it can't be done from outside the analyzer. View resolution only runs when `ResolveRelations` resolves an `UnresolvedRelation` to a view ([3.5.3](https://github.com/apache/spark/blob/v3.5.3/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala#L1153), [4.0.3](https://github.com/apache/spark/blob/v4.0.3/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala#L1179)), so a `View` node we build ourselves is skipped and its child still resolves against the session. The `AnalysisContext` that carries the view's catalog and namespace can't be set from outside either: `execute` starts a new context, and the method that keeps the current one, `executeSameContext`, is private ([3.5.3](https://github.com/apache/spark/blob/v3.5.3/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala#L234-L240), [4.0.3](https://github.com/apac he/spark/blob/v4.0.3/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala#L334-L340)). So this PR only rewrites table names, and functions, `IDENTIFIER(...)` and `view.sqlConfig.*` behave as before, as the description says. Spark 4.2 adds the path you describe. When `loadTable` finds nothing, the analyzer calls [`ViewCatalog.loadView`](https://github.com/apache/spark/blob/v4.2.0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/RelationResolution.scala#L275-L288), turns the returned view's current catalog, namespace and SQL configs into the [`view.catalogAndNamespace.*` and `view.sqlConfig.*` properties](https://github.com/apache/spark/blob/v4.2.0/sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/V1Table.scala#L167-L183), and resolves it as a [native view](https://github.com/apache/spark/blob/v4.2.0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala#L1134-L1138). Once the connector has a Spark 4.2 module, the Hive catalog can implement it there, which would also cover functions and SQL configs and drop the driver-side `executeCollect`. That fits the Spark connector `ViewCatalog` work under #604, so I'd keep it out of this PR. 2. Done in 0fd691d88: the Javadoc of `relationQualifier` now says that mapping `spark_catalog` to this catalog assumes the Spark session that created the view used the same Hive metastore. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
