nastra commented on PR #14930: URL: https://github.com/apache/iceberg/pull/14930#issuecomment-3749203612
@stuxuhai thanks for submitting the PR. We might need to revisit the behavior of views in the `SparkSessionCatalog` (there was also https://github.com/apache/iceberg/issues/14557 that ran into issues with Legacy Hive views) and define the behavior we actually want to have. Basically right now the `SparkSessionCatalog` only supports Iceberg views since 1.8.0, hence why we have checks like ``` public boolean viewExists(Identifier ident) { return (asViewCatalog != null && asViewCatalog.viewExists(ident)) || (isViewCatalog() && getSessionCatalog().viewExists(ident)); } ``` where we either have the actual Iceberg catalog or the underlying Spark session catalog implementing Spark's `ViewCatalog` API. If that's not the case, we don't fall back Spark's session catalog and check `getSessionCatalog().tableExists(ident)`, which would also detect a v1 view. I understand that using a `IF NOT EXISTS` on a v1 Hive view you'd expect to not fail, but what are the implications of that when you e.g. run describe or show views? I haven't tested that, so we might want to explore that all of the operations you can do against a View don't produce weird results when we apply this diff. I'm still currently undecided on what the right approach here would be, and I've seen that e.g. creating a [Table](https://github.com/apache/spark/blob/dc46b1479450e8e656bce703d831e3aada95ea5c/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/CreateTableExec.scala#L43-L59) behaves slightly different than creating a [View](https://github.com/apache/iceberg/blob/3149892438ec7d4114fc2d79e970188dd3c90a44/spark/v4.1/spark-extensions/src/main/scala/org/apache/spark/sql/execution/datasources/v2/CreateV2ViewExec.scala#L80-L85). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
