szehon-ho commented on PR #57802: URL: https://github.com/apache/spark/pull/57802#issuecomment-5199012030
@cloud-fan could you take a look? If the v2 view path isn't your area, I'd appreciate a pointer to a better reviewer. Some context on why this matters outside Spark. I hit this while reviewing Apache Iceberg's Spark 4.2 support (apache/iceberg#14984). Iceberg implements `RelationCatalog` and is working out how much of its own view DDL handling it can hand back to Spark's native v2 view execs, and this bug is one of the reasons it can't delegate the ALTER path yet: a catalog that persists view lineage silently loses it on any metadata-only `ALTER VIEW`. Because `viewDependencies` is a typed field on `View` rather than an encoded string property, there is no way for the catalog to reconstruct it after the fact. The fix is one line and matches how the other nullable fields (`currentCatalog`, `schemaMode`) are already carried through. Both tests fail without it and pass with it. Happy to drop the end-to-end one in `MetricViewV2CatalogSuite` and keep only the unit test if you'd prefer a smaller patch. Two related gaps I ran into in the same area, which I'm planning to file separately rather than bundling here — flagging them in case you have an opinion on direction: 1. `ResolveSessionCatalog` routes view DDL by catalog name (`ResolvedViewIdentifier` matches only when `isSessionCatalog(catalog)`), so a `RelationCatalog` installed as `spark_catalog` lands on the V1 path instead of the v2 view execs, even though it fully implements `ViewCatalog`. Routing on capability instead would let connectors stop intercepting `CreateView` themselves. `ShowViewsExec` not merging in temp views looks like part of the same story. 2. `ViewCatalog` has no partial-update operation, so a property change must be expressed as a full `replaceView` rebuilt from `viewInfoBuilderFrom`. That works for Spark's own view model but is lossy for catalogs whose views carry more structure — earlier iterations of the interface had `alterView`/`ViewChange`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
