HeartSaVioR commented on pull request #31541: URL: https://github.com/apache/spark/pull/31541#issuecomment-781744483
OK the reason of using session catalog looks to be clear (support v1/v2 tables altogether in the same metastore), and I agree that's probably not something we could do with custom catalog. (Some tricks would work, like implementing both session catalog and custom catalog which session catalog ignores v2 tables it can't read, and vice versa, but I'd agree that's quite redundant.) Probably the possible concern is that once the table name resolution is delegated to the catalog, it gives diverged UX among different implementations of session catalog. I'd guess each session catalog would describe how the resolution works kindly and how it is different from the built-in session catalog, so that Spark issue doesn't go to Delta/Iceberg and vice versa. Otherwise I agree that's not something wrong. We may still want to have the check in somewhere else just for the built-in session catalog, as there's a comment in #31553. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
