juliuszsompolski commented on PR #53215: URL: https://github.com/apache/spark/pull/53215#issuecomment-3586558757
@cloud-fan I hit an issue with a Table mixin: The option doesn't get applied if table doesn't exist. At first glance, it seem ok, because we are concerned with overwrite/replace. However, there can be a case of two saveAsTable with mode overwrite executing concurrently, where both calls will see the table as not existing yet during the saveAsTable call, but then one will create the table, and the other will replace it. With the mixin on Table, the option to use saveAsTable overwrite semantics does not get passed to the one doing the replace. Because of that, I reverted to using a TableProvider interface. I shortened the name as requested (the current name makes sense for TableProvider as well) and I added a boolean method because it can still be useful to be able to override it. I tested it end to end in my local environment with Spark and Delta together. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
