rdblue commented on pull request #3543: URL: https://github.com/apache/iceberg/pull/3543#issuecomment-968375529
I think that the original reason why we removed the cache was that Spark wasn't correctly invalidating table references. From the original PR, #1203: > As long as a reference to a table exists, the cache should continue to use that reference so that writes update the table for all queries. This was finally fixed on the Spark side so Spark will invalidate its internal caches when it executes a write to a table. This relieves the need for Iceberg to keep all table references in sync underneath Spark. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
