JeroenSchmidt commented on issue #10493: URL: https://github.com/apache/iceberg/issues/10493#issuecomment-6165872523
Hi all, I recently encountered this same problem (as reported in this ticket and in this OpenLineage issue https://github.com/OpenLineage/OpenLineage/issues/4040). TLDR: A spark session reads a table with stale metadata due to the cache invalidation not triggering after a write occurring on the same table earlier in the session run because the OpenLineage metadata read event keep the cache alive. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
