JoshRosen commented on pull request #33989: URL: https://github.com/apache/spark/pull/33989#issuecomment-926277286
Do we know how the Guava version upgrade will interact with the `IsolatedClientLoader` used for metastore clients? If I'm understanding the `IsolatedClientLoader` code correctly, it looks like Guava classes will be shared between the root classloader and the metastore classloader (i.e. the metastore client will use the Spark classloader's version of that class): https://github.com/apache/spark/blob/ef7441bb4245e2fb08d70a8025a2c6d8bebfe75b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala#L195-L209 It looks like we also pin Guava 14.0.1 when using Ivy to resolve metastore artifacts: https://github.com/apache/spark/blob/ef7441bb4245e2fb08d70a8025a2c6d8bebfe75b/sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala#L132-L135 Given this, could we run risks of breakage if old Hive metastore clients receive a newer version of Guava? Maybe we can remove this from `isSharedClass`; I'm not sure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
