LuciferYang edited a comment on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-896504990
Thank you for your review @gatorsmile ~ On the one hand, I think the main purpose of using Guava cache in the code is to solve the problem of hot path access, so I want to recommend a faster library assuming I understand it correctly , there are some other benchmark results in SPARK-34309. On the other hand, Spark is still using guava 14.0.1, which is a 2013 version and the latest version is 30.1. There are some known issues related to Guava Cache(such as https://github.com/google/guava/issues/1761, https://github.com/google/guava/issues/1337 and so on), and I also found that some JIRA wanted to upgrade the Guava version(like SPARK-33090，SPARK-32502, SPARK-23897, SPARK-25762 and so on) but didn't completed (I'm very sorry I didn't study the reasons for why they weren't completed, maybe it's related to some cascading dependencies?), so I try to use a new Cache library to get rid of the dependence on Guava cache and it's easier to upgrade it to fix some possible issues Does the current use of Caffeine have any negative effects? If it exists, we can revert this pr. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
