HiuKwok commented on PR #45154: URL: https://github.com/apache/spark/pull/45154#issuecomment-1955967683
> @HiuKwok Hmm... seems a class initialization would trigger all referenced classes loading. we may need to fix it on the Hive side by breaking the `Utils` up to avoid `javax.*` classes loading. https://github.com/apache/hive/blob/rel/release-2.3.9/shims/common/src/main/java/org/apache/hadoop/hive/shims/Utils.java I agree, that would be great if Hive could separate the part of the Servlet portion from the Utils, however, I would argue, that this is a nice to have rather than a must. In this particular case, we should be introducing the Javax-servlet-api as required by Hive, because at the end of the day, we (Apache Spark) are the end users of the Hive library, and our role is to make sure all transitive dependencies introduced by the third parties library are being bundled along with Spark distribution, so no ClassNotFound would be thrown during the runtime. In the ideal world, we would like to have all transitive dependencies match up with what Spark is using, but this is hardly the reality, this can only be done on best best-effort basis. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
