Aggarwal-Raghav commented on PR #6181: URL: https://github.com/apache/hive/pull/6181#issuecomment-3588818227
@deniskuzZ , please find the dependency tree and explanation: [dependency_tree.txt](https://github.com/user-attachments/files/23821658/dependency_tree.txt) **Dependency Updates** Previous State: Hive shipped with commons-collections 3.2.2 and 4.1. New State after this PR: Hive will ship with commons-collections 3.2.2 and **4.4.** We cannot fully migrate to commons-collections 4.x at this time due to transitive dependencies. Libraries such as `hadoop-3.4.1, commons-beanutils, accumulo, atlas, and opencsv` still require commons-collections 3.x. To support these third-party dependencies, Hive will continue to ship both version 3.x and 4.x. Based on the stacktrace attached in description, ClassNotFound was thrown by Hive https://github.com/apache/hive/blob/7a1ffc7dd8edb88837099167ae2c6fd16be91f55/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java#L496 **That's why import stament change from 3.x to 4.x is done.** Its possible that with only import changes we're good and don't need to shade commons-collection-3.x but **to be on safe side** shading is done in-case if any codeflow, while running insite tez container, make use of these 3rd party dependency (accumulo, beanutils, opencsv) which used commons-collectcions3.x then ClassNotFound will be thrown. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
