William created BIGTOP-3813:
-------------------------------
Summary: Incompatible guava & commons-lang3 Versions in Oozie
Sharelib
Key: BIGTOP-3813
URL: https://issues.apache.org/jira/browse/BIGTOP-3813
Project: Bigtop
Issue Type: Bug
Components: hadoop
Affects Versions: 3.0.1
Reporter: William
Attachments: Oozie-Sharelib-Dependency-Update.patch
Within the Oozie do-component-build script, the version of dependent Hadoop
components (Hadoop, Spark, Hive, etc) are updated to the target Bigtop
versions. These target versions are included in the Oozie sharelib. However,
the build does not update the version of dependencies for these components,
namely Spark in the case we ran into. We found that when Spark was running in
yarn-cluster mode, and the oozie sharelib was uploaded to HDFS, any Oozie
workflows with Spark tasks would fail due to guava and commons-lang3 versions
included in the oozie sharelib that were still targetting the old spark version.
Updating these dependencies in the oozie sharelib resolved the workflow
failures:
guava: 14.0.1 -> 27.0-jre
commons-lang3: 3.3.9 -> 3.9
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
