[
https://issues.apache.org/jira/browse/TEZ-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057284#comment-17057284
]
Jonathan Turner Eagles commented on TEZ-4131:
---------------------------------------------
Thanks for the patch, [~zhangbutao]. The test failure is related in hadoop
shims as apache commons collections 3.x is needed for Configuration class from
hadoop. Please remove the exclusions from org.apache.hadoop:hadoop-common and
org.apache.hadoop:hadoop-yarn-server-timeline-pluginstorage. The are needed in
the full distribution, but not directly depending on apache commons collections
3.x will remove it from the minimal distribution, which is the preferred way of
installing and using tez.
> Reduce apache commons collections direct dependencies
> -----------------------------------------------------
>
> Key: TEZ-4131
> URL: https://issues.apache.org/jira/browse/TEZ-4131
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Jonathan Turner Eagles
> Assignee: zhangbutao
> Priority: Minor
> Attachments: TEZ-4131.1.patch
>
>
> Tez has limited use of direct apache commons collects, but ships both commons
> collections 3 and 4
> {noformat:title=Dependent jars}
> commons-collections-3.2.2.jar
> commons-collections4-4.1.jar
> {noformat}
> {noformat:title=Usage}
> $ git grep org.apache.commons.coll
> tez-api/src/main/java/org/apache/tez/dag/api/DAG.java:import
> org.apache.commons.collections4.BidiMap;
> tez-api/src/main/java/org/apache/tez/dag/api/DAG.java:import
> org.apache.commons.collections4.bidimap.DualLinkedHashBidiMap;
> tez-dag/src/main/java/org/apache/tez/dag/app/TaskCommunicatorManager.java:import
> org.apache.commons.collections4.ListUtils;
> tez-mapreduce/src/main/java/org/apache/tez/mapreduce/hadoop/mapred/MRCounters.java:import
> org.apache.commons.collections.IteratorUtils;
> tez-plugins/tez-history-parser/src/main/java/org/apache/tez/history/parser/datamodel/DagInfo.java:import
> org.apache.commons.collections.BidiMap;
> tez-plugins/tez-history-parser/src/main/java/org/apache/tez/history/parser/datamodel/DagInfo.java:import
> org.apache.commons.collections.bidimap.DualHashBidiMap;
> {noformat}
> Likely, we can either converge on either commons collections 3 or 4 or remove
> them all together if a viable substitute for the bidirectional maps can be
> found.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
