[
https://issues.apache.org/jira/browse/TEZ-4154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093670#comment-17093670
]
László Bodor commented on TEZ-4154:
-----------------------------------
TestSpeculation is failed which is not related
could you please take a look at [^TEZ-4154.04.patch] [~ashutoshc],
[~jeagles], [~rizhang]?
this patch is made as a preparation for handling hadoop 3.3 upgrade, the main
goal is to abstract out everything which I bumped into while compiling tez on
hadoop-3.4 snapshot in TEZ-4152
the patch basically:
1. introduces a new module, tez-protobuf, which encloses protobuf inclusion
into tez project
2. defines maven replacer plugin in order modify source generation, using a new
GeneratedMessage class, which extends protobuf's one --> my plan is that in
case of a hadoop 3.3 upgrade, this GeneratedMessage inheritance can be easily
changed to the new, relocated hadoop's GeneratedMessage in a single place (you
can check [this
comment|https://issues.apache.org/jira/browse/TEZ-4152?focusedCommentId=17090713&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17090713]
for my previously failing attempts to achieve this)
3. ShuffleHandler TokenProto change: it was a compilation failure while
depending on hadoop 3.3 (due to package collision, because of the same root
cause, relocated protobuf classes in hadoop)
> tez-protobuf: a single module for handling protobuf dependency
> --------------------------------------------------------------
>
> Key: TEZ-4154
> URL: https://issues.apache.org/jira/browse/TEZ-4154
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Attachments: TEZ-4154.01.patch, TEZ-4154.02.patch, TEZ-4154.03.patch,
> TEZ-4154.04.patch
>
>
> Similarly to what I've seen in TEZ-2164, I would like to create a module that
> is a single point of control of how tez handles protobuf dependency. In the
> near future, when tez will upgrade to hadoop 3.3, there will be a couple of
> things which should be taken care of (see TEZ-4152), e.g. upgrade to protobuf
> 3.x and handle the possible mess that can come from the fact that hadoop has
> relocated its protobuf.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
