[
https://issues.apache.org/jira/browse/TEZ-4298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17349141#comment-17349141
]
László Bodor edited comment on TEZ-4298 at 5/21/21, 10:18 AM:
--------------------------------------------------------------
[~jeagles]: this change looks more and more valuable as I realized that in
order to upgrade to hadoop 3.3, we don't have to upgrade to protobuf 3 in tez
(as opposed to as I described on TEZ-4152). Protobuf was completely relocated
in source level in hadoop 3.3 (not only shaded), so theoretically we won't have
conflicts. I'm about to simply testing this in the scope of TEZ-4311.
I can imagine a scenario, where we upgrade to hadoop 3.3, and still using
protobuf 2.5 within tez, as hadoop has nothing to do with the protobuf messages
sent between tez components. If my assumption is true, we don't have to take
care of breaking changes.
Please let me know if you're aware of anything that I'm missing here!
was (Author: abstractdog):
[~jeagles]: this change looks more and more valuable as I realized that in
order to upgrade to hadoop 3.3, we don't have to upgrade to protobuf 3 in tez
(as opposed to as I described on TEZ-4152). Protobuf was completely relocated
in source level in hadoop 3.3 (not only shaded), so theoretically we won't have
conflicts. I'm about to simply testing this in the scope of TEZ-4311.
I can imagine a scenario, where we upgrade to hadoop 3.3, and still using
protobuf 2.5 within tez, as hadoop has nothing to do with the protobuf messages
sent between tez components. If my assumption is true, we don't have to take
care of breaking changes.
> ShuffleHandler is not source compatible with Hadoop 3.3.0
> ---------------------------------------------------------
>
> Key: TEZ-4298
> URL: https://issues.apache.org/jira/browse/TEZ-4298
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Jonathan Turner Eagles
> Assignee: Jonathan Turner Eagles
> Priority: Major
> Attachments: TEZ-4298.001.patch, TEZ-4298.001.patch,
> TEZ-4298.002.patch
>
>
> {noformat}
> mvn clean package -DskipTests -pl '!tez-ui' -Dhadoop.version=3.3.0
> [ERROR] COMPILATION ERROR :
> [INFO] -------------------------------------------------------------
> [ERROR]
> tez/tez-plugins/tez-aux-services/src/main/java/org/apache/tez/auxservices/ShuffleHandler.java:[764,45]
> incompatible types: com.google.protobuf.ByteString cannot be converted to
> org.apache.hadoop.thirdparty.protobuf.ByteString
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
