[
https://issues.apache.org/jira/browse/TEZ-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17494509#comment-17494509
]
László Bodor edited comment on TEZ-4300 at 2/18/22, 10:18 AM:
--------------------------------------------------------------
regarding testing failures: it's not necessarily related to protobuf, rather
hadoop version
if hadoopShim is null, it could mean that hadoop shim loader hasn't been able
to e.g. figure out hadoop version
I'm not really sure about your build environment
for further details could you please attach logs of an actual test, e.g.
TestDAGRecovery
you can find it under a folder like below:
{code}
tez-tests/target/tmp/org.apache.tez.test.TestDAGRecovery/yarn-555841331/org.apache.tez.test.TestDAGRecovery-logDir-nm-0_0/
{code}
was (Author: abstractdog):
regarding testing failures: it's not necessarily related to protobuf, rather
hadoop version
if hadoopShim is null, it could mean that hadoop shim loader hasn't been able
to e.g. figure out hadoop version
I'm not really sure about your build environment
for further details could you please attach logs of an actual test, e.g.
TestDAGRecovery
you can find it under a folder like below:
{code}
tmp/org.apache.tez.test.TestDAGRecovery/yarn-555841331/org.apache.tez.test.TestDAGRecovery-logDir-nm-0_0/
{code}
> Download protoc automatically compile/development time
> ------------------------------------------------------
>
> Key: TEZ-4300
> URL: https://issues.apache.org/jira/browse/TEZ-4300
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Attachments: org.apache.tez.dag.app.TestRecoveryParser-output.txt,
> org.apache.tez.dag.app.dag.impl.TestDAGRecovery-output.txt,
> org.apache.tez.dag.history.events.TestHistoryEventsProtoConversion-output.txt,
> tez_build_protobuf_3_7_1.log
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Currently, Tez cannot be built out of the box because tez-api needs a
> protobuf compiler (protoc executable), which was also a problematic
> dependency in a sense that downloading/installing it is a burden on
> developers. Even if we have tons of
> [gists|https://gist.github.com/abstractdog/1eddeac9cc09815ded5253e3a83293a8]
> for that, it's still uncomfortable. Not to mention if we want to support more
> versions at the same time.
> Shipping the typically used versions (2.5.0, 3.x for later) of the protoc
> executable could make sense, but I'm not sure at the moment if it has any
> license issues, [need to
> check|https://github.com/protocolbuffers/protobuf/blob/master/LICENSE], also
> it's not general, instead an intelligent script could do it for developers.
> For 2.5, it should be done from source:
> https://github.com/protocolbuffers/protobuf/releases/tag/v2.5.0
> But for newer versions, the protoc executable can be downloaded from the
> releases page: https://github.com/protocolbuffers/protobuf/releases/tag/v3.7.1
> https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protoc-3.7.1-osx-x86_64.zip
> https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protoc-3.7.1-linux-x86_64.zip
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
