[jira] [Commented] (TEZ-3900) upgrade to a recent guava version
[ https://issues.apache.org/jira/browse/TEZ-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393675#comment-16393675 ] Sergey Shelukhin commented on TEZ-3900: --- Nm, we actually just broke our own shading so tez jar came first on the classpath after that. I guess this can wait for TEZ-2164 > upgrade to a recent guava version > - > > Key: TEZ-3900 > URL: https://issues.apache.org/jira/browse/TEZ-3900 > Project: Apache Tez > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: TEZ-3900.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (TEZ-3902) Upgrade to netty-3.10.5.Final.jar
[
https://issues.apache.org/jira/browse/TEZ-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Wohlstadter updated TEZ-3902:
--
Comment: was deleted
(was: [~jlowe] [~jeagles]
Follow up info from Thurs. meeting:
Netty 3.6.2 has CVE vulnerabilities. None are listed for 3.10.5.
[https://www.cvedetails.com/vulnerability-list/vendor_id-13290/product_id-27592/Netty-Project-Netty.html]
compile scope is including netty jar in the tez-dist artifacts:
{{/pom.xml}}
{code:java}
3.6.2.Final
...
io.netty
netty
compile
${netty.version}
{code}
)
> Upgrade to netty-3.10.5.Final.jar
> -
>
> Key: TEZ-3902
> URL: https://issues.apache.org/jira/browse/TEZ-3902
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Eric Wohlstadter
>Assignee: Jonathan Eagles
>Priority: Major
>
> Hadoop 3 and Hive have upgraded to netty-3.10.5.Final, which is not
> compatible with current Tez dependency netty-3.6.2.Final.
>
> However, org.apache.tez.shufflehandler.ShuffleHandler depends on 3.6.2
> specific methods.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (TEZ-3902) Upgrade to netty-3.10.5.Final.jar
[ https://issues.apache.org/jira/browse/TEZ-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles reassigned TEZ-3902: Assignee: Jonathan Eagles > Upgrade to netty-3.10.5.Final.jar > - > > Key: TEZ-3902 > URL: https://issues.apache.org/jira/browse/TEZ-3902 > Project: Apache Tez > Issue Type: Improvement >Reporter: Eric Wohlstadter >Assignee: Jonathan Eagles >Priority: Major > > Hadoop 3 and Hive have upgraded to netty-3.10.5.Final, which is not > compatible with current Tez dependency netty-3.6.2.Final. > > However, org.apache.tez.shufflehandler.ShuffleHandler depends on 3.6.2 > specific methods. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-3902) Upgrade to netty-3.10.5.Final.jar
[
https://issues.apache.org/jira/browse/TEZ-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393459#comment-16393459
]
Eric Wohlstadter commented on TEZ-3902:
---
[~jlowe] [~jeagles]
Follow up info from Thurs. meeting:
Netty 3.6.2 has CVE vulnerabilities. None are listed for 3.10.5.
[https://www.cvedetails.com/vulnerability-list/vendor_id-13290/product_id-27592/Netty-Project-Netty.html]
compile scope is including netty jar in the tez-dist artifacts:
{{/pom.xml}}
{code:java}
3.6.2.Final
...
io.netty
netty
compile
${netty.version}
{code}
> Upgrade to netty-3.10.5.Final.jar
> -
>
> Key: TEZ-3902
> URL: https://issues.apache.org/jira/browse/TEZ-3902
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Eric Wohlstadter
>Priority: Major
>
> Hadoop 3 and Hive have upgraded to netty-3.10.5.Final, which is not
> compatible with current Tez dependency netty-3.6.2.Final.
>
> However, org.apache.tez.shufflehandler.ShuffleHandler depends on 3.6.2
> specific methods.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (TEZ-3874) NPE in TezClientUtils when "yarn.resourcemanager.zk-address" is present in Configuration
[
https://issues.apache.org/jira/browse/TEZ-3874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393049#comment-16393049
]
Jason Lowe commented on TEZ-3874:
-
>From precommit build:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12913644/TEZ-3874.6.patch
against master revision 82d73b3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2741//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2741//console
Committing this.
> NPE in TezClientUtils when "yarn.resourcemanager.zk-address" is present in
> Configuration
>
>
> Key: TEZ-3874
> URL: https://issues.apache.org/jira/browse/TEZ-3874
> Project: Apache Tez
> Issue Type: Bug
>Affects Versions: 0.9.1
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Blocker
> Attachments: TEZ-3874.1.patch, TEZ-3874.3.patch, TEZ-3874.4.patch,
> TEZ-3874.5.patch, TEZ-3874.6.patch, TEZ-3874.patch.2
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> "yarn.resourcemanager.zk-address" is deprecated in favor of
> "hadoop.zk.address" for Hadoop 2.9+.
> Configuration base class does't auto-translate the deprecation. Only
> YarnConfiguration applies the translation.
> In TezClientUtils.createFinalConfProtoForApp, a NPE is throw if
> "yarn.resourcemanager.zk-address" is present in the Configuration.
> {code}
> for (Entry entry : amConf) {
> PlanKeyValuePair.Builder kvp = PlanKeyValuePair.newBuilder();
> kvp.setKey(entry.getKey());
> kvp.setValue(amConf.get(entry.getKey()));
> builder.addConfKeyValues(kvp);
> }
> {code}
> Even though Tez is not specifically looking for the deprecated property,
> {{amConf.get(entry.getKey())}} will find it during the iteration, if it is in
> any of the merged xml property resources.
> {{amConf.get(entry.getKey())}} will return null, and {{kvp.setValue(null)}}
> will trigger NPE.
> Suggested solution is to change to:
> {code}
> YarnConfiguration wrappedConf = new YarnConfiguration(amConf);
> for (Entry entry : wrappedConf) {
> PlanKeyValuePair.Builder kvp = PlanKeyValuePair.newBuilder();
> kvp.setKey(entry.getKey());
> kvp.setValue(wrappedConf.get(entry.getKey()));
> builder.addConfKeyValues(kvp);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
