[jira] [Commented] (TEZ-3916) Add hadoop-azure-datalake jar to azure profile
[
https://issues.apache.org/jira/browse/TEZ-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540531#comment-16540531
]
TezQA commented on TEZ-3916:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12931201/TEZ-3916.3.patch
against master revision 919d328.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2857//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2857//console
This message is automatically generated.
> Add hadoop-azure-datalake jar to azure profile
> --
>
> Key: TEZ-3916
> URL: https://issues.apache.org/jira/browse/TEZ-3916
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Critical
> Fix For: 0.10.0
>
> Attachments: TEZ-3916.1.patch, TEZ-3916.2.patch, TEZ-3916.3.patch
>
>
> This jar is required for secure access to Azure object storage:
> https://hadoop.apache.org/docs/current/hadoop-azure-datalake/index.html
> There is already an azure profile in Tez but it doesn't include this jar.
> Since the jar is only supported on Hadoop 2.8+, will either need to:
> 1. Determine that including it in a 2.7 build is fine
> 2. Or if it is not fine, then include the jar only when both the 2.8 profile
> and the azure profile are activated
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (TEZ-3916) Add hadoop-azure-datalake jar to azure profile
[ https://issues.apache.org/jira/browse/TEZ-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16540447#comment-16540447 ] Eric Wohlstadter commented on TEZ-3916: --- Resubmitting patch. Expecting no test failures this time. > Add hadoop-azure-datalake jar to azure profile > -- > > Key: TEZ-3916 > URL: https://issues.apache.org/jira/browse/TEZ-3916 > Project: Apache Tez > Issue Type: Improvement >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Critical > Fix For: 0.10.0 > > Attachments: TEZ-3916.1.patch, TEZ-3916.2.patch, TEZ-3916.3.patch > > > This jar is required for secure access to Azure object storage: > https://hadoop.apache.org/docs/current/hadoop-azure-datalake/index.html > There is already an azure profile in Tez but it doesn't include this jar. > Since the jar is only supported on Hadoop 2.8+, will either need to: > 1. Determine that including it in a 2.7 build is fine > 2. Or if it is not fine, then include the jar only when both the 2.8 profile > and the azure profile are activated -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-3916) Add hadoop-azure-datalake jar to azure profile
[ https://issues.apache.org/jira/browse/TEZ-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16537724#comment-16537724 ] Eric Wohlstadter commented on TEZ-3916: --- [~gopalv] The test failure is expected because of HADOOP-15450, which will be fixed by TEZ-3955. > Add hadoop-azure-datalake jar to azure profile > -- > > Key: TEZ-3916 > URL: https://issues.apache.org/jira/browse/TEZ-3916 > Project: Apache Tez > Issue Type: Improvement >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Critical > Fix For: 0.10.0 > > Attachments: TEZ-3916.1.patch, TEZ-3916.2.patch > > > This jar is required for secure access to Azure object storage: > https://hadoop.apache.org/docs/current/hadoop-azure-datalake/index.html > There is already an azure profile in Tez but it doesn't include this jar. > Since the jar is only supported on Hadoop 2.8+, will either need to: > 1. Determine that including it in a 2.7 build is fine > 2. Or if it is not fine, then include the jar only when both the 2.8 profile > and the azure profile are activated -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-3916) Add hadoop-azure-datalake jar to azure profile
[
https://issues.apache.org/jira/browse/TEZ-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16537715#comment-16537715
]
TezQA commented on TEZ-3916:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12930897/TEZ-3916.2.patch
against master revision 12ff937.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2855//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2855//console
This message is automatically generated.
> Add hadoop-azure-datalake jar to azure profile
> --
>
> Key: TEZ-3916
> URL: https://issues.apache.org/jira/browse/TEZ-3916
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Critical
> Fix For: 0.10.0
>
> Attachments: TEZ-3916.1.patch, TEZ-3916.2.patch
>
>
> This jar is required for secure access to Azure object storage:
> https://hadoop.apache.org/docs/current/hadoop-azure-datalake/index.html
> There is already an azure profile in Tez but it doesn't include this jar.
> Since the jar is only supported on Hadoop 2.8+, will either need to:
> 1. Determine that including it in a 2.7 build is fine
> 2. Or if it is not fine, then include the jar only when both the 2.8 profile
> and the azure profile are activated
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (TEZ-3916) Add hadoop-azure-datalake jar to azure profile
[ https://issues.apache.org/jira/browse/TEZ-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16537541#comment-16537541 ] Gopal V commented on TEZ-3916: -- These jars are in the tez.tar.gz (similar to other hadoop-* jars) when building with -Pazure options. The change LGTM - +1 > Add hadoop-azure-datalake jar to azure profile > -- > > Key: TEZ-3916 > URL: https://issues.apache.org/jira/browse/TEZ-3916 > Project: Apache Tez > Issue Type: Improvement >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Critical > Fix For: 0.10.0 > > Attachments: TEZ-3916.1.patch > > > This jar is required for secure access to Azure object storage: > https://hadoop.apache.org/docs/current/hadoop-azure-datalake/index.html > There is already an azure profile in Tez but it doesn't include this jar. > Since the jar is only supported on Hadoop 2.8+, will either need to: > 1. Determine that including it in a 2.7 build is fine > 2. Or if it is not fine, then include the jar only when both the 2.8 profile > and the azure profile are activated -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-3916) Add hadoop-azure-datalake jar to azure profile
[ https://issues.apache.org/jira/browse/TEZ-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463857#comment-16463857 ] Steve Loughran commented on TEZ-3916: - # if you want to use adl:// as a source or dest, you need hadoop-azure-datalake in your classpath. # same as s3a: ==> hadoop-aws, and wasb => hadoop-azure. If you put the new azure JAR in the azure profile then people who don't ask for the azure packaging, don't get it. Which is clearly what they want. > Add hadoop-azure-datalake jar to azure profile > -- > > Key: TEZ-3916 > URL: https://issues.apache.org/jira/browse/TEZ-3916 > Project: Apache Tez > Issue Type: Improvement >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Critical > Fix For: 0.10.0 > > Attachments: TEZ-3916.1.patch > > > This jar is required for secure access to Azure object storage: > https://hadoop.apache.org/docs/current/hadoop-azure-datalake/index.html > There is already an azure profile in Tez but it doesn't include this jar. > Since the jar is only supported on Hadoop 2.8+, will either need to: > 1. Determine that including it in a 2.7 build is fine > 2. Or if it is not fine, then include the jar only when both the 2.8 profile > and the azure profile are activated -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-3916) Add hadoop-azure-datalake jar to azure profile
[ https://issues.apache.org/jira/browse/TEZ-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463312#comment-16463312 ] Eric Wohlstadter commented on TEZ-3916: --- [~jeagles] This additional dependency is needed to continue supporting use-cases that were already introduced by https://issues.apache.org/jira/browse/TEZ-2632 [~gopalv] [~ste...@apache.org] Can you provide additional context? > Add hadoop-azure-datalake jar to azure profile > -- > > Key: TEZ-3916 > URL: https://issues.apache.org/jira/browse/TEZ-3916 > Project: Apache Tez > Issue Type: Improvement >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Critical > Fix For: 0.10.0 > > Attachments: TEZ-3916.1.patch > > > This jar is required for secure access to Azure object storage: > https://hadoop.apache.org/docs/current/hadoop-azure-datalake/index.html > There is already an azure profile in Tez but it doesn't include this jar. > Since the jar is only supported on Hadoop 2.8+, will either need to: > 1. Determine that including it in a 2.7 build is fine > 2. Or if it is not fine, then include the jar only when both the 2.8 profile > and the azure profile are activated -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-3916) Add hadoop-azure-datalake jar to azure profile
[ https://issues.apache.org/jira/browse/TEZ-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461898#comment-16461898 ] Jonathan Eagles commented on TEZ-3916: -- Without really knowing background, adding a dependency without adding any code that uses that dependency seems broken or fragile at the very least. Can you show how Tez needs this dependency to be part of tez and shipped with tez. > Add hadoop-azure-datalake jar to azure profile > -- > > Key: TEZ-3916 > URL: https://issues.apache.org/jira/browse/TEZ-3916 > Project: Apache Tez > Issue Type: Improvement >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Critical > Fix For: 0.10.0 > > Attachments: TEZ-3916.1.patch > > > This jar is required for secure access to Azure object storage: > https://hadoop.apache.org/docs/current/hadoop-azure-datalake/index.html > There is already an azure profile in Tez but it doesn't include this jar. > Since the jar is only supported on Hadoop 2.8+, will either need to: > 1. Determine that including it in a 2.7 build is fine > 2. Or if it is not fine, then include the jar only when both the 2.8 profile > and the azure profile are activated -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-3916) Add hadoop-azure-datalake jar to azure profile
[
https://issues.apache.org/jira/browse/TEZ-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461854#comment-16461854
]
TezQA commented on TEZ-3916:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12921664/TEZ-3916.1.patch
against master revision c5c26c6.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2783//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2783//console
This message is automatically generated.
> Add hadoop-azure-datalake jar to azure profile
> --
>
> Key: TEZ-3916
> URL: https://issues.apache.org/jira/browse/TEZ-3916
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Eric Wohlstadter
>Assignee: Eric Wohlstadter
>Priority: Critical
> Fix For: 0.10.0
>
> Attachments: TEZ-3916.1.patch
>
>
> This jar is required for secure access to Azure object storage:
> https://hadoop.apache.org/docs/current/hadoop-azure-datalake/index.html
> There is already an azure profile in Tez but it doesn't include this jar.
> Since the jar is only supported on Hadoop 2.8+, will either need to:
> 1. Determine that including it in a 2.7 build is fine
> 2. Or if it is not fine, then include the jar only when both the 2.8 profile
> and the azure profile are activated
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (TEZ-3916) Add hadoop-azure-datalake jar to azure profile
[ https://issues.apache.org/jira/browse/TEZ-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461814#comment-16461814 ] Eric Wohlstadter commented on TEZ-3916: --- [~jeagles] Just an FYI: I'd like to include this in 0.10.0 as it depends on Hadoop 2.8+ > Add hadoop-azure-datalake jar to azure profile > -- > > Key: TEZ-3916 > URL: https://issues.apache.org/jira/browse/TEZ-3916 > Project: Apache Tez > Issue Type: Improvement >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Critical > Fix For: 0.10.0 > > Attachments: TEZ-3916.1.patch > > > This jar is required for secure access to Azure object storage: > https://hadoop.apache.org/docs/current/hadoop-azure-datalake/index.html > There is already an azure profile in Tez but it doesn't include this jar. > Since the jar is only supported on Hadoop 2.8+, will either need to: > 1. Determine that including it in a 2.7 build is fine > 2. Or if it is not fine, then include the jar only when both the 2.8 profile > and the azure profile are activated -- This message was sent by Atlassian JIRA (v7.6.3#76005)
