[jira] [Commented] (HADOOP-16869) mvn findbugs:findbugs fails
[
https://issues.apache.org/jira/browse/HADOOP-16869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039896#comment-17039896
]
Tsuyoshi Ozawa commented on HADOOP-16869:
-
Marking this as a blocking issue of HADOOP-16866 because the upgrade task
should be done from a clean state.
> mvn findbugs:findbugs fails
> ---
>
> Key: HADOOP-16869
> URL: https://issues.apache.org/jira/browse/HADOOP-16869
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Akira Ajisaka
>Priority: Major
>
> mvn findbugs:findbugs is failing:
> {noformat}
> [ERROR] Failed to execute goal
> org.codehaus.mojo:findbugs-maven-plugin:3.0.0:findbugs (default-cli) on
> project hadoop-project: Unable to parse configuration of mojo
> org.codehaus.mojo:findbugs-maven-plugin:3.0.0:findbugs for parameter
> pluginArtifacts: Cannot assign configuration entry 'pluginArtifacts' with
> value '${plugin.artifacts}' of type
> java.util.Collections.UnmodifiableRandomAccessList to property of type
> java.util.ArrayList -> [Help 1]
> {noformat}
> We have to update the version of findbugs-maven-plugin.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16866) Upgrade spotbugs' version
[ https://issues.apache.org/jira/browse/HADOOP-16866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated HADOOP-16866: Description: [https://github.com/spotbugs/spotbugs/releases] spotbugs 4.0.0 is now released. We can upgrade spotbugs' version to: 1. 3.1.12 (conservative option) 2. 4.0.0 (which might includes incompatible changes, according to the migration guide: [https://spotbugs.readthedocs.io/en/stable/migration.html]) Step by step approach is also acceptable. was: [https://github.com/spotbugs/spotbugs/releases] spotbugs 4.0.0 is now released. We can upgrade spotbugs' version to: 1. 3.1.12 (conservative option) 2. 4.0.0 (which might includes incompatible changes) Step by step approach is also acceptable. > Upgrade spotbugs' version > - > > Key: HADOOP-16866 > URL: https://issues.apache.org/jira/browse/HADOOP-16866 > Project: Hadoop Common > Issue Type: Task >Reporter: Tsuyoshi Ozawa >Priority: Minor > > [https://github.com/spotbugs/spotbugs/releases] > spotbugs 4.0.0 is now released. > > We can upgrade spotbugs' version to: > 1. 3.1.12 (conservative option) > 2. 4.0.0 (which might includes incompatible changes, according to the > migration guide: [https://spotbugs.readthedocs.io/en/stable/migration.html]) > > Step by step approach is also acceptable. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-16866) Upgrade spotbugs' version
Tsuyoshi Ozawa created HADOOP-16866: --- Summary: Upgrade spotbugs' version Key: HADOOP-16866 URL: https://issues.apache.org/jira/browse/HADOOP-16866 Project: Hadoop Common Issue Type: Task Reporter: Tsuyoshi Ozawa [https://github.com/spotbugs/spotbugs/releases] spotbugs 4.0.0 is now released. We can upgrade spotbugs' version to: 1. 3.1.12 (conservative option) 2. 4.0.0 (which might includes incompatible changes) Step by step approach is also acceptable. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-16596) [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency
[ https://issues.apache.org/jira/browse/HADOOP-16596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17032055#comment-17032055 ] Tsuyoshi Ozawa edited comment on HADOOP-16596 at 2/7/20 4:08 AM: - > One problem I come up with is ease of upgrading protobufs. When we upgrade >the protobuf version still at the server side, we need to change all lines of >code including "protobuf_3_7" (import statements). This is a safer way, but >this makes protobuf upgrading messy. Can we discuss this problem later since >the current patch is valuable enough to be merged? Sorry, this point was wrong. Your latest patch has already addressed the problem. I built your branch and ran some test cases on my local. Tests pass safely. +1 was (Author: ozawa): > One problem I come up with is ease of upgrading protobufs. When we upgrade >the protobuf version still at the server side, we need to change all lines of >code including "protobuf_3_7" (import statements). This is a safer way, but >this makes protobuf upgrading messy. Can we discuss this problem later since >the current patch is valuable enough to be merged? Sorry, this point was wrong. Your latest patch has already addressed the problem. I built your branch and ran some test cases on my local. They pass safely. +1 > [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency > -- > > Key: HADOOP-16596 > URL: https://issues.apache.org/jira/browse/HADOOP-16596 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Major > > Use the shaded protobuf classes from "hadoop-thirdparty" in hadoop codebase. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-16596) [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency
[ https://issues.apache.org/jira/browse/HADOOP-16596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17032055#comment-17032055 ] Tsuyoshi Ozawa edited comment on HADOOP-16596 at 2/7/20 1:35 AM: - > One problem I come up with is ease of upgrading protobufs. When we upgrade >the protobuf version still at the server side, we need to change all lines of >code including "protobuf_3_7" (import statements). This is a safer way, but >this makes protobuf upgrading messy. Can we discuss this problem later since >the current patch is valuable enough to be merged? Sorry, this point was wrong. Your latest patch has already addressed the problem. I built your branch and ran some test cases on my local. They pass safely. +1 was (Author: ozawa): > One problem I come up with is ease of upgrading protobufs. When we upgrade >the protobuf version still at the server side, we need to change all lines of >code including "protobuf_3_7" (import statements). This is a safer way, but >this makes protobuf upgrading messy. Can we discuss this problem later since >the current patch is valuable enough to be merged? Sorry, this point was wrong. Your latest patch has already addressed the problem. Build passes on my local. +1 from me. > [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency > -- > > Key: HADOOP-16596 > URL: https://issues.apache.org/jira/browse/HADOOP-16596 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Major > > Use the shaded protobuf classes from "hadoop-thirdparty" in hadoop codebase. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16596) [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency
[ https://issues.apache.org/jira/browse/HADOOP-16596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17032055#comment-17032055 ] Tsuyoshi Ozawa commented on HADOOP-16596: - > One problem I come up with is ease of upgrading protobufs. When we upgrade >the protobuf version still at the server side, we need to change all lines of >code including "protobuf_3_7" (import statements). This is a safer way, but >this makes protobuf upgrading messy. Can we discuss this problem later since >the current patch is valuable enough to be merged? Sorry, this point was wrong. Your latest patch has already addressed the problem. Build passes on my local. +1 from me. > [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency > -- > > Key: HADOOP-16596 > URL: https://issues.apache.org/jira/browse/HADOOP-16596 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Major > > Use the shaded protobuf classes from "hadoop-thirdparty" in hadoop codebase. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031444#comment-17031444 ] Tsuyoshi Ozawa commented on HADOOP-13363: - Commented on HADOOP-16596. > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Anu Engineer >Assignee: Vinayakumar B >Priority: Major > Labels: security > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-16596) [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency
[ https://issues.apache.org/jira/browse/HADOOP-16596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031440#comment-17031440 ] Tsuyoshi Ozawa edited comment on HADOOP-16596 at 2/6/20 10:22 AM: -- [~vinayakumarb] Thanks for your patch! I am happy since this jira moves forward now. The source of this problem is that Hadoop shares 'different jars with same name' between client and server. The patch solves the problem correctly. This work is awesome. One problem I come up with is ease of upgrading protobufs. When we upgrade the protobuf version still at the server side, we need to change all lines of code including "protobuf_3_7" (import statements). This is a safer way, but this makes protobuf upgrading messy. Can we discuss this problem later since the current patch is valuable enough to be merged? I would like to run tests with your patch locally, trying to check whether it works correctly. was (Author: ozawa): [~vinayakumarb] Thanks for your patch! I am happy since this jira moves forward now. The source of this problem is that Hadoop shares 'different jars with same name' between client and server. The patch solves the problem correctly. This work is awesome. One problem I come up with is ease of upgrading protobufs. When we upgrade the protobuf version still at the server side, we need to change all lines of code including "protobuf_3_7" (import statements). This is a safer way, but this makes protobuf upgrading messy. Can we discuss this problem later since the current patch is both valuable enough to be merged? I would like to run tests with your patch locally, trying to check whether it works correctly. > [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency > -- > > Key: HADOOP-16596 > URL: https://issues.apache.org/jira/browse/HADOOP-16596 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Major > > Use the shaded protobuf classes from "hadoop-thirdparty" in hadoop codebase. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16596) [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency
[ https://issues.apache.org/jira/browse/HADOOP-16596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031440#comment-17031440 ] Tsuyoshi Ozawa commented on HADOOP-16596: - [~vinayakumarb] Thanks for your patch! I am happy since this jira moves forward now. The source of this problem is that Hadoop shares 'different jars with same name' between client and server. The patch solves the problem correctly. This work is awesome. One problem I come up with is ease of upgrading protobufs. When we upgrade the protobuf version still at the server side, we need to change all lines of code including "protobuf_3_7" (import statements). This is a safer way, but this makes protobuf upgrading messy. Can we discuss this problem later since the current patch is both valuable enough to be merged? I would like to run tests with your patch locally, trying to check whether it works correctly. > [pb-upgrade] Use shaded protobuf classes from hadoop-thirdparty dependency > -- > > Key: HADOOP-16596 > URL: https://issues.apache.org/jira/browse/HADOOP-16596 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Vinayakumar B >Assignee: Vinayakumar B >Priority: Major > > Use the shaded protobuf classes from "hadoop-thirdparty" in hadoop codebase. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17031400#comment-17031400 ] Tsuyoshi Ozawa commented on HADOOP-13363: - Sure, I would like to review the code today. Please wait a moment. > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Anu Engineer >Assignee: Vinayakumar B >Priority: Major > Labels: security > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923879#comment-16923879 ] Tsuyoshi Ozawa commented on HADOOP-13363: - [~aajisaka] If we upgrade protobuf version, is any Yetus-side update needed to build Hadoop correctly? > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer >Assignee: Vinayakumar B >Priority: Major > Labels: security > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923878#comment-16923878 ] Tsuyoshi Ozawa edited comment on HADOOP-13363 at 9/6/19 2:41 AM: - [~vinayakumarb] > Have you verified this already? No, I have not. I'm sorry. [~anu] about the build failure, if I understand correctly, we need to update docker file (or a build environment) to include protobuf 3.2.0 too. was (Author: ozawa): [~vinayakumarb] > Have you verified this already? Sorry, I have no time to do so currently. [~anu] about the build failure, if I understand correctly, we need to update docker file (or a build environment) to include protobuf 3.2.0 too. > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer >Assignee: Vinayakumar B >Priority: Major > Labels: security > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923878#comment-16923878 ] Tsuyoshi Ozawa commented on HADOOP-13363: - [~vinayakumarb] > Have you verified this already? Sorry, I have no time to do so currently. [~anu] about the build failure, if I understand correctly, we need to update docker file (or a build environment) to include protobuf 3.2.0 too. > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer >Assignee: Vinayakumar B >Priority: Major > Labels: security > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883990#comment-16883990 ] Tsuyoshi Ozawa commented on HADOOP-13363: - I'm happy that Anu, Yikun, and Steven resume the conversation :-) I don't know it is good time to do the upgrade event. Maybe the most difficult part of this task is to get consensus among us. This is because the upgrade can disrupt other projects which depend on Apache Hadoop as Steve said. In my experience, a lesson which I learned from Guava updating, though I have recognized it as a failure because the patch was reverted, is we should keep the dependencies on common libraries even if Apache Hadoop itself doesn't use it. So, a safer way for the ecosystem I came up with is as follows: 1. Shading updated protobuf version e.g. protobuf v3. 2. Gradually replacing existent parts where protobuf v2.5 is used with protobuf v3. This can be done on a non-master branch. Here, we remain the dependency on protobuf v 2.5. This is because other projects may use it. 3. Announcing when to delete the dependency. 4. Removing the dependency on the future version. This kind of gradual replacing approach might be acceptable by the Hadoop ecosystem, I think. > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer >Priority: Major > Labels: security > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa reassigned HADOOP-13363: --- Assignee: (was: Tsuyoshi Ozawa) > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer >Priority: Major > Labels: security > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14238) [Umbrella] Rechecking Guava's object is not exposed to user-facing API
[ https://issues.apache.org/jira/browse/HADOOP-14238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596292#comment-16596292 ] Tsuyoshi Ozawa commented on HADOOP-14238: - I think it is suitable to close this since HADOOP-14386 reverted the Guava's upgrading on trunk too. Thanks Sunil, Xiao, Wangda, Sean, Haibo, Andrew for taking your time. > [Umbrella] Rechecking Guava's object is not exposed to user-facing API > -- > > Key: HADOOP-14238 > URL: https://issues.apache.org/jira/browse/HADOOP-14238 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Tsuyoshi Ozawa >Priority: Critical > > This is reported by [~hitesh] on HADOOP-10101. > At least, AMRMClient#waitFor takes Guava's Supplier instance as an instance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13007) cherry pick s3 ehancements from PrestoS3FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-13007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16529827#comment-16529827 ] Tsuyoshi Ozawa commented on HADOOP-13007: - I fixed the link to the PrestoS3FileSystem.java. Now, it is under s3 directory. > cherry pick s3 ehancements from PrestoS3FileSystem > -- > > Key: HADOOP-13007 > URL: https://issues.apache.org/jira/browse/HADOOP-13007 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 2.9.0 >Reporter: Steve Loughran >Priority: Minor > > Looking at > [https://github.com/prestodb/presto/blob/master/presto-hive/src/main/java/com/facebook/presto/hive/s3/PrestoS3FileSystem.java], > they've done some interesting things: configurable connection timeouts and, > retry options, statistics to count exceptions caught/re-opened, and more > review them, if there is good stuff there, add it to S3a -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13007) cherry pick s3 ehancements from PrestoS3FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-13007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated HADOOP-13007: Description: Looking at [https://github.com/prestodb/presto/blob/master/presto-hive/src/main/java/com/facebook/presto/hive/s3/PrestoS3FileSystem.java], they've done some interesting things: configurable connection timeouts and, retry options, statistics to count exceptions caught/re-opened, and more review them, if there is good stuff there, add it to S3a was: Looking at [[https://github.com/prestodb/presto/blob/master/presto-hive/src/main/java/com/facebook/presto/hive/PrestoS3FileSystem.java]], they've done some interesting things: configurable connection timeouts and, retry options, statistics to count exceptions caught/re-opened, and more review them, if there is good stuff there, add it to S3a > cherry pick s3 ehancements from PrestoS3FileSystem > -- > > Key: HADOOP-13007 > URL: https://issues.apache.org/jira/browse/HADOOP-13007 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 2.9.0 >Reporter: Steve Loughran >Priority: Minor > > Looking at > [https://github.com/prestodb/presto/blob/master/presto-hive/src/main/java/com/facebook/presto/hive/s3/PrestoS3FileSystem.java], > they've done some interesting things: configurable connection timeouts and, > retry options, statistics to count exceptions caught/re-opened, and more > review them, if there is good stuff there, add it to S3a -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-15151) MapFile.fix creates a wrong index file in case of block-compressed data file.
[ https://issues.apache.org/jira/browse/HADOOP-15151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa reassigned HADOOP-15151: --- Assignee: Grigori Rybkine > MapFile.fix creates a wrong index file in case of block-compressed data file. > - > > Key: HADOOP-15151 > URL: https://issues.apache.org/jira/browse/HADOOP-15151 > Project: Hadoop Common > Issue Type: Bug > Components: common >Reporter: Grigori Rybkine >Assignee: Grigori Rybkine >Priority: Major > Labels: patch > Attachments: HADOOP-15151.001.patch, HADOOP-15151.002.patch, > HADOOP-15151.003.patch, HADOOP-15151.004.patch, HADOOP-15151.004.patch, > HADOOP-15151.005.patch > > > Index file created with MapFile.fix for an ordered block-compressed data file > does not allow to find values for keys existing in the data file via the > MapFile.get method. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245733#comment-16245733 ] Tsuyoshi Ozawa commented on HADOOP-13363: - I think now we can upgrade protobuf more safely especially in Hadoop 3.0 because: 1. Now HBase uses own protobuf, 2. Hadoop 3.0 obligate the ecosystem to use shaded client. What do you think? I personally okay to upgrade the version step by step, meaning upgrading this to 2.6.x instead of 3.x, if we do this carefully. > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer >Assignee: Tsuyoshi Ozawa > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[
https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167229#comment-16167229
]
Tsuyoshi Ozawa commented on HADOOP-13363:
-
{quote}
We need wire-compatibility no matter when we upgrade, whether it's from 2.x to
3.0, or 2.x to 3.1.
{quote}
How can we ensure it? My first thought to ensure this is to run cluster with
protobuf 3.x cluster jar and clients with protobuf 2.x jar.
By the way, the current latest version of protobuf seems to be v3.4.0.
https://github.com/google/protobuf/releases
> Upgrade protobuf from 2.5.0 to something newer
> --
>
> Key: HADOOP-13363
> URL: https://issues.apache.org/jira/browse/HADOOP-13363
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2
>Reporter: Allen Wittenauer
>Assignee: Tsuyoshi Ozawa
> Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch,
> HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch
>
>
> Standard protobuf 2.5.0 does not work properly on many platforms. (See, for
> example, https://gist.github.com/BennettSmith/7111094 ). In order for us to
> avoid crazy work arounds in the build environment and the fact that 2.5.0 is
> starting to slowly disappear as a standard install-able package for even
> Linux/x86, we need to either upgrade or self bundle or something else.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa resolved HADOOP-14284.
-
Resolution: Invalid
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha4
>Reporter: Andrew Wang
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch,
> HADOOP-14284.012.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14238) [Umbrella] Rechecking Guava's object is not exposed to user-facing API
[ https://issues.apache.org/jira/browse/HADOOP-14238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156339#comment-16156339 ] Tsuyoshi Ozawa commented on HADOOP-14238: - [~andrew.wang] [~vinodkv] As far as I know and as [~bharatviswa] mentioned, it would be ideal to substitute Guava's Supplier with java's one. > [Umbrella] Rechecking Guava's object is not exposed to user-facing API > -- > > Key: HADOOP-14238 > URL: https://issues.apache.org/jira/browse/HADOOP-14238 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Tsuyoshi Ozawa >Priority: Blocker > > This is reported by [~hitesh] on HADOOP-10101. > At least, AMRMClient#waitFor takes Guava's Supplier instance as an instance. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-14238) [Umbrella] Rechecking Guava's object is not exposed to user-facing API
[ https://issues.apache.org/jira/browse/HADOOP-14238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa reassigned HADOOP-14238: --- Assignee: (was: Tsuyoshi Ozawa) > [Umbrella] Rechecking Guava's object is not exposed to user-facing API > -- > > Key: HADOOP-14238 > URL: https://issues.apache.org/jira/browse/HADOOP-14238 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Tsuyoshi Ozawa >Priority: Blocker > > This is reported by [~hitesh] on HADOOP-10101. > At least, AMRMClient#waitFor takes Guava's Supplier instance as an instance. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa reassigned HADOOP-14284:
---
Assignee: (was: Tsuyoshi Ozawa)
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha4
>Reporter: Andrew Wang
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch,
> HADOOP-14284.012.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139568#comment-16139568
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
In summary, the Guava's problem is caused by the usages of HDFS's internal
interface by downstream projects. Hence, we will have an effort for urging
downstream project to use the shaded client instead. Is this correct?
Can we think the same way about protobuf problem?
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha4
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch,
> HADOOP-14284.012.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058601#comment-16058601
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
Sorry for the delay.
I would like to wrap up all opinions discussed here before moving forward.
After that, I would like to continue to create a patch based on the approach.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha4
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch,
> HADOOP-14284.012.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Status: Open (was: Patch Available)
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch,
> HADOOP-14284.012.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16001801#comment-16001801
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
[~djp] As I mentioned on HADOOP-14386, I'm okay to downgrade version of Guava
to 0.11.2. On the other hand, I'm still thinking that shading itself is very
useful.
{quote}
However, the other approach - to stay at a version with other downstream
projects unless must be changed (and should change with downstream projects
together) - seems the best strategy here. I didn't see any necessary for
upgrading guava and HADOOP-10101 failed to convince me.
{quote}
First of all, the opinions of users on HADOOP-10101 looks to me like positive
against shading Guava. Secondly, as stack mentioned, the quality of Guava
itself is nice. Guava also announces deprecated methods in the future on ahead.
For example.
https://google.github.io/guava/releases/14.0/api/docs/com/google/common/io/LimitInputStream.html
If we shade and relocate Guava inside Hadoop, we can catch up these changes
easily because the dependency doesn't leak to user-level. It's not a problem of
Guava, IIUC - it's a problem that we failed to decouple with user-level library
and server-side library. There is a possibility that the reams of complaint
come from IDE problem. Hence, I will post a next patch with IDE support after
HADOOP-14386.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch,
> HADOOP-14284.012.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000516#comment-16000516
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
[~andrew.wang] Once HADOOP-14386 is merged, I think we can finish this jira
soon. The test failures we're facing with the latest patch is caused by
updating Guava in timelineserver test module to the newer version than 0.14,
which is the version removing LimitInputStream. After HADOOP-14386, the problem
will be solved at the implementation level.
More important point is that whether we will reach consensus or not.
[~djp] [~vinodkv] could you share your opinion based on the information we
shared after your comments? The looks large patch, however, it only changes the
code of import statement and pom.xml. diff size is as follows as a reference:
{quote}
$ git diff --shortstat d6eed5acca222a281401c1d524824f0460ea7826
1241 files changed, 2946 insertions(+), 2137 deletions(-) 1241 files changed,
2946 insertions(+), 2137 deletions(-)
{quote}
I think maintaining all of staffs of Guava in Hadoop side is also tough work,
so shading Guava looks to me like a good solution. The Guava problem is an
ecosystem-wide problem as you know. If we can solve here, it's good for Hadoop
ecosystem.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch,
> HADOOP-14284.012.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14386) Make trunk work with Guava 11.0.2 again
[ https://issues.apache.org/jira/browse/HADOOP-14386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000244#comment-16000244 ] Tsuyoshi Ozawa edited comment on HADOOP-14386 at 5/8/17 4:21 AM: - [~andrew.wang] I'm surprised that the bugs related Guava cannot be found in unit tests. Anyway, it's basically okay to back the version of Guava to 0.11.2 because the update of Guava caused some unexpected behaviour especially like breaking Timeline Server and incompatibility like YARN-6564. Thanks [~steve_l] for working the topic. We're now choosing the version of Curator with shaded Guava, so we can safely downgrade the version of Guava. was (Author: ozawa): [~andrew.wang] I'm surprised that the bugs related Guava cannot be found in unit tests. Anyway, it's basically okay to back the version of Guava to 0.11.2 because the update of Guava caused some unexpected behaviour especially like breaking Timeline Server and incompatibility like HADOOP-14380. Thanks [~steve_l] for working the topic. We're now choosing the version of Curator with shaded Guava, so we can safely downgrade the version of Guava. > Make trunk work with Guava 11.0.2 again > --- > > Key: HADOOP-14386 > URL: https://issues.apache.org/jira/browse/HADOOP-14386 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.0.0-alpha3 >Reporter: Andrew Wang >Assignee: Andrew Wang > Attachments: HADOOP-14386.001.patch, HADOOP-14386.002.patch > > > As an alternative to reverting or shading HADOOP-10101 (the upgrade of Guava > from 11.0.2 to 21.0), HADOOP-14380 makes the Guava version configurable. > However, it still doesn't compile with Guava 11.0.2, since HADOOP-10101 chose > to use the moved Guava classes rather than replacing them with alternatives. > This JIRA aims to make Hadoop compatible with Guava 11.0.2 as well as 21.0 by > replacing usage of these moved Guava classes. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14386) Make trunk work with Guava 11.0.2 again
[ https://issues.apache.org/jira/browse/HADOOP-14386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000244#comment-16000244 ] Tsuyoshi Ozawa edited comment on HADOOP-14386 at 5/8/17 4:20 AM: - [~andrew.wang] I'm surprised that the bugs related Guava cannot be found in unit tests. Anyway, it's basically okay to back the version of Guava to 0.11.2 because the update of Guava caused some unexpected behaviour especially like breaking Timeline Server and incompatibility like HADOOP-14380. Thanks [~steve_l] for working the topic. We're now choosing the version of Curator with shaded Guava, so we can safely downgrade the version of Guava. was (Author: ozawa): [~andrew.wang] I'm surprised that this bug cannot be found in tests. Anyway, it's basically okay to back the version of Guava to 0.11.2 because the update of Guava caused some unexpected behaviour especially like breaking Timeline Server and incompatibility like HADOOP-14380. Thanks [~steve_l] for working the topic. We're now choosing the version of Curator with shaded Guava, so we can safely downgrade the version of Guava. > Make trunk work with Guava 11.0.2 again > --- > > Key: HADOOP-14386 > URL: https://issues.apache.org/jira/browse/HADOOP-14386 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.0.0-alpha3 >Reporter: Andrew Wang >Assignee: Andrew Wang > Attachments: HADOOP-14386.001.patch, HADOOP-14386.002.patch > > > As an alternative to reverting or shading HADOOP-10101 (the upgrade of Guava > from 11.0.2 to 21.0), HADOOP-14380 makes the Guava version configurable. > However, it still doesn't compile with Guava 11.0.2, since HADOOP-10101 chose > to use the moved Guava classes rather than replacing them with alternatives. > This JIRA aims to make Hadoop compatible with Guava 11.0.2 as well as 21.0 by > replacing usage of these moved Guava classes. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14386) Make trunk work with Guava 11.0.2 again
[ https://issues.apache.org/jira/browse/HADOOP-14386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000244#comment-16000244 ] Tsuyoshi Ozawa commented on HADOOP-14386: - [~andrew.wang] I'm surprised that this bug cannot be found in tests. Anyway, it's basically okay to back the version of Guava to 0.11.2 because the update of Guava caused some unexpected behaviour especially like breaking Timeline Server and incompatibility like HADOOP-14380. Thanks [~steve_l] for working the topic. We're now choosing the version of Curator with shaded Guava, so we can safely downgrade the version of Guava. > Make trunk work with Guava 11.0.2 again > --- > > Key: HADOOP-14386 > URL: https://issues.apache.org/jira/browse/HADOOP-14386 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.0.0-alpha3 >Reporter: Andrew Wang >Assignee: Andrew Wang > Attachments: HADOOP-14386.001.patch, HADOOP-14386.002.patch > > > As an alternative to reverting or shading HADOOP-10101 (the upgrade of Guava > from 11.0.2 to 21.0), HADOOP-14380 makes the Guava version configurable. > However, it still doesn't compile with Guava 11.0.2, since HADOOP-10101 chose > to use the moved Guava classes rather than replacing them with alternatives. > This JIRA aims to make Hadoop compatible with Guava 11.0.2 as well as 21.0 by > replacing usage of these moved Guava classes. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999486#comment-15999486
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
[~djp] It's another option. IMHO, it sounds more tough work than we expected -
e.g. HDFS core including DataNode.java uses CacheBuilder. Could you share the
examples that can be caused even with shading? The patch looks large, but it's
just a change of importing and pom.xml.
[~busbey] thanks a lot! Your advice is correct. I used jarjar-maven-plugin to
be before the package phase([I'm using process-classes to pass tests for
now|https://github.com/oza/hadoop/blob/HADOOP-14284/hadoop-shaded-thirdparty/pom.xml#L84])
because shade:shade is bound to the package phase. The v12 patch includes the
change. PR for review on github is available
[here|https://github.com/apache/hadoop/pull/210].
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch,
> HADOOP-14284.012.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: HADOOP-14284.012.patch
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch,
> HADOOP-14284.012.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999058#comment-15999058
]
Tsuyoshi Ozawa edited comment on HADOOP-14284 at 5/5/17 10:52 PM:
--
-After making shadeTestJar true in hadoop-shaded-thirdparty jar, it works!
Attaching patch soon.-
This was wrong. It still had same error message unfortunately.
was (Author: ozawa):
Oh, after making shadeTestJar true in hadoop-shaded-thirdparty jar, it works!
Attaching patch soon.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999058#comment-15999058
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
Oh, after making shadeTestJar true in hadoop-shaded-thirdparty jar, it works!
Attaching patch soon.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15999022#comment-15999022
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
[~stack] thanks you for your help.
{quote}
Tell us more please. Shading bundles the relocated .class files of guava and
curator; they are included in the thirdparty jar... and the thirdparty jar is
on the classpath, no?
{quote}
Yes, they are. This is a list of the contents of hadoop-shaded-thirdparty:
https://gist.github.com/oza/62fdea66a55c86eda02d2a8530058153
The list shows that shading works well. As a result, mvn install -DskipTests
succeeds with the patch: it means the thirdparty jar is on the classpath at
compile time.
On the other hand, mvn test of hadoop-auth fails with the patch although I
added dependency on hadoop-shaded-thirdparty in compile scope.
https://github.com/oza/hadoop/blob/HADOOP-14284/hadoop-common-project/hadoop-auth/pom.xml#L46
the result of mvn test is as follows:
{quote}
Running
org.apache.hadoop.security.authentication.client.TestKerberosAuthenticator
Tests run: 20, Failures: 5, Errors: 11, Skipped: 0, Time elapsed: 23.407 sec
<<< FAILURE! - in
org.apache.hadoop.security.authentication.client.TestKerberosAuthenticator
testNotAuthenticatedWithMultiAuthHandler\[0\](org.apache.hadoop.security.authentication.client.TestKerberosAuthenticator)
Time elapsed: 1.649 sec <<< ERROR!
java.lang.NoClassDefFoundError:
org/apache/hadoop/shaded/com/google/common/base/Splitter
{quote}
I thought that this might be a problem because the thirdparty jar is not on the
classpath at testing time, Please let me know if I'm missing something.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996429#comment-15996429
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
[~vinodkv] and [~djp], thanks a lot for your feedback.
> A couple of questions: can we just shade some client jars instead of
> everywhere?
> As we keep doing this for other libraries, I'm concerned if our code becomes
> more brittle (changing imports everywhere) and if the build times explode.
We're now doing to shade Guava and Curator in hadoop-shaded-thirdparty, and
trying to import it from hadoop-* projects. The build time of Hadoop doesn't
get increased so much with this approach because of just referring to
hadoop-shaded-thirdparty project from hadoop-* projects. However, I found one
problem in this approach: shaded artifacts(shaded Guava and Curator) in
hadoop-shaded-thirdparty is NOT in classpath, if I understand correctly. To go
with this approach, we need to unzip source code and compile it like HBase does
in hbase-protocol-shaded. This can make Hadoop build fragile and the build time
of Hadoop can increase as Junping Vinod mentioned.
https://github.com/apache/hbase/blob/7700a7fac1262934fe538a96b040793c6ff171ce/hbase-protocol-shaded/pom.xml#L321
Gradle seems to have a feature to do this.
http://stackoverflow.com/questions/26244936/how-to-include-only-project-and-relocated-classes-when-using-gradle-shadow-plugi
> Isn't it better to just shade our final artifacts instead of shading
> individual libraries' jars?
Do you mean that we prepare new project "hadoop-server-modules" and shading
Guava and Curator inside them like hadoop-client-modules? It sounds better
approach to me. By adding skipShade option here, we can overcome build time
problem. [~andrew.wang] [~busbey] [~ajisakaa] What do you think?
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984219#comment-15984219
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
80% work to shade Curator and Guava was done. I'm going to post new patch after
passing tests locally.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: (was: HADOOP-14284.011.patch)
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: HADOOP-14284.011.patch
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch,
> HADOOP-14284.011.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15979662#comment-15979662
]
Tsuyoshi Ozawa edited comment on HADOOP-14284 at 4/22/17 1:02 AM:
--
[~andrew.wang] [~busbey] I see. Let me try in this weekend. Thanks for your
pointing.
was (Author: ozawa):
[~andrew.wang] [~busbey] ah, I see. Let me try in this weekend. Thanks for your
pointing.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15979662#comment-15979662
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
[~andrew.wang] [~busbey] ah, I see. Let me try in this weekend. Thanks for your
pointing.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15974406#comment-15974406
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
{quote}
Does this mean we'd have guava in hadoop-client-modules and in
hadoop-shaded-thirdparty? What you thinking Tsuyoshi Ozawa? Thanks.
{quote}
Yes, it does. I choosed the way as a workaround because I think
hadoop-client-modules shouldn't expose unshaded Guava to end-users of hadoop.
Ideally, I would like to enclose shade inside hadoop-shaded-thirdparty
completely to avoid its complexity.
I think using Curator itself is a good choice for stable ZKRMStateStore. There
are lots things to use ZK correctly as you know, including connection handling,
error handling, etc. Curator does these works instead of us.
{quote}
> Could we get Curator to add a new set of APIs that use Curator-shaded Guava
> objects in their API instead? This would let us fully shade our Guava; the
> partial shading is unsatisfying.
Couldn't we do this shading ourselves? Or worst case, shade our curator use as
well?
{quote}
I think we can. For now, I created branch on my github as a trial.
https://github.com/oza/curator/tree/apache-curator-2.12.0-hadoop
Its diff from original version is small.
https://github.com/oza/curator/commit/7bacc5f79cebaec870f30a85f908fadf3edc95ca
The problem is where to manage the code of forked version and where to push jar
file. Under hadoop project?
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968813#comment-15968813
]
Tsuyoshi Ozawa edited comment on HADOOP-14284 at 4/14/17 9:06 AM:
--
Compilation failure is caused by not refering maven local repository - how can
we install newly added components on test machines? Does anyone know how to do
it?
{quote}
\[ERROR\] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on
project hadoop-auth: Compilation failure: Compilation failure:
\[ERROR\]
/testptch/hadoop/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/ZKSignerSecretProvider.java:\[36,61\]
error: package org.apache.hadoop.shaded.com.google.common.annotations does not
exist
{quote}
was (Author: ozawa):
Compilation failure is caused by not refering maven local repository - how can
we install newly added components on test machines? Does anyone know how to do
it?
{quote}
\[ERROR\] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on
project hadoop-auth: Compilation failure: Compilation failure:
\[ERROR\]
/testptch/hadoop/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/ZKSignerSecretProvider.java:\[36,61\]
error: package org.apache.hadoop.shaded.com.google.common.annotations does not
exist
{quote]
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968813#comment-15968813
]
Tsuyoshi Ozawa edited comment on HADOOP-14284 at 4/14/17 9:06 AM:
--
Compilation failure is caused by not refering maven local repository - how can
we install newly added components on test machines? Does anyone know how to do
it?
{quote}
\[ERROR\] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on
project hadoop-auth: Compilation failure: Compilation failure:
\[ERROR\]
/testptch/hadoop/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/ZKSignerSecretProvider.java:\[36,61\]
error: package org.apache.hadoop.shaded.com.google.common.annotations does not
exist
{quote]
was (Author: ozawa):
Compilation failure is caused by not refering maven local repository - how can
we install newly added components on test machines? Does anyone know how to do
it?
{quote}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on
project hadoop-auth: Compilation failure: Compilation failure:
[ERROR]
/testptch/hadoop/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/ZKSignerSecretProvider.java:[36,61]
error: package org.apache.hadoop.shaded.com.google.common.annotations does not
exist
{quote]
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968813#comment-15968813
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
Compilation failure is caused by not refering maven local repository - how can
we install newly added components on test machines? Does anyone know how to do
it?
{quote}]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on
project hadoop-auth: Compilation failure: Compilation failure:
[ERROR]
/testptch/hadoop/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/ZKSignerSecretProvider.java:[36,61]
error: package org.apache.hadoop.shaded.com.google.common.annotations does not
exist
{quote]
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968813#comment-15968813
]
Tsuyoshi Ozawa edited comment on HADOOP-14284 at 4/14/17 9:05 AM:
--
Compilation failure is caused by not refering maven local repository - how can
we install newly added components on test machines? Does anyone know how to do
it?
{quote}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on
project hadoop-auth: Compilation failure: Compilation failure:
[ERROR]
/testptch/hadoop/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/ZKSignerSecretProvider.java:[36,61]
error: package org.apache.hadoop.shaded.com.google.common.annotations does not
exist
{quote]
was (Author: ozawa):
Compilation failure is caused by not refering maven local repository - how can
we install newly added components on test machines? Does anyone know how to do
it?
{quote}]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on
project hadoop-auth: Compilation failure: Compilation failure:
[ERROR]
/testptch/hadoop/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/ZKSignerSecretProvider.java:[36,61]
error: package org.apache.hadoop.shaded.com.google.common.annotations does not
exist
{quote]
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967832#comment-15967832
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
Attaching v10 patch. Summary:
* (from v7) Creating hadoop-shaded-thirdparty which includes shaded Guava.
However, it doesn't include com.google.common.base.Function, Predicate, and
TypeToken. It's because the classes are required to use Apache Curator:
https://cwiki.apache.org/confluence/display/CURATOR/TN13
* (from v7) Using Guava in
hadoop-shaded-thirdparty(org.apache.hadoop.com.google.common) in server-side
code instead of non-shaded Guava directly. This change subsumes what
HADOOP-14238 is addressing.
* (from v10) Shading Guava inside hadoop-client-modules to shade all Guava
unlike hadoop-shaded-thirdparty.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966061#comment-15966061
]
Tsuyoshi Ozawa edited comment on HADOOP-14284 at 4/13/17 4:22 PM:
--
Thanks Akira for trying the patch.
{quote}
The failure occurs because both the two module has shaded guava. Would you
remove shaded guava from hadoop-client-runtime module?
{quote}
Thanks for the suggestion. I could fix it by using shaded Guava in
hadoop-shaded-thirdparty, but I rechecked dependencies again before doing so.
After rechecking the dependency under hadoop-client-modules, I found that
hadoop-client-modules shouldn't use hadoop-shaded-thirdparty. It's because
hadoop-shaded-thirdparty exposes Guava's partial classes to use Apache Curator
but Apache Curator is only necessary in server side. In hadoop-client-modules,
we can shade Guava separately without using hadoop-shaded-thirdparty. I will
upload v10 patch with above approach.
was (Author: ozawa):
Thanks Akira for trying the patch.
{quote}
The failure occurs because both the two module has shaded guava. Would you
remove shaded guava from hadoop-client-runtime module?
{quote}
Thanks for the suggestion. I could fix it by using shaded Guava in
hadoop-shaded-thirdparty, but I checked dependencies again. After rechecking
the dependency under hadoop-client-modules, I found that hadoop-client-modules
shouldn't use hadoop-shaded-thirdparty. It's because hadoop-shaded-thirdparty
exposes Guava's partial classes to use Apache Curator but Apache Curator is
only necessary in server side. In hadoop-client-modules, we can shade Guava
separately without using hadoop-shaded-thirdparty. I will upload v10 patch
which includes above approach.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: HADOOP-14284.010.patch
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.010.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15966061#comment-15966061
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
Thanks Akira for trying the patch.
{quote}
The failure occurs because both the two module has shaded guava. Would you
remove shaded guava from hadoop-client-runtime module?
{quote}
Thanks for the suggestion. I could fix it by using shaded Guava in
hadoop-shaded-thirdparty, but I checked dependencies again. After rechecking
the dependency under hadoop-client-modules, I found that hadoop-client-modules
shouldn't use hadoop-shaded-thirdparty. It's because hadoop-shaded-thirdparty
exposes Guava's partial classes to use Apache Curator but Apache Curator is
only necessary in server side. In hadoop-client-modules, we can shade Guava
separately without using hadoop-shaded-thirdparty. I will upload v10 patch
which includes above approach.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: (was: HADOOP-14284.009.patch)
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: HADOOP-14284.009.patch
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.009.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: (was: HADOOP-14284.008.patch)
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: HADOOP-14284.008.patch
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch, HADOOP-14284.008.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964651#comment-15964651
]
Tsuyoshi Ozawa edited comment on HADOOP-14284 at 4/11/17 5:10 PM:
--
v7 patch includes following changes:
* Created hadoop-shaded-thirdparty which includes shaded guava. It, however,
doesn't include com.google.common.base.Function, Predicate, and TypeToken. It's
because the classes are required to use Apache Curator:
https://cwiki.apache.org/confluence/display/CURATOR/TN13
* Changed namespaces of Guava to use shaded Guava under
org.apache.hadoop.com.google.common. This change subsume what HADOOP-14238 is
addressing.
was (Author: ozawa):
v7 patch includes following changes:
* hadoop-shaded-thirdparty includes shaded guava, but it doesn't include
com.google.common.base.Function, Predicate, and TypeToken. They are required to
use Apache Curator: https://cwiki.apache.org/confluence/display/CURATOR/TN13
* Changed namespaces of Guava to use shaded Guava under
org.apache.hadoop.com.google.common. This change subsume what HADOOP-14238 is
addressing.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964651#comment-15964651
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
v7 patch includes following changes:
* hadoop-shaded-thirdparty includes shaded guava, but it doesn't include
com.google.common.base.Function, Predicate, and TypeToken. They are required to
use Apache Curator: https://cwiki.apache.org/confluence/display/CURATOR/TN13
* Changed namespaces of Guava to use shaded Guava under
org.apache.hadoop.com.google.common. This change subsume what HADOOP-14238 is
addressing.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: HADOOP-14284.007.patch
Some changes in v6 were not intended, so fixing it in v7.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.007.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: (was: HADOOP-14284.005.patch)
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: (was: HADOOP-14284.006.patch)
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: HADOOP-14284.006.patch
Rebasing on trunk.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.005.patch, HADOOP-14284.006.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Status: Patch Available (was: Open)
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.005.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: HADOOP-14284.005.patch
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch, HADOOP-14284.005.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962982#comment-15962982
]
Tsuyoshi Ozawa edited comment on HADOOP-14284 at 4/10/17 3:17 PM:
--
I prefer the way which HBase choose, because it enable us to debug code easily
without uploading jar. Let me know if you have any idea.
was (Author: ozawa):
I prefer to the way which HBase choose, because it enable us to debug code
easily without uploading jar. Let me know if you have any idea.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962982#comment-15962982
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
I prefer to the way which HBase choose, because it enable us to debug code
easily without uploading jar. Let me know if you have any idea.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Status: Open (was: Patch Available)
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962870#comment-15962870
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
Thanks Sean! I've suspected the possibility of not using third-party jar.
Hence, the information of Avro and HBase is helpful for me. Let me check the
documentation for a while.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962737#comment-15962737
]
Tsuyoshi Ozawa edited comment on HADOOP-14284 at 4/10/17 12:45 PM:
---
Thanks Sean for your review.
{quote}
I presume we end up using our project wide LICENSE/NOTICE in this jar?
{quote}
{quote}
What are we trying to exclude here? I don't see much of consequence in the
guava jar.
{quote}
I think we shouldn't include our L files here since we only use original
Guava. Hence, my intension to exclude META-INF/** is to remove our L files
and to include L file which is included in original Guava. Precisely, the L
file is COPYING file in Guava's repository. The change of excluding
META-INF/**, however, was not correct solution as you pointed out - it doesn't
work. Thanks for pointing out. Instead of that, I added dummy
maven-remote-resources-plugin's process-resources section to remove our
original L files.
{quote}
Don't we need to have a dependency reduced pom in order to avoid having Guava
show up as a transitive dependency?
{quote}
I addressed to make the flag true in v4 patch.
was (Author: ozawa):
Thanks Sean for your review.
{quote}
I presume we end up using our project wide LICENSE/NOTICE in this jar?
{quote}
{quote}
What are we trying to exclude here? I don't see much of consequence in the
guava jar.
{quote}
I think we shouldn't our L files here since we only use original Guava.
Hence, my intension is to remove our L files and to include L file which is
included in original Guava. Precisely, the L file is COPYING file in Guava's
repository. The change of excluding META-INF/**, however was my mistake - it
doesn't work. Thanks for pointing out. Instead of that, I added dummy
maven-remote-resources-plugin's process-resources section to remove our
original L files.
{quote}
Don't we need to have a dependency reduced pom in order to avoid having Guava
show up as a transitive dependency?
{quote}
I addressed to make the flag true in v4 patch.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Assignee: Tsuyoshi Ozawa
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962737#comment-15962737
]
Tsuyoshi Ozawa edited comment on HADOOP-14284 at 4/10/17 11:42 AM:
---
Thanks Sean for your review.
{quote}
I presume we end up using our project wide LICENSE/NOTICE in this jar?
{quote}
{quote}
What are we trying to exclude here? I don't see much of consequence in the
guava jar.
{quote}
I think we shouldn't our L files here since we only use original Guava.
Hence, my intension is to remove our L files and to include L file which is
included in original Guava. Precisely, the L file is COPYING file in Guava's
repository. The change of excluding META-INF/**, however was my mistake - it
doesn't work. Thanks for pointing out. Instead of that, I added dummy
maven-remote-resources-plugin's process-resources section to remove our
original L files.
{quote}
Don't we need to have a dependency reduced pom in order to avoid having Guava
show up as a transitive dependency?
{quote}
I addressed to make the flag true in v4 patch.
was (Author: ozawa):
Thanks Seam for your review.
{quote}
I presume we end up using our project wide LICENSE/NOTICE in this jar?
{quote}
{quote}
What are we trying to exclude here? I don't see much of consequence in the
guava jar.
{quote}
I think we shouldn't our L files here since we only use original Guava.
Hence, my intension is to remove our L files and to include L file which is
included in original Guava. Precisely, the L file is COPYING file in Guava's
repository. The change of excluding META-INF/**, however was my mistake - it
doesn't work. Thanks for pointing out. Instead of that, I added dummy
maven-remote-resources-plugin's process-resources section to remove our
original L files.
{quote}
Don't we need to have a dependency reduced pom in order to avoid having Guava
show up as a transitive dependency?
{quote}
I addressed to make the flag true in v4 patch.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962737#comment-15962737
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
Thanks Seam for your review.
{quote}
I presume we end up using our project wide LICENSE/NOTICE in this jar?
{quote}
{quote}
What are we trying to exclude here? I don't see much of consequence in the
guava jar.
{quote}
I think we shouldn't our L files here since we only use original Guava.
Hence, my intension is to remove our L files and to include L file which is
included in original Guava. Precisely, the L file is COPYING file in Guava's
repository. The change of excluding META-INF/**, however was my mistake - it
doesn't work. Thanks for pointing out. Instead of that, I added dummy
maven-remote-resources-plugin's process-resources section to remove our
original L files.
{quote}
Don't we need to have a dependency reduced pom in order to avoid having Guava
show up as a transitive dependency?
{quote}
I addressed to make the flag true in v4 patch.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: HADOOP-14284.004.patch
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: (was: HADOOP-14284.003.patch)
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.004.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Status: Patch Available (was: Open)
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.003.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: HADOOP-14284.003.patch
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch,
> HADOOP-14284.003.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: HADOOP-14284.002.patch
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: (was: HADOOP-14238.001.patch)
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Status: Open (was: Patch Available)
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Status: Patch Available (was: Open)
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Priority: Blocker
> Attachments: HADOOP-14238.001.patch, HADOOP-14238.pre001.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: HADOOP-14238.001.patch
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Priority: Blocker
> Attachments: HADOOP-14238.001.patch, HADOOP-14238.pre001.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960308#comment-15960308
]
Tsuyoshi Ozawa commented on HADOOP-14284:
-
Thanks Andrew for the feedback! Okay, I will attach a patch including your
suggestion, renaming the package name to hadoop-shaded-thirdparty.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated HADOOP-13363: Status: Open (was: Patch Available) > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha2, 3.0.0-alpha1 >Reporter: Allen Wittenauer >Priority: Blocker > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15960248#comment-15960248 ] Tsuyoshi Ozawa commented on HADOOP-13363: - I will be back here after HADOOP-14284. > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer >Priority: Blocker > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15959154#comment-15959154
]
Tsuyoshi Ozawa edited comment on HADOOP-14284 at 4/6/17 3:53 PM:
-
This is a WIP patch which includes lots redundant parts and useless changes.
However, it helps to clarify an approach I'm trying, so attached it here. Based
on discussion on HADOOP-13363 with Stack, I'm trying to add
hadoop-shaded-3rdparty-libraries.
was (Author: ozawa):
WIP patch. Based on discussion on HADOOP-13363 with stack, I'm trying to add
hadoop-shaded-3rdparty-libraries.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14284) Shade Guava everywhere
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa updated HADOOP-14284:
Attachment: HADOOP-14238.pre001.patch
WIP patch. Based on discussion on HADOOP-13363 with stack, I'm trying to add
hadoop-shaded-3rdparty-libraries.
> Shade Guava everywhere
> --
>
> Key: HADOOP-14284
> URL: https://issues.apache.org/jira/browse/HADOOP-14284
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha3
>Reporter: Andrew Wang
>Priority: Blocker
> Attachments: HADOOP-14238.pre001.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts.
> Unfortunately, these projects also consume our private artifacts like
> {{hadoop-hdfs}}. They also are unlikely on the new shaded client introduced
> by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams.
> This isn't a requirement for all dependency upgrades, but it's necessary for
> known-bad dependencies like Guava.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958507#comment-15958507 ] Tsuyoshi Ozawa commented on HADOOP-13363: - Make sense, thanks for clarification. It sounds awful. Let me try to create shaded-3rdparty-libs. I will try to shade both Guava and protobuf there. > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer >Priority: Blocker > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958449#comment-15958449 ] Tsuyoshi Ozawa commented on HADOOP-13363: - [~stack] Thanks for your comment. After playing with Guava's problem for a while, I'm thinking that it's enough to shade protobuf/guava in hadoop-common because other components depend on common. What do you think? I think this workaround works well. > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer >Priority: Blocker > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958340#comment-15958340 ] Tsuyoshi Ozawa edited comment on HADOOP-13363 at 4/6/17 6:08 AM: - Maybe I have not enough time to finish this until May 15th, to finish Guava related works until May 15th. [~stack] could you turn over this issue? was (Author: ozawa): Maybe I have not enough time to do this, to finish Guava related works until May 15th. [~stack] could you turn over this issue? > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer >Priority: Blocker > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958340#comment-15958340 ] Tsuyoshi Ozawa commented on HADOOP-13363: - Maybe I have not enough time to do this, to finish Guava related works until May 15th. [~stack] could you turn over this issue? > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer >Priority: Blocker > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[
https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15958318#comment-15958318
]
Tsuyoshi Ozawa commented on HADOOP-13363:
-
[~andrew.wang] I thought this should be in alpha3, personally. After discussion
on ML, we confirmed that proto2 syntax is compatible with protobuf spec v2. We
have enough confidence to update protobuf version, I think.
{quote}
Would it make sense having a shaded-3rdparty-libs jar that had relocated
protobuf and other libs we want to shade?
{quote}
It makes sense to me, but IIUC, this is one kind of optimization to reduce jar
size. IMHO, we can make this change later. What do you think?
> Upgrade protobuf from 2.5.0 to something newer
> --
>
> Key: HADOOP-13363
> URL: https://issues.apache.org/jira/browse/HADOOP-13363
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2
>Reporter: Allen Wittenauer
>Priority: Blocker
> Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch,
> HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch
>
>
> Standard protobuf 2.5.0 does not work properly on many platforms. (See, for
> example, https://gist.github.com/BennettSmith/7111094 ). In order for us to
> avoid crazy work arounds in the build environment and the fact that 2.5.0 is
> starting to slowly disappear as a standard install-able package for even
> Linux/x86, we need to either upgrade or self bundle or something else.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14238) [Umbrella] Rechecking Guava's object is not exposed to user-facing API
[ https://issues.apache.org/jira/browse/HADOOP-14238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954415#comment-15954415 ] Tsuyoshi Ozawa commented on HADOOP-14238: - Thanks Andrew for the information. Annotation File Utilities, being included in Checker Framework, can extract Annotations included in classes. Hence, we can use it to check whether classes have IA.Public and LimitedPrivate. Let me try. https://checkerframework.org/annotation-file-utilities/#extract-annotations > [Umbrella] Rechecking Guava's object is not exposed to user-facing API > -- > > Key: HADOOP-14238 > URL: https://issues.apache.org/jira/browse/HADOOP-14238 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Tsuyoshi Ozawa >Priority: Critical > > This is reported by [~hitesh] on HADOOP-10101. > At least, AMRMClient#waitFor takes Guava's Supplier instance as an instance. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15946281#comment-15946281 ] Tsuyoshi Ozawa commented on HADOOP-13363: - I would like to try proto v2 mode on protoc 3.2 by using syntax = "proto2": maybe this is what Stack mentioned as compatibility mode. Let me know if I'm wrong. > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944773#comment-15944773 ] Tsuyoshi Ozawa commented on HADOOP-13363: - Andrew, I've started discussion on ML, here. I summarized opinions by developers discussed on HADOOP-13363 and HDFS-11010. I'm waiting for the opinions by other developers. http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201703.mbox/%3CCAAD07OLiwnCH2%3D1gw79vz9nUWX-VOMy8KtGt2pj8_Ln32oeDnw%40mail.gmail.com%3E Thanks Stack and Allen for sharing the information. The experience and information you shared makes us get motivated to update protobuf's version, I think. > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15944773#comment-15944773 ] Tsuyoshi Ozawa edited comment on HADOOP-13363 at 3/28/17 8:38 AM: -- Andrew, I've started discussion on ML: http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201703.mbox/%3CCAAD07OLiwnCH2%3D1gw79vz9nUWX-VOMy8KtGt2pj8_Ln32oeDnw%40mail.gmail.com%3E I summarized opinions by developers discussed on HADOOP-13363 and HDFS-11010. I'm waiting for the opinions by other developers. Thanks Stack and Allen for sharing the information. The experience and information you shared makes us get motivated to update protobuf's version, I think. was (Author: ozawa): Andrew, I've started discussion on ML, here. I summarized opinions by developers discussed on HADOOP-13363 and HDFS-11010. I'm waiting for the opinions by other developers. http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201703.mbox/%3CCAAD07OLiwnCH2%3D1gw79vz9nUWX-VOMy8KtGt2pj8_Ln32oeDnw%40mail.gmail.com%3E Thanks Stack and Allen for sharing the information. The experience and information you shared makes us get motivated to update protobuf's version, I think. > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-10101) Update guava dependency to the latest version
[ https://issues.apache.org/jira/browse/HADOOP-10101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated HADOOP-10101: Release Note: Guava is updated to version 21.0. In the background of merging this patch into trunk, there is a work, shaded Hadoop client artifacts and minicluster, on HADOOP-11804. hadoop-client has its own Guava which is shaded, so we can update dependency with minimum effect compare to previous HADOOP-11804. See also HADOOP-14238 as related problem. > Update guava dependency to the latest version > - > > Key: HADOOP-10101 > URL: https://issues.apache.org/jira/browse/HADOOP-10101 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.0.0-alpha2 >Reporter: Rakesh R >Assignee: Tsuyoshi Ozawa > Labels: BB2015-05-TBR > Fix For: 3.0.0-alpha3 > > Attachments: HADOOP-10101-002.patch, HADOOP-10101-004.patch, > HADOOP-10101-005.patch, HADOOP-10101-006.patch, HADOOP-10101-007.patch, > HADOOP-10101-008.patch, HADOOP-10101-009.patch, HADOOP-10101-009.patch, > HADOOP-10101-010.patch, HADOOP-10101-010.patch, HADOOP-10101-011.patch, > HADOOP-10101.012.patch, HADOOP-10101.013.patch, HADOOP-10101.014.patch, > HADOOP-10101.015.patch, HADOOP-10101.016.patch, HADOOP-10101.017.patch, > HADOOP-10101.018.patch, HADOOP-10101.patch, HADOOP-10101.patch > > > The existing guava version is 11.0.2 which is quite old. This issue tries to > update the version to as latest version as possible. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943537#comment-15943537 ] Tsuyoshi Ozawa commented on HADOOP-13363: - protoc fails to generate files? Let me survey for a while. > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated HADOOP-13363: Attachment: HADOOP-13363.005.patch Trying to fix a problem by lack of execution permission. > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch, HADOOP-13363.005.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15943073#comment-15943073 ] Tsuyoshi Ozawa commented on HADOOP-13363: - Thanks Allen! Updated. @stack unfortunately, not yet. I've also checked discussion on HDFS-11010. Based on the discussion on the jira and here, I would like to start next argument. > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-13363) Upgrade protobuf from 2.5.0 to something newer
[ https://issues.apache.org/jira/browse/HADOOP-13363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsuyoshi Ozawa updated HADOOP-13363: Attachment: HADOOP-13363.004.patch Attaching a patch to update Dockerfile to download protobuf 3.2.0. > Upgrade protobuf from 2.5.0 to something newer > -- > > Key: HADOOP-13363 > URL: https://issues.apache.org/jira/browse/HADOOP-13363 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2 >Reporter: Allen Wittenauer > Attachments: HADOOP-13363.001.patch, HADOOP-13363.002.patch, > HADOOP-13363.003.patch, HADOOP-13363.004.patch > > > Standard protobuf 2.5.0 does not work properly on many platforms. (See, for > example, https://gist.github.com/BennettSmith/7111094 ). In order for us to > avoid crazy work arounds in the build environment and the fact that 2.5.0 is > starting to slowly disappear as a standard install-able package for even > Linux/x86, we need to either upgrade or self bundle or something else. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
