[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez and move to guava-18
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17069429#comment-17069429 ] László Bodor commented on TEZ-2164: --- [~euigeun_chung]: thanks for looking into this, I think guava shading is not a priority anymore for tez after TEZ-4101, TEZ-4124 > Shade the guava version used by Tez and move to guava-18 > > > Key: TEZ-2164 > URL: https://issues.apache.org/jira/browse/TEZ-2164 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Hitesh Shah >Priority: Blocker > Attachments: TEZ-2164.3.patch, TEZ-2164.4.patch, > TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch > > > Should allow us to upgrade to a newer version without shipping a guava > dependency. > Would be good to do this in 0.7 so that we stop shipping guava as early as > possible. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez and move to guava-18
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17069212#comment-17069212 ] Eugene Chung commented on TEZ-2164: --- Recently, I have successfully done shading guava of hive-exec.jar. https://issues.apache.org/jira/browse/HIVE-22126 Maybe I could resolve this issue. > Shade the guava version used by Tez and move to guava-18 > > > Key: TEZ-2164 > URL: https://issues.apache.org/jira/browse/TEZ-2164 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Hitesh Shah >Priority: Blocker > Attachments: TEZ-2164.3.patch, TEZ-2164.4.patch, > TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch > > > Should allow us to upgrade to a newer version without shipping a guava > dependency. > Would be good to do this in 0.7 so that we stop shipping guava as early as > possible. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez and move to guava-18
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17005985#comment-17005985 ] Jianguo Tian commented on TEZ-2164: --- [~hitesh] [~sseth] Hi, is there any progress about this issue? or some solutions to workaround? We now use Hadoop 3.2.1，and we just want to run Hive on Tez in this environment. Looking forward to your suggestions, thx. > Shade the guava version used by Tez and move to guava-18 > > > Key: TEZ-2164 > URL: https://issues.apache.org/jira/browse/TEZ-2164 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Hitesh Shah >Priority: Blocker > Attachments: TEZ-2164.3.patch, TEZ-2164.4.patch, > TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch > > > Should allow us to upgrade to a newer version without shipping a guava > dependency. > Would be good to do this in 0.7 so that we stop shipping guava as early as > possible. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez and move to guava-18
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16849772#comment-16849772 ] Ahmed Hussein commented on TEZ-2164: Hadoop upgraded guava to 27.0-jre ([HADOOP-16210|https://issues.apache.org/jira/browse/HADOOP-16210]). TEZ running 11.0.2 fails with runtime exceptions [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.136 s <<< FAILURE! - in org.apache.tez.dag.app.TestSpeculation [ERROR] org.apache.tez.dag.app.TestSpeculation Time elapsed: 0.136 s <<< ERROR! java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V at org.apache.tez.dag.app.TestSpeculation.setupSpeculation(TestSpeculation.java:86) > Shade the guava version used by Tez and move to guava-18 > > > Key: TEZ-2164 > URL: https://issues.apache.org/jira/browse/TEZ-2164 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Hitesh Shah >Priority: Blocker > Attachments: TEZ-2164.3.patch, TEZ-2164.4.patch, > TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch > > > Should allow us to upgrade to a newer version without shipping a guava > dependency. > Would be good to do this in 0.7 so that we stop shipping guava as early as > possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez and move to guava-18
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16654693#comment-16654693 ] Rajiv commented on TEZ-2164: Hi Due to existing CVE in current guava version ([https://www.cvedetails.com/cve/CVE-2018-10237/)] we would like to switch to latest guava dependency if possible. Changing dependency does allows us to compile 0.9.1 code successfully however we see failure in unit tests because of known issue in hadoop Minicluster api. For now we are using the shipped guava version 11.0.2 but it would be helpful if this issue got resolved. > Shade the guava version used by Tez and move to guava-18 > > > Key: TEZ-2164 > URL: https://issues.apache.org/jira/browse/TEZ-2164 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Hitesh Shah >Priority: Blocker > Attachments: TEZ-2164.3.patch, TEZ-2164.4.patch, > TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch > > > Should allow us to upgrade to a newer version without shipping a guava > dependency. > Would be good to do this in 0.7 so that we stop shipping guava as early as > possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez and move to guava-18
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15281961#comment-15281961 ] Jonathan Eagles commented on TEZ-2164: -- FYI. Removed the stopwatch dependency as part of TEZ-3256 to buy ourselves time to correctly work out shading guava. > Shade the guava version used by Tez and move to guava-18 > > > Key: TEZ-2164 > URL: https://issues.apache.org/jira/browse/TEZ-2164 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Hitesh Shah >Priority: Critical > Attachments: TEZ-2164.3.patch, TEZ-2164.4.patch, > TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch > > > Should allow us to upgrade to a newer version without shipping a guava > dependency. > Would be good to do this in 0.7 so that we stop shipping guava as early as > possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez and move to guava-18
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101991#comment-15101991 ] Hitesh Shah commented on TEZ-2164: -- The problem is due to module dependencies. If all tez modules are combined into a single tez-assembly.jar at the very end, then relocation would work. However, if tez-api is relocated and tez-dag say depends on tez-api, this will create problems if any internal functions in tez-api use guava objects ( and tez-dag tries to invoke such functions ) as tez-dag will be referring to vanilla guava packages ( compile happens before relocation ) whereas tez-api will only know about the relocated package location. > Shade the guava version used by Tez and move to guava-18 > > > Key: TEZ-2164 > URL: https://issues.apache.org/jira/browse/TEZ-2164 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Hitesh Shah >Priority: Critical > Attachments: TEZ-2164.3.patch, TEZ-2164.4.patch, > TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch > > > Should allow us to upgrade to a newer version without shipping a guava > dependency. > Would be good to do this in 0.7 so that we stop shipping guava as early as > possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez and move to guava-18
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101997#comment-15101997 ] Hitesh Shah commented on TEZ-2164: -- Also, both curator and YARN ( yarn depends on curator ) expose public APIs that use guava objects so there are still a bunch of issues that resolving before guava is cleanly separated out/removed. > Shade the guava version used by Tez and move to guava-18 > > > Key: TEZ-2164 > URL: https://issues.apache.org/jira/browse/TEZ-2164 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Hitesh Shah >Priority: Critical > Attachments: TEZ-2164.3.patch, TEZ-2164.4.patch, > TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch > > > Should allow us to upgrade to a newer version without shipping a guava > dependency. > Would be good to do this in 0.7 so that we stop shipping guava as early as > possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez and move to guava-18
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15103010#comment-15103010 ] Rajat Jain commented on TEZ-2164: - Thanks for the explanation. > Shade the guava version used by Tez and move to guava-18 > > > Key: TEZ-2164 > URL: https://issues.apache.org/jira/browse/TEZ-2164 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Hitesh Shah >Priority: Critical > Attachments: TEZ-2164.3.patch, TEZ-2164.4.patch, > TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch > > > Should allow us to upgrade to a newer version without shipping a guava > dependency. > Would be good to do this in 0.7 so that we stop shipping guava as early as > possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez and move to guava-18
[
https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101216#comment-15101216
]
Rajat Jain commented on TEZ-2164:
-
Just for my understanding. Why can't we just apply {{relocation}} using
{{maven-shade-plugin}} in all the sub-folders? What is the need of adding
another module here?
> Shade the guava version used by Tez and move to guava-18
>
>
> Key: TEZ-2164
> URL: https://issues.apache.org/jira/browse/TEZ-2164
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Hitesh Shah
>Priority: Critical
> Attachments: TEZ-2164.3.patch, TEZ-2164.4.patch,
> TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch
>
>
> Should allow us to upgrade to a newer version without shipping a guava
> dependency.
> Would be good to do this in 0.7 so that we stop shipping guava as early as
> possible.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez and move to guava-18
[
https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15090047#comment-15090047
]
TezQA commented on TEZ-2164:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12755022/TEZ-2164.4.patch
against master revision 85637c6.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1418//console
This message is automatically generated.
> Shade the guava version used by Tez and move to guava-18
>
>
> Key: TEZ-2164
> URL: https://issues.apache.org/jira/browse/TEZ-2164
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Hitesh Shah
>Priority: Critical
> Attachments: TEZ-2164.3.patch, TEZ-2164.4.patch,
> TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch
>
>
> Should allow us to upgrade to a newer version without shipping a guava
> dependency.
> Would be good to do this in 0.7 so that we stop shipping guava as early as
> possible.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez and move to guava-18
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14739344#comment-14739344 ] Siddharth Seth commented on TEZ-2164: - The one time publish sounds good. I don't think we want to make the default build a 2 step process. > Shade the guava version used by Tez and move to guava-18 > > > Key: TEZ-2164 > URL: https://issues.apache.org/jira/browse/TEZ-2164 > Project: Apache Tez > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Hitesh Shah >Priority: Critical > Attachments: TEZ-2164.3.patch, TEZ-2164.4.patch, > TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch > > > Should allow us to upgrade to a newer version without shipping a guava > dependency. > Would be good to do this in 0.7 so that we stop shipping guava as early as > possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez and move to guava-18
[
https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737893#comment-14737893
]
TezQA commented on TEZ-2164:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12755022/TEZ-2164.4.patch
against master revision 00508f8.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 89 new
or modified test files.
{color:red}-1 javac{color:red}. The patch appears to cause the build to
fail.
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1102//console
This message is automatically generated.
> Shade the guava version used by Tez and move to guava-18
>
>
> Key: TEZ-2164
> URL: https://issues.apache.org/jira/browse/TEZ-2164
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Hitesh Shah
>Priority: Critical
> Attachments: TEZ-2164.3.patch, TEZ-2164.4.patch,
> TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch
>
>
> Should allow us to upgrade to a newer version without shipping a guava
> dependency.
> Would be good to do this in 0.7 so that we stop shipping guava as early as
> possible.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705422#comment-14705422 ] Hitesh Shah commented on TEZ-2164: -- Thanks for the review [~rajesh.balamohan]. The unintentional changes are due to the handling of core.autocrlf in git. It seems the general good practice is to have is to have a newline at the end of files. Will file a follow up jira for the import re-orders. [~sseth] Any suggestions on the build issue for guava-tez? Should there be a one-off build to publish/deploy the guava-tez shaded jar before committing this patch? Shade the guava version used by Tez --- Key: TEZ-2164 URL: https://issues.apache.org/jira/browse/TEZ-2164 Project: Apache Tez Issue Type: Improvement Reporter: Siddharth Seth Assignee: Hitesh Shah Priority: Critical Attachments: TEZ-2164.3.patch, TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch Should allow us to upgrade to a newer version without shipping a guava dependency. Would be good to do this in 0.7 so that we stop shipping guava as early as possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705742#comment-14705742 ] Hitesh Shah commented on TEZ-2164: -- [~fs111] [~cchepelov] Any comments on this approach? Shade the guava version used by Tez --- Key: TEZ-2164 URL: https://issues.apache.org/jira/browse/TEZ-2164 Project: Apache Tez Issue Type: Improvement Reporter: Siddharth Seth Assignee: Hitesh Shah Priority: Critical Attachments: TEZ-2164.3.patch, TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch Should allow us to upgrade to a newer version without shipping a guava dependency. Would be good to do this in 0.7 so that we stop shipping guava as early as possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705740#comment-14705740 ] Hitesh Shah commented on TEZ-2164: -- bq. Should guava-tez reside in tez-tools, or some such sub-module. Kept it separate as it is not meant to be built each time and therefore outside of the main build tree. Shade the guava version used by Tez --- Key: TEZ-2164 URL: https://issues.apache.org/jira/browse/TEZ-2164 Project: Apache Tez Issue Type: Improvement Reporter: Siddharth Seth Assignee: Hitesh Shah Priority: Critical Attachments: TEZ-2164.3.patch, TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch Should allow us to upgrade to a newer version without shipping a guava dependency. Would be good to do this in 0.7 so that we stop shipping guava as early as possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705642#comment-14705642 ] Siddharth Seth commented on TEZ-2164: - If we're taking this approach, I think we'll have to publish the guava-tez jar into a repository. Changing the build step to first compile guava-tez and then run the mvn install command would be a terrible experience. This also means any project which depends on Tez will end up seeing two versions of Guava classes - which can lead to accidental usage of the tez version. I'm not sure about this, but we may be able to continue depending on guava, and set the dependency to optional - so that downstream components do not automatically get the dependency. Don't think it's possible to set this for guava-tez though. Should guava-tez reside in tez-tools, or some such sub-package. Shade the guava version used by Tez --- Key: TEZ-2164 URL: https://issues.apache.org/jira/browse/TEZ-2164 Project: Apache Tez Issue Type: Improvement Reporter: Siddharth Seth Assignee: Hitesh Shah Priority: Critical Attachments: TEZ-2164.3.patch, TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch Should allow us to upgrade to a newer version without shipping a guava dependency. Would be good to do this in 0.7 so that we stop shipping guava as early as possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez
[
https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14701884#comment-14701884
]
Siddharth Seth commented on TEZ-2164:
-
{code}
[ERROR] Failed to execute goal on project tez-api: Could not resolve
dependencies for project org.apache.tez:tez-api:jar:0.8.0-SNAPSHOT: Failure to
find org.apache.tez:guava-tez:jar:18.0 in
https://repository.apache.org/content/repositories/snapshots was cached in the
local repository, resolution will not be reattempted until the update interval
of apache.snapshots.https has elapsed or updates are forced - [Help 1]
{code}
Including guava-tez in the modules set in the top level pom gets further, but
then fails with
{code}
[INFO] tez-job-analyzer .. SUCCESS [0.194s]
[INFO] tez-dist .. FAILURE [0.072s]
[INFO] Tez ... SKIPPED
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 39.757s
[INFO] Finished at: Tue Aug 18 12:54:32 PDT 2015
[INFO] Final Memory: 81M/480M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-assembly-plugin:2.4:single (package-tez) on
project tez-dist: Failed to create assembly: Error creating assembly archive
tez-dist: You must set at least one file. - [Help 1]
[ERROR]
{code}
One question. Does the shade plugin allow usage of the original package names
in code, and have the shading done post compile ? Otherwise, there'll be two
options of each guava class - and we'll have to monitor each patch to avoid
this.
Shade the guava version used by Tez
---
Key: TEZ-2164
URL: https://issues.apache.org/jira/browse/TEZ-2164
Project: Apache Tez
Issue Type: Improvement
Reporter: Siddharth Seth
Assignee: Hitesh Shah
Priority: Critical
Attachments: TEZ-2164.3.patch, TEZ-2164.wip.2.patch,
allow-guava-16.0.1.patch
Should allow us to upgrade to a newer version without shipping a guava
dependency.
Would be good to do this in 0.7 so that we stop shipping guava as early as
possible.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702181#comment-14702181 ] Rajesh Balamohan commented on TEZ-2164: --- - Built (guava-tez-18.jar got added to $TEZ_HOME/lib/) and tested on multi-node cluster setup. And ran couple of jobs (hive workload). Works fine without issues. - Possibly can remove unintentional changes to ConcatenatedMergedKeyValueInput DAGEventStartDag, DAGRecoveredEvent, EdgeManagerForTest, HistoryACLPolicyException, InitialMemoryAllocator, KVDataGen, OnStateChangedCallback, MultiStageMRConfToTezTranslator, Output, StateMachineTez, SVGUtils, TaskEventScheduleTask, TestHistoryEventTimelineConversion, TestShuffleInputEventHandlerOrderedGrouped, TezBodyDeferringAsyncHandler - Imports need to be rearranged (e.g ExternalSorter, InputIntializerEvent, MROutput etC). or can be deferred for later Shade the guava version used by Tez --- Key: TEZ-2164 URL: https://issues.apache.org/jira/browse/TEZ-2164 Project: Apache Tez Issue Type: Improvement Reporter: Siddharth Seth Assignee: Hitesh Shah Priority: Critical Attachments: TEZ-2164.3.patch, TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch Should allow us to upgrade to a newer version without shipping a guava dependency. Would be good to do this in 0.7 so that we stop shipping guava as early as possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702036#comment-14702036 ] Hitesh Shah commented on TEZ-2164: -- You will need to compile guava-tez first - it is not part of the top-level module list. bq. Does the shade plugin allow usage of the original package names in code, and have the shading done post compile ? I tried that approach but was not successful due to a set of reasons: - the relocation happens on jar creation - unit tests in other modules when referencing internal apis using guava breaks as they use normal guava packages and not the relocated ones - the only seamless way to do this is create a fat jar in tez-dist assembly with all guava relocated. However this still does not solve the case where tez wants to use a newer guava version. And yes, we will need to monitor each patch to see that com.google does not creep in - which is a possibility given that we cannot remove guava-11 as a compile time dependency ( caused by hadoop yarn using guava objects in its apis ) Shade the guava version used by Tez --- Key: TEZ-2164 URL: https://issues.apache.org/jira/browse/TEZ-2164 Project: Apache Tez Issue Type: Improvement Reporter: Siddharth Seth Assignee: Hitesh Shah Priority: Critical Attachments: TEZ-2164.3.patch, TEZ-2164.wip.2.patch, allow-guava-16.0.1.patch Should allow us to upgrade to a newer version without shipping a guava dependency. Would be good to do this in 0.7 so that we stop shipping guava as early as possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez
[
https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14700588#comment-14700588
]
TezQA commented on TEZ-2164:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12750920/TEZ-2164.3.patch
against master revision 6cb8206.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 71 new
or modified test files.
{color:red}-1 javac{color:red}. The patch appears to cause the build to
fail.
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1002//console
This message is automatically generated.
Shade the guava version used by Tez
---
Key: TEZ-2164
URL: https://issues.apache.org/jira/browse/TEZ-2164
Project: Apache Tez
Issue Type: Improvement
Reporter: Siddharth Seth
Assignee: Hitesh Shah
Priority: Critical
Attachments: TEZ-2164.3.patch, TEZ-2164.wip.2.patch,
allow-guava-16.0.1.patch
Should allow us to upgrade to a newer version without shipping a guava
dependency.
Would be good to do this in 0.7 so that we stop shipping guava as early as
possible.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14517395#comment-14517395 ] Hitesh Shah commented on TEZ-2164: -- Moving to 0.8.0 as this work may result in some level of incompatibilities. Shade the guava version used by Tez --- Key: TEZ-2164 URL: https://issues.apache.org/jira/browse/TEZ-2164 Project: Apache Tez Issue Type: Improvement Reporter: Siddharth Seth Priority: Critical Attachments: allow-guava-16.0.1.patch Should allow us to upgrade to a newer version without shipping a guava dependency. Would be good to do this in 0.7 so that we stop shipping guava as early as possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2164) Shade the guava version used by Tez
[ https://issues.apache.org/jira/browse/TEZ-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14511480#comment-14511480 ] Hitesh Shah commented on TEZ-2164: -- Deleting patch. Some issues left to be ironed out. Shade the guava version used by Tez --- Key: TEZ-2164 URL: https://issues.apache.org/jira/browse/TEZ-2164 Project: Apache Tez Issue Type: Improvement Reporter: Siddharth Seth Priority: Critical Attachments: allow-guava-16.0.1.patch Should allow us to upgrade to a newer version without shipping a guava dependency. Would be good to do this in 0.7 so that we stop shipping guava as early as possible. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
