[
https://issues.apache.org/jira/browse/HADOOP-17288?focusedWorklogId=496839&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-496839
]
ASF GitHub Bot logged work on HADOOP-17288:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 07/Oct/20 18:36
Start Date: 07/Oct/20 18:36
Worklog Time Spent: 10m
Work Description: vinayakumarb commented on a change in pull request
#2342:
URL: https://github.com/apache/hadoop/pull/2342#discussion_r501229119
##########
File path: hadoop-common-project/hadoop-auth/pom.xml
##########
@@ -234,6 +235,24 @@
<excludeFilterFile>${basedir}/dev-support/findbugsExcludeFile.xml</excludeFilterFile>
</configuration>
</plugin>
+ <plugin>
+ <groupId>com.google.code.maven-replacer-plugin</groupId>
+ <artifactId>replacer</artifactId>
+ <executions>
+ <execution>
+ <id>replace-sources</id>
+ <configuration>
+ <skip>false</skip>
+ </configuration>
+ </execution>
+ <execution>
+ <id>replace-test-sources</id>
+ <configuration>
+ <skip>false</skip>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
Review comment:
> A few questions so I can better understand the suggested approach:
>
> So source would continue to refer to unshaded guava? (unshaded guava 11?)
Hadoop source uses shaded guava itself from hadoop-thirdparty.
>
> Rather than rewrite once as this PR does, we'd rewrite on every build? How
long does the rewrite take?
>
This replacer is basically to make the Job easy for this PR. Since we
commit the replaced code itself, further replaces wont happen on same file. It
will help to catch further references to unshaded guava. Rewrite doesnot take
much time, it finishes in fraction of second.
> hadoop-yarn-csi is bound to guava-20? Why is that? The grpc used?
Interested in how a guava 20 in classpath won't mess up downstreamers who want
to use something newer (or older).
yes, yarn-csi uses grpc. And grpc refers to guava-20 and it uses protobuf
3.x even before protobuf was upgraded for entire hadoop. It keeps separate
directory (share/hadoop/yarn/csi/lib) for its dependencies and existing scripts
doesnt add this directory into classpath of other processes.
>
> For 4., hopefully we can (later) add an enforcer so non-shaded references
get flagged at build time. Rather than have hadoop depend on something it
doesn't use, hopefully we can work on fixing the curator dependency to fix its
pom?
>
yes, If we could add this enforcer, may be we need not find and replace
unshaded usage of guava on every build.
For curator, curator community is not ready to shade 3 classes of guava as
they are part of their API. These classes seems to be pretty stable across
versions. So curator does work with guava-11 in classpath.
If we want to fix this and completely remove hadoop's dependency on guava,
we may need to shade curator as well in hadoop-thirdparty and use that in
hadoop.
We are fortunate enough that, these dependencies are not part of any exposed
APIs.
> Thanks @vinayakumarb
Thanks @saintstack
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 496839)
Time Spent: 50m (was: 40m)
> Use shaded guava from thirdparty
> --------------------------------
>
> Key: HADOOP-17288
> URL: https://issues.apache.org/jira/browse/HADOOP-17288
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Ayush Saxena
> Assignee: Ayush Saxena
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Use the shaded version of guava in hadoop-thirdparty
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
