Hi folks,
I believe we need prevent future commits from re-introducing Guava classes
This can be achieved by two options:
1. We add rules on the go. Each time we add an illegal import to the
class that has been replaced. This won't prevent developers from committing
code that uses Guava
[
https://issues.apache.org/jira/browse/HADOOP-17102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ahmed Hussein reopened HADOOP-17102:
Lets see if we can add checkstyle so that no one would import any further Guava
base
Ahmed Hussein created HADOOP-17111:
--
Summary: Replace Guava Optional with Java8+ Optional
Key: HADOOP-17111
URL: https://issues.apache.org/jira/browse/HADOOP-17111
Project: Hadoop Common
Ahmed Hussein created HADOOP-17110:
--
Summary: Replace Guava Preconditions to avoid Guava dependency
Key: HADOOP-17110
URL: https://issues.apache.org/jira/browse/HADOOP-17110
Project: Hadoop Common
Ahmed Hussein created HADOOP-17109:
--
Summary: Replace Guava base64Url and base64 with Java8+ base64
Key: HADOOP-17109
URL: https://issues.apache.org/jira/browse/HADOOP-17109
Project: Hadoop Common
Ahmed Hussein created HADOOP-17108:
--
Summary: Create Classes to wrap Guava code replacement
Key: HADOOP-17108
URL: https://issues.apache.org/jira/browse/HADOOP-17108
Project: Hadoop Common
Ahmed Hussein created HADOOP-17106:
--
Summary: Replace Guava Joiner with Java8 String Join
Key: HADOOP-17106
URL: https://issues.apache.org/jira/browse/HADOOP-17106
Project: Hadoop Common
Ahmed Hussein created HADOOP-17104:
--
Summary: Replace Guava Supplier with Java8+ Supplier in hdfs
Key: HADOOP-17104
URL: https://issues.apache.org/jira/browse/HADOOP-17104
Project: Hadoop Common
Ahmed Hussein created HADOOP-17103:
--
Summary: Replace Guava Supplier with Java8+ Supplier in MAPREDUCE
Key: HADOOP-17103
URL: https://issues.apache.org/jira/browse/HADOOP-17103
Project: Hadoop Common
merged with its
relevant subtak.
> Add checkstyle rule to prevent further usage of Guava classes
> -
>
> Key: HADOOP-17102
> URL: https://issues.apache.org/jira/browse/HADOOP-17102
>
Ahmed Hussein created HADOOP-17102:
--
Summary: Add checkstyle rule to prevent further usage of Guava
classes
Key: HADOOP-17102
URL: https://issues.apache.org/jira/browse/HADOOP-17102
Project: Hadoop
Ahmed Hussein created HADOOP-17101:
--
Summary: Replace Guava Function with Java8+ Function
Key: HADOOP-17101
URL: https://issues.apache.org/jira/browse/HADOOP-17101
Project: Hadoop Common
Ahmed Hussein created HADOOP-17100:
--
Summary: Replace Guava Supplier with Java8+ Supplier
Key: HADOOP-17100
URL: https://issues.apache.org/jira/browse/HADOOP-17100
Project: Hadoop Common
Ahmed Hussein created HADOOP-17099:
--
Summary: Replace Guava Predicate with Java8+ Predicate
Key: HADOOP-17099
URL: https://issues.apache.org/jira/browse/HADOOP-17099
Project: Hadoop Common
Ahmed Hussein created HADOOP-17098:
--
Summary: Reduce Guava dependency in Hadoop source code
Key: HADOOP-17098
URL: https://issues.apache.org/jira/browse/HADOOP-17098
Project: Hadoop Common
org/jira/browse/HADOOP-16924> where we shade and
then update guava. There is more work inside Hadoop to change references to
the shaded guava classpath, but it'll save you more time later.
On Tue, Jun 23, 2020 at 9:09 AM Ahmed Hussein wrote:
> Hi folks,
>
> I was looking into upgrading g
Ahmed Hussein created HADOOP-17083:
--
Summary: Update guava to 27.0-jre in hadoop branch-2.10
Key: HADOOP-17083
URL: https://issues.apache.org/jira/browse/HADOOP-17083
Project: Hadoop Common
Hi folks,
I was looking into upgrading guava to 27.0-jre on branch-2.10 in order to
address the vulnerabilities reported as CVE-2018-10237
<https://nvd.nist.gov/vuln/detail/CVE-2018-10237>.
Since there are concerns using Java8, the plan is to stick to JDK7.
Obviously, it is ex
+1
Thanks for initiating this Weichiu.
-Dinesh
On Sat, Apr 4, 2020 at 3:13 PM Wei-Chiu Chuang wrote:
> Hi Hadoop devs,
>
> I spent a good part of the past 7 months working with a dozen of colleagues
> to update the guava version in Cloudera's software (that includes Hadoop,
&g
of
colleagues
to update the guava version in Cloudera's software (that includes Hadoop,
HBase, Spark, Hive, Cloudera Manager ... more than 20+ projects)
After 7 months, I finally came to a conclusion: Update to Hadoop 3.3 /
3.2.1 / 3.1.3, even if you just go from Hadoop 3.0/ 3.1.0 is going to be
really hard
with a dozen of
> colleagues
> > to update the guava version in Cloudera's software (that includes Hadoop,
> > HBase, Spark, Hive, Cloudera Manager ... more than 20+ projects)
> >
> > After 7 months, I finally came to a conclusion: Update to Hadoop 3.3 /
> > 3.2.1 / 3.1.3
+1
-Ayush
> On 05-Apr-2020, at 12:43 AM, Wei-Chiu Chuang wrote:
>
> Hi Hadoop devs,
>
> I spent a good part of the past 7 months working with a dozen of colleagues
> to update the guava version in Cloudera's software (that includes Hadoop,
> HBase, Spark, Hive, Cloudera
+1
Masatake Iwasaki
On 2020/04/06 10:32, Akira Ajisaka wrote:
+1
Thanks,
Akira
On Sun, Apr 5, 2020 at 4:13 AM Wei-Chiu Chuang wrote:
Hi Hadoop devs,
I spent a good part of the past 7 months working with a dozen of colleagues
to update the guava version in Cloudera's software
+1
Thanks,
Akira
On Sun, Apr 5, 2020 at 4:13 AM Wei-Chiu Chuang wrote:
> Hi Hadoop devs,
>
> I spent a good part of the past 7 months working with a dozen of colleagues
> to update the guava version in Cloudera's software (that includes Hadoop,
> HBase, Spark, Hive, Cloudera M
Great question!
I can run Java API Compliance Checker to detect any API changes. Guess
that's the only one to find out.
On Sat, Apr 4, 2020 at 1:19 PM Igor Dvorzhak wrote:
> How this proposal will impact public APIs? I.e does Hadoop expose any
> Guava classes in the clien
How this proposal will impact public APIs? I.e does Hadoop expose any Guava
classes in the client APIs that will require recompiling all client
applications because they need to use shaded Guava classes?
On Sat, Apr 4, 2020 at 12:13 PM Wei-Chiu Chuang wrote:
> Hi Hadoop devs,
>
> I spe
Hi Hadoop devs,
I spent a good part of the past 7 months working with a dozen of colleagues
to update the guava version in Cloudera's software (that includes Hadoop,
HBase, Spark, Hive, Cloudera Manager ... more than 20+ projects)
After 7 months, I finally came to a conclusion: Update to Hadoop
Wei-Chiu Chuang created HADOOP-16924:
Summary: Update guava version to 28.1-jre
Key: HADOOP-16924
URL: https://issues.apache.org/jira/browse/HADOOP-16924
Project: Hadoop Common
Issue
[
https://issues.apache.org/jira/browse/HADOOP-15218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Bokor reopened HADOOP-15218:
---
> Make Hadoop compatible with Guava 2
[
https://issues.apache.org/jira/browse/HADOOP-15218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Bokor resolved HADOOP-15218.
---
Resolution: Duplicate
> Make Hadoop compatible with Guava 2
[
https://issues.apache.org/jira/browse/HADOOP-15960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Bota resolved HADOOP-15960.
-
Resolution: Fixed
All subtasks are resolved, guava updated on branches 3.0, 3.1, 3.2 and trunk
[
https://issues.apache.org/jira/browse/HADOOP-15272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Bota resolved HADOOP-15272.
-
Resolution: Fixed
Fixed in HADOOP-16210.
> Update Guava, see what bre
Gabor Bota created HADOOP-16237:
---
Summary: Fix new findbugs issues after update guava to 27.0-jre in
hadoop-project trunk
Key: HADOOP-16237
URL: https://issues.apache.org/jira/browse/HADOOP-16237
; On Tue, Apr 2, 2019 at 9:54 AM Steve Loughran wrote:
>
> >
> > I know that the number of guava updates we could call painless is 0, but
> > we need to do this.
> >
> > The last time we successfully updated Guava was 2012: h
> > ttps://issues.apache
.
> Correct findbug ignores for unjustified issues during update to guava to
> 27.0-jre in hadoop-project
> ---
>
> Key: HADOOP-16230
> URL: https://issues.apac
I am taking silence as happiness here.
+1 to the patch
On Tue, Apr 2, 2019 at 9:54 AM Steve Loughran wrote:
>
> I know that the number of guava updates we could call painless is 0, but
> we need to do this.
>
> The last time we successfully updated Guava was 2012: h
> ttps:/
Gabor Bota created HADOOP-16230:
---
Summary: Corrcet findbug ignores for unjustified issues during
update to guava to 27.0-jre in hadoop-project
Key: HADOOP-16230
URL: https://issues.apache.org/jira/browse/HADOOP
I know that the number of guava updates we could call painless is 0, but we
need to do this.
The last time we successfully updated Guava was 2012: h
ttps://issues.apache.org/jira/browse/HDFS-3187
That was the java 6 era
The last unsuccessful attempt, April 2017:
https://issues.apache.org/jira
Hi devs,
I'm working on the guava version from 11.0.2 to 27.0-jre in hadoop-project.
We need to do the upgrade because of CVE-2018-10237
<https://nvd.nist.gov/vuln/detail/CVE-2018-10237>.
I've created an issue (HADOOP-15960
<https://issues.apache.org/jira/browse/HADOOP-15960>) to tr
Gabor Bota created HADOOP-16220:
---
Summary: Add findbugs ignores for unjustified issues during update
to guava to 27.0-jre in hadoop-project
Key: HADOOP-16220
URL: https://issues.apache.org/jira/browse/HADOOP-16220
Steve Loughran created HADOOP-16218:
---
Summary: findbugs warning of null param to non-nullable method in
Configuration with Guava update
Key: HADOOP-16218
URL: https://issues.apache.org/jira/browse/HADOOP-16218
Gabor Bota created HADOOP-16213:
---
Summary: Update guava to 27.0-jre in hadoop-project branch-3.1
Key: HADOOP-16213
URL: https://issues.apache.org/jira/browse/HADOOP-16213
Project: Hadoop Common
Gabor Bota created HADOOP-16212:
---
Summary: Update guava to 27.0-jre in hadoop-project branch-3.0
Key: HADOOP-16212
URL: https://issues.apache.org/jira/browse/HADOOP-16212
Project: Hadoop Common
Gabor Bota created HADOOP-16211:
---
Summary: Update guava to 27.0-jre in hadoop-project branch-3.2 and
branch-3.1
Key: HADOOP-16211
URL: https://issues.apache.org/jira/browse/HADOOP-16211
Project: Hadoop
Gabor Bota created HADOOP-16210:
---
Summary: Update guava to 27.0-jre in hadoop-project trunk
Key: HADOOP-16210
URL: https://issues.apache.org/jira/browse/HADOOP-16210
Project: Hadoop Common
Gabor Bota created HADOOP-15960:
---
Summary: Update guava to 27.0-jre in hadoop-common
Key: HADOOP-15960
URL: https://issues.apache.org/jira/browse/HADOOP-15960
Project: Hadoop Common
Issue Type
Steve Loughran created HADOOP-15272:
---
Summary: Update Guava, see what breaks
Key: HADOOP-15272
URL: https://issues.apache.org/jira/browse/HADOOP-15272
Project: Hadoop Common
Issue Type
Igor Dvorzhak created HADOOP-15214:
--
Summary: Make Hadoop compatible with Guava 21.0
Key: HADOOP-15214
URL: https://issues.apache.org/jira/browse/HADOOP-15214
Project: Hadoop Common
Issue
Haibo Chen created HADOOP-14957:
---
Summary: ReconfigurationTaskStatus is exposing guava Optional in
its public api
Key: HADOOP-14957
URL: https://issues.apache.org/jira/browse/HADOOP-14957
Project
Jonathan Eagles created HADOOP-14891:
Summary: Guava 21.0+ libraries not compatible with user jobs
Key: HADOOP-14891
URL: https://issues.apache.org/jira/browse/HADOOP-14891
Project: Hadoop Common
[
https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsuyoshi Ozawa resolved HADOOP-14284.
-
Resolution: Invalid
> Shade Guava everywh
Bharat Viswanadham created HADOOP-14847:
---
Summary: Remove Guava Supplier and change to java Supplier in
AMRMClient and AMRMClientAysnc
Key: HADOOP-14847
URL: https://issues.apache.org/jira/browse/HADOOP
Andrew Wang created HADOOP-14386:
Summary: Make trunk work with Guava 11.0.2 again
Key: HADOOP-14386
URL: https://issues.apache.org/jira/browse/HADOOP-14386
Project: Hadoop Common
Issue Type
Steve Loughran created HADOOP-14380:
---
Summary: Make Guava version Hadoop builds with configurable
Key: HADOOP-14380
URL: https://issues.apache.org/jira/browse/HADOOP-14380
Project: Hadoop Common
Andrew Wang created HADOOP-14284:
Summary: Shade Guava everywhere
Key: HADOOP-14284
URL: https://issues.apache.org/jira/browse/HADOOP-14284
Project: Hadoop Common
Issue Type: Bug
Affects
Haohui Mai created HADOOP-12592:
---
Summary: Remove guava usage in the hdfs-client module
Key: HADOOP-12592
URL: https://issues.apache.org/jira/browse/HADOOP-12592
Project: Hadoop Common
Issue
Walter Su created HADOOP-12475:
--
Summary: Replace guava Cache with ConcurrentHashMap for caching
Connection in ipc Client
Key: HADOOP-12475
URL: https://issues.apache.org/jira/browse/HADOOP-12475
Robert Kanter created HADOOP-11616:
--
Summary: Remove workaround for Curator's ChildReaper requiring
Guava 15+
Key: HADOOP-11616
URL: https://issues.apache.org/jira/browse/HADOOP-11616
Project
Robert Kanter created HADOOP-11612:
--
Summary: Workaround for Curator's ChildReaper requiring Guava 15+
Key: HADOOP-11612
URL: https://issues.apache.org/jira/browse/HADOOP-11612
Project: Hadoop Common
Tsuyoshi OZAWA created HADOOP-11600:
---
Summary: Fix up source codes to be compiled with Guava 18.0
Key: HADOOP-11600
URL: https://issues.apache.org/jira/browse/HADOOP-11600
Project: Hadoop Common
Sangjin Lee created HADOOP-11470:
Summary: eliminate use of incompatible guava APIs from the hadoop
codebase
Key: HADOOP-11470
URL: https://issues.apache.org/jira/browse/HADOOP-11470
Project: Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-11319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved HADOOP-11319.
-
Resolution: Duplicate
Update Guava to 18.0
Key
Tim Robertson created HADOOP-11319:
--
Summary: Update Guava to 18.0
Key: HADOOP-11319
URL: https://issues.apache.org/jira/browse/HADOOP-11319
Project: Hadoop Common
Issue Type: Sub-task
Yes, Guava is a constant pain; there's lots of open JIRAs related to it, as
its the one we can't seamlessly upgrade. Not unless we do our own fork and
reinsert the missing classes.
The most common uses in the code are
@VisibleForTesting (easily replicated)
and the Precondition.check() operations
a prototype
of how #2 would work (I don't think it will be more than 200 lines of code)
On Mon, Nov 10, 2014 at 5:18 AM, Steve Loughran ste...@hortonworks.com
wrote:
Yes, Guava is a constant pain; there's lots of open JIRAs related to it, as
its the one we can't seamlessly upgrade. Not unless we do
the idea of isolating usage of guava that
breaks with guava 16 and later. I assume (but I haven't looked into it)
that it's fairly straightforward to isolate them and fix them. That work
could be done at any time without any version upgrades or impacting users.
On Mon, Nov 10, 2014 at 9:26 AM
I'm usually an advocate for getting rid of unnecessary dependencies
(cough, jetty, cough), but a lot of the things in Guava are really
useful.
Immutable collections, BiMap, Multisets, Arrays#asList, the stuff for
writing hashCode() and equals(), String#Joiner, the list goes on. We
particularly
… has been a constant pain w.r.t compatibility etc.
Should we consider adopting a policy to not use guava in Common/HDFS/YARN?
MR doesn't matter too much since it's application-side issue, it does hurt
end-users though since they still might want a newer guava-version, but at
least they can
Guava did make the lives of Hadoop development easier in many cases -- What
I've been consistently hearing is that the version of Guava used is Hadoop
is so old that it starts to hurt the application developers.
I appreciate the value of Guava -- things like CacheMap are fairly
difficult
As Haohui Mai said, removing the dependency on the Guava may not be a good
idea.
But, instead can we use a fixed guava version in Hadoop which is stable as
of now, with a shaded package structure ?
so that it will not break the application level dependency on another
version of the Guava. Inside
I've a patch for HADOOP-11102 which rolls curator back to v 2.4.1, which
only pulls in Guava 14...hadoop should now be weakly consistent -at least
not strongly inconsistent- in its guava versions.
allowing hadoop to work on 16.x while still remaining compatible with 11.x
is still something
[
https://issues.apache.org/jira/browse/HADOOP-10961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved HADOOP-10961.
-
Resolution: Duplicate
Use of deprecated Google Guava (v17) Stopwatch constructor
The use of an unnecessarily old dependency encourages problems like
HDFS-7040. The current Guava dependency is a big problem for downstream
apps and I'd really like to see it addressed.
On Tue, Sep 23, 2014 at 2:09 PM, Steve Loughran ste...@hortonworks.com
wrote:
I'm using curator elsewhere
+1 to upgrading guava. Irrespective of downstream apps, the hadoop source
tree is now internally inconsistent
On 22 September 2014 17:56, Sangjin Lee sj...@apache.org wrote:
I agree that a more robust solution is to have better classloading
isolation.
Still, IMHO guava (and possibly protobuf
If we've broken compatibility in branch-2, that's a bug that we need to
fix. HADOOP-10868 has not yet made it into a release; I don't see it as a
justification for solidifying the breakage.
-1 to upgrading Guava in branch-2.
On Tue, Sep 23, 2014 at 3:06 AM, Steve Loughran ste...@hortonworks.com
a
while, and (b) it seems silly to reinvent the wheel when Curator already
does all this.
I agree that upgrading Guava will be a compatibility problem though...
On Tue, Sep 23, 2014 at 9:30 AM, Sandy Ryza sandy.r...@cloudera.com wrote:
If we've broken compatibility in branch-2, that's a bug
I agree that a more robust solution is to have better classloading
isolation.
Still, IMHO guava (and possibly protobuf as well) sticks out like a sore
thumb. There are just too many issues in trying to support both guava 11
and guava 16. Independent of what we may do with the classloading
Upgrading Guava version is tricky. While it helps in many cases, it can
break existing applications/deployments. I understand we do not have a
policy for updating dependencies, but still we should be careful with
Guava.
I would be more inclined towards a more permanent solution to this problem
I would also agree on upgrading guava. Yes I am aware of the potential
impact on customers who might rely on hadoop bringing in guava 11. However,
IMHO the balance tipped over to the other side a while ago; i.e. I think
there are far more people using guava 16 in their code and scrambling to
make
I know we've been ignoring the Guava version problem, but HADOOP-10868
added a transitive dependency on Guava 16 by way of Curator 2.6.
Maven currently forces the build to use Guava 11.0.2, but this is hiding at
compile timeall code paths from curator which may use classes methods
that aren't
Steve Loughran created HADOOP-11102:
---
Summary: Hadoop now has transient dependency on Guava 16
Key: HADOOP-11102
URL: https://issues.apache.org/jira/browse/HADOOP-11102
Project: Hadoop Common
Daniel Nydegger created HADOOP-10961:
Summary: Use of deprecated Google Guava (v17) Stopwatch
constructor in Hadoop FileInputFormat causes an exception
Key: HADOOP-10961
URL: https://issues.apache.org/jira
Rakesh R created HADOOP-10101:
-
Summary: Update guava dependency to the latest version 15.0
Key: HADOOP-10101
URL: https://issues.apache.org/jira/browse/HADOOP-10101
Project: Hadoop Common
Issue
+1. guava is the new apache commons, maintained by java experts with
comprehensive test coverage.
I also propose to deprecate any existing utils in hadoop-common with
duplicate functionality.
On Thu, Feb 10, 2011 at 10:25 PM, Jakob Homan jgho...@gmail.com wrote:
+1
On Thu, Feb 10, 2011 at 10
Cool, seems like enough people are on board. I'll just include this in the
patch for HADOOP-7132 (naming the IPC Reader Threads) since that's where I
wanted to use it.
Can't wait to use this stuff in more patches. I love guava :)
-Todd
On Fri, Feb 11, 2011 at 4:09 AM, Luke Lu l...@vicaya.com
On Feb 11, 2011, at 10:01 AM, Todd Lipcon wrote:
Cool, seems like enough people are on board. I'll just include this
in the
patch for HADOOP-7132 (naming the IPC Reader Threads) since that's
where I
wanted to use it.
Can't wait to use this stuff in more patches. I love guava :)
+1
+1
Arun
On Feb 10, 2011, at 7:45 PM, Todd Lipcon wrote:
Anyone mind if I pull in the Guava library as a dependency for
Common? It
has a bunch of very useful utilities - in this particular case the
one I'm
itching to use is ThreadFactoryBuilder:
http://guava-libraries.googlecode.com/svn
+1, Guava is the magical piece that can get rid of all those
UnsupportedEncodingException ...!
Anyone mind if I pull in the Guava library as a dependency for Common? It
has a bunch of very useful utilities - in this particular case the one I'm
itching to use is ThreadFactoryBuilder:
http://guava-libraries.googlecode.com/svn/tags/release05/javadoc/com/google/common/util/concurrent
Actually it seems that Pig uses this already, which perhaps means it
is good enough for us as well ;)
--
Take care,
Konstantin (Cos) Boudnik
On Thu, Feb 10, 2011 at 19:45, Todd Lipcon t...@cloudera.com wrote:
Anyone mind if I pull in the Guava library as a dependency for Common? It
has
:
Anyone mind if I pull in the Guava library as a dependency for Common? It
has a bunch of very useful utilities - in this particular case the one I'm
itching to use is ThreadFactoryBuilder:
http://guava-libraries.googlecode.com/svn/tags/release05/javadoc/com/google/common/util/concurrent
101 - 191 of 191 matches
Mail list logo
