Hi Mike,

This only shows that some other component needs to be updated as well.
Which software relies on Lucene 4.3.0, which was released in 2013?
It is most probably not used by anyone else anymore.

This needs to be solved. It makes no sense to release Flume 1.9 with support 
for ElasticSearch 0.90.1.
And it doesn't make sense either to release Flume 1.9 with support for a 
component that relies on Lucene 4.3.0.

We will always have dependency problems, if one component is updated to a newer 
version and others not.

To get rid of the conflict is probably to set "optional" to true in the pom.xml.
Then we don't distribute those Jars in the flume lib and the users need to set 
their class path to point e.g. to Elasticsearch and then the correct libs will 
be used.
I doubt that a Elasticsearch user will also use some other component with e.g. 
this outdated Lucene.

best regards,

Helmut
 

-----Original Message-----
From: Mike Percy [mailto:mpe...@apache.org] 
Sent: Donnerstag, 25. Jänner 2018 23:01
To: dev@flume.apache.org
Subject: Re: Merging to trunk?

Hi Helmut,
I wrote a small Perl script to identify dependency conflicts based on output 
from mvn dependency:tree and posted it here: https://gist.github.com/
mpercy/39614d770864bdd0c386befd5e8a1840

I ran that on the current trunk and it actually found some errors which should 
be fixed:

Version conflict: package com.google.guava:guava:jar needed in 2 different
versions: (11.0.2, 18.0)
Version conflict: package commons-httpclient:commons-httpclient:jar needed in 2 
different versions: (3.0.1, 3.1) Version conflict: package 
commons-logging:commons-logging:jar needed in 2 different versions: (1.1.3, 
1.2) Version conflict: package commons-pool:commons-pool:jar needed in 2 
different versions: (1.5.4, 1.6) Version conflict: package 
net.sf.jopt-simple:jopt-simple:jar needed in 2 different versions: (3.2, 4.7) 
Version conflict: package org.codehaus.jackson:jackson-jaxrs:jar needed in
2 different versions: (1.8.3, 1.8.8)
ERROR: 6 package version conflicts identified

However, after applying your patch on top of trunk and running it again, it 
identified new conflicts:

Version conflict: package com.google.guava:guava:jar needed in 2 different
versions: (11.0.2, 18.0)
Version conflict: package commons-httpclient:commons-httpclient:jar needed in 2 
different versions: (3.0.1, 3.1) Version conflict: package 
commons-logging:commons-logging:jar needed in 2 different versions: (1.1.3, 
1.2) Version conflict: package commons-pool:commons-pool:jar needed in 2 
different versions: (1.5.4, 1.6) Version conflict: package 
net.sf.jopt-simple:jopt-simple:jar needed in 3 different versions: (3.2, 4.7, 
5.0.2) Version conflict: package org.apache.lucene:lucene-analyzers-common:jar
needed in 2 different versions: (4.3.0, 7.1.0) Version conflict: package 
org.apache.lucene:lucene-core:jar needed in 2 different versions: (4.3.0, 
7.1.0) Version conflict: package org.apache.lucene:lucene-grouping:jar needed 
in 2 different versions: (4.3.0, 7.1.0) Version conflict: package 
org.apache.lucene:lucene-highlighter:jar needed in 2 different versions: 
(4.3.0, 7.1.0) Version conflict: package org.apache.lucene:lucene-memory:jar 
needed in 2 different versions: (4.3.0, 7.1.0) Version conflict: package 
org.apache.lucene:lucene-misc:jar needed in 2 different versions: (4.3.0, 
7.1.0) Version conflict: package org.apache.lucene:lucene-queries:jar needed in 
2 different versions: (4.3.0, 7.1.0) Version conflict: package 
org.apache.lucene:lucene-queryparser:jar needed in 2 different versions: 
(4.3.0, 7.1.0) Version conflict: package org.apache.lucene:lucene-spatial:jar 
needed in 2 different versions: (4.3.0, 7.1.0) Version conflict: package 
org.apache.lucene:lucene-suggest:jar needed in 2 different versions: (4.3.0, 
7.1.0) Version conflict: package org.codehaus.jackson:jackson-jaxrs:jar needed 
in
2 different versions: (1.8.3, 1.8.8)
Version conflict: package org.yaml:snakeyaml:jar needed in 2 different
versions: (1.10, 1.17)
ERROR: 17 package version conflicts identified

So while it appears Guava is no longer a concern, there are other conflicting 
dependencies involved in the conflict here.

I'm interested to hear your thoughts on this.

Regards,
Mike

On Wed, Jan 24, 2018 at 7:57 AM, Wahrmann, Helmut <helmut.wahrm...@rsa.com>
wrote:

> Hi Mike,
>
> Thanks for the quick response.
>
> I fully agree that we need to take care about interop between the 
> different components and dependencies.
> But if you look at the patch in FLUME-3021, it only puts in 
> Elasticsearch dependencies.
>
> The issue with Guava, which you saw in FLUME-2921 is no longer there, 
> because since mid-2016 a lot has changed in the Flume trunk and all 
> those dependencies are fixed.
> It won't even be possible gto apply this patch anymore.
>
> The patch in FLUME-3021 works with the latest Elasticsearch versions, 
> without introducing additional dependencies, which might cause 
> problems to other projects.
> So I think it can be safely merged into trunk.
>
> best regards,
>
> Helmut
>
>
> -----Original Message-----
> From: Mike Percy [mailto:mpe...@apache.org]
> Sent: Dienstag, 23. Jänner 2018 20:39
> To: dev@flume.apache.org
> Subject: Re: Merging to trunk?
>
> Hi Helmut,
> Thank you for bringing this up on dev@ and thank you for the patch. I 
> see there are other people people interested in this component upgrade as 
> well.
>
> As you are probably aware, a Flume committer will need to approve the 
> change before I gets merged to trunk.
>
> My primary concern w/ merging this would be compatibility of the 
> dependencies. Flume suffers from a kind of a "dependency hell" because 
> of a lack of classloading support (either via OSGI or Java modules). 
> See
> https://issues.apache.org/jira/browse/FLUME-2293 for a tracking ticket 
> about that issue. What that means is that all of the components that 
> Flume ships must have compatible dependencies with each other which 
> makes changes like this more complex. Therefore I would like someone 
> to verify that mvn dependency:tree does not show conflicts when run 
> from the top level with the new patches. If memory serves, I believe 
> Google Guava is likely to conflict.
>
> Also, I think that https://issues.apache.org/jira/browse/FLUME-3021 is 
> a duplicate of https://issues.apache.org/jira/browse/FLUME-2921 which 
> has additional information about the various issues that we need to 
> solve to do this upgrade.
>
> I'd be happy to discuss this issue some more on this thread.
>
> Mike
>
>
> On Tue, Jan 23, 2018 at 12:57 AM, Wahrmann, Helmut < 
> helmut.wahrm...@rsa.com>
> wrote:
>
> > Hi,
> >
> > Who decides if/when a patch or pull request gets merged to trunk?
> > Reason I am asking is for the Elasticsearch support. The current 
> > code in trunk does not work with ES > 2.x.
> > Currently Elasticsearch is at 6.1.
> >
> > In FLUME-3021 we have several patches since March last year.
> > I have a patched Flume running at a customer since April last year 
> > without any problems.
> >
> > So why not merging those changes into the trunk?
> > As I stated in my comment in FLUME-3021, it will not cause any 
> > problems, cause the current trunk won't work with newer ES versions
> anyhow.
> > I doubt that someone is out there still running ES 0.95.
> >
> > best regards,
> >
> > Helmut
> >
>

Reply via email to