Maybe the Maven shade plugin can solve this, we can package each module to a fat jar and they will not depend on other modules.
2018-01-27 6:14 GMT+08:00 Wahrmann, Helmut <helmut.wahrm...@rsa.com>: > fully agreed. > But we shouldn't do that do support a 4 years old component. > The goal should be to support the latest releases. > It is ridiculous to release flume 1.9 with elasticsearch 0.90.1 support, > when 6.x is the current version. > > > > Best regards, > > Helmut Wahrmann > > > -------- Ursprüngliche Nachricht -------- > Von: Ralph Goers <ralph.go...@dslextreme.com> > Datum: 26.01.18 22:12 (GMT+01:00) > An: dev@flume.apache.org > Betreff: Re: Merging to trunk? > > The “right” way to deal with this from a Maven perspective is to declare > the version you want in the dependencyManagement section of the parent pom > and to not specify any versions in child poms. Then use the Maven enforcer > plugin to make sure everything required has a declaration in the > dependencyManagement section. > > Ralph > > > On Jan 26, 2018, at 8:28 AM, Wahrmann, Helmut <helmut.wahrm...@rsa.com> > wrote: > > > > Hi Mike, > > > > This only shows that some other component needs to be updated as well. > > Which software relies on Lucene 4.3.0, which was released in 2013? > > It is most probably not used by anyone else anymore. > > > > This needs to be solved. It makes no sense to release Flume 1.9 with > support for ElasticSearch 0.90.1. > > And it doesn't make sense either to release Flume 1.9 with support for a > component that relies on Lucene 4.3.0. > > > > We will always have dependency problems, if one component is updated to > a newer version and others not. > > > > To get rid of the conflict is probably to set "optional" to true in the > pom.xml. > > Then we don't distribute those Jars in the flume lib and the users need > to set their class path to point e.g. to Elasticsearch and then the correct > libs will be used. > > I doubt that a Elasticsearch user will also use some other component > with e.g. this outdated Lucene. > > > > best regards, > > > > Helmut > > > > > > -----Original Message----- > > From: Mike Percy [mailto:mpe...@apache.org] > > Sent: Donnerstag, 25. Jänner 2018 23:01 > > To: dev@flume.apache.org > > Subject: Re: Merging to trunk? > > > > Hi Helmut, > > I wrote a small Perl script to identify dependency conflicts based on > output from mvn dependency:tree and posted it here: > https://gist.github.com/ > > mpercy/39614d770864bdd0c386befd5e8a1840 > > > > I ran that on the current trunk and it actually found some errors which > should be fixed: > > > > Version conflict: package com.google.guava:guava:jar needed in 2 > different > > versions: (11.0.2, 18.0) > > Version conflict: package commons-httpclient:commons-httpclient:jar > needed in 2 different versions: (3.0.1, 3.1) Version conflict: package > commons-logging:commons-logging:jar needed in 2 different versions: > (1.1.3, 1.2) Version conflict: package commons-pool:commons-pool:jar needed > in 2 different versions: (1.5.4, 1.6) Version conflict: package > net.sf.jopt-simple:jopt-simple:jar needed in 2 different versions: (3.2, > 4.7) Version conflict: package org.codehaus.jackson:jackson-jaxrs:jar > needed in > > 2 different versions: (1.8.3, 1.8.8) > > ERROR: 6 package version conflicts identified > > > > However, after applying your patch on top of trunk and running it again, > it identified new conflicts: > > > > Version conflict: package com.google.guava:guava:jar needed in 2 > different > > versions: (11.0.2, 18.0) > > Version conflict: package commons-httpclient:commons-httpclient:jar > needed in 2 different versions: (3.0.1, 3.1) Version conflict: package > commons-logging:commons-logging:jar needed in 2 different versions: > (1.1.3, 1.2) Version conflict: package commons-pool:commons-pool:jar needed > in 2 different versions: (1.5.4, 1.6) Version conflict: package > net.sf.jopt-simple:jopt-simple:jar needed in 3 different versions: (3.2, > 4.7, 5.0.2) Version conflict: package org.apache.lucene:lucene- > analyzers-common:jar > > needed in 2 different versions: (4.3.0, 7.1.0) Version conflict: package > org.apache.lucene:lucene-core:jar needed in 2 different versions: (4.3.0, > 7.1.0) Version conflict: package org.apache.lucene:lucene-grouping:jar > needed in 2 different versions: (4.3.0, 7.1.0) Version conflict: package > org.apache.lucene:lucene-highlighter:jar needed in 2 different versions: > (4.3.0, 7.1.0) Version conflict: package org.apache.lucene:lucene-memory:jar > needed in 2 different versions: (4.3.0, 7.1.0) Version conflict: package > org.apache.lucene:lucene-misc:jar needed in 2 different versions: (4.3.0, > 7.1.0) Version conflict: package org.apache.lucene:lucene-queries:jar > needed in 2 different versions: (4.3.0, 7.1.0) Version conflict: package > org.apache.lucene:lucene-queryparser:jar needed in 2 different versions: > (4.3.0, 7.1.0) Version conflict: package org.apache.lucene:lucene-spatial:jar > needed in 2 different versions: (4.3.0, 7.1.0) Version conflict: package > org.apache.lucene:lucene-suggest:jar needed in 2 different versions: > (4.3.0, 7.1.0) Version conflict: package > org.codehaus.jackson:jackson-jaxrs:jar > needed in > > 2 different versions: (1.8.3, 1.8.8) > > Version conflict: package org.yaml:snakeyaml:jar needed in 2 different > > versions: (1.10, 1.17) > > ERROR: 17 package version conflicts identified > > > > So while it appears Guava is no longer a concern, there are other > conflicting dependencies involved in the conflict here. > > > > I'm interested to hear your thoughts on this. > > > > Regards, > > Mike > > > > On Wed, Jan 24, 2018 at 7:57 AM, Wahrmann, Helmut < > helmut.wahrm...@rsa.com> > > wrote: > > > >> Hi Mike, > >> > >> Thanks for the quick response. > >> > >> I fully agree that we need to take care about interop between the > >> different components and dependencies. > >> But if you look at the patch in FLUME-3021, it only puts in > >> Elasticsearch dependencies. > >> > >> The issue with Guava, which you saw in FLUME-2921 is no longer there, > >> because since mid-2016 a lot has changed in the Flume trunk and all > >> those dependencies are fixed. > >> It won't even be possible gto apply this patch anymore. > >> > >> The patch in FLUME-3021 works with the latest Elasticsearch versions, > >> without introducing additional dependencies, which might cause > >> problems to other projects. > >> So I think it can be safely merged into trunk. > >> > >> best regards, > >> > >> Helmut > >> > >> > >> -----Original Message----- > >> From: Mike Percy [mailto:mpe...@apache.org] > >> Sent: Dienstag, 23. Jänner 2018 20:39 > >> To: dev@flume.apache.org > >> Subject: Re: Merging to trunk? > >> > >> Hi Helmut, > >> Thank you for bringing this up on dev@ and thank you for the patch. I > >> see there are other people people interested in this component upgrade > as well. > >> > >> As you are probably aware, a Flume committer will need to approve the > >> change before I gets merged to trunk. > >> > >> My primary concern w/ merging this would be compatibility of the > >> dependencies. Flume suffers from a kind of a "dependency hell" because > >> of a lack of classloading support (either via OSGI or Java modules). > >> See > >> https://issues.apache.org/jira/browse/FLUME-2293 for a tracking ticket > >> about that issue. What that means is that all of the components that > >> Flume ships must have compatible dependencies with each other which > >> makes changes like this more complex. Therefore I would like someone > >> to verify that mvn dependency:tree does not show conflicts when run > >> from the top level with the new patches. If memory serves, I believe > >> Google Guava is likely to conflict. > >> > >> Also, I think that https://issues.apache.org/jira/browse/FLUME-3021 is > >> a duplicate of https://issues.apache.org/jira/browse/FLUME-2921 which > >> has additional information about the various issues that we need to > >> solve to do this upgrade. > >> > >> I'd be happy to discuss this issue some more on this thread. > >> > >> Mike > >> > >> > >> On Tue, Jan 23, 2018 at 12:57 AM, Wahrmann, Helmut < > >> helmut.wahrm...@rsa.com> > >> wrote: > >> > >>> Hi, > >>> > >>> Who decides if/when a patch or pull request gets merged to trunk? > >>> Reason I am asking is for the Elasticsearch support. The current > >>> code in trunk does not work with ES > 2.x. > >>> Currently Elasticsearch is at 6.1. > >>> > >>> In FLUME-3021 we have several patches since March last year. > >>> I have a patched Flume running at a customer since April last year > >>> without any problems. > >>> > >>> So why not merging those changes into the trunk? > >>> As I stated in my comment in FLUME-3021, it will not cause any > >>> problems, cause the current trunk won't work with newer ES versions > >> anyhow. > >>> I doubt that someone is out there still running ES 0.95. > >>> > >>> best regards, > >>> > >>> Helmut > >>> > >> > > > -- *Yonghao Zou* *Shanghai China* *China Merchants Bank* *EN_BLOG <http://blog.haozi1994.com/>* *CH_BLOG* <http://www.fxiaoqing.com/>
