2017-02-01 12:35 GMT+01:00 Łukasz Dywicki <l...@code-house.org>: > @Guillaume you are right on findEntries part, this will break with > maven execution. >
What findEntries part are you talking about ? I was just referring to the fact that a modified version of MavenRepositorySystemUtil class would be better embedded directly into pax-url-aether. Else, you'd have to provide 2 alternative ways to use it, one by using a fragment and the other one by ensuring that the hacked class is loaded first in the classloader (in case of a simple classloader). It just seems easier to me to just move that class in the shaded pax-url-aether and hack it so that the behavior can be easily controlled using a boolean on the usual pax-url-aether configuration. > > @Grzegorz I do not negate that version, metadata or range resolving > works. It does with some small exceptions (see PAXURL-342). What > doesn't work is re-use of ranges from maven build into karaf runtime. > When you use maven <dependency> and aether to build custom assembly > feature resolver in runtime will not work for certain cases because of > version edges. > When you will take a look on these two files: > https://github.com/splatch/maven-osgi-resolver/blob/ > master/shared/src/test/java/org/code_house/maven/osgi/ > resolver/shared/version/OsgiVersionRangeTest.java > https://github.com/splatch/maven-osgi-resolver/blob/ > master/test/src/test/java/org/eclipse/aether/util/version/ > GenericVersionRangeTest.java > and diff between them https://www.diffchecker.com/eyzDSSXU you will > find quite big area for troubles. > > We about to start using ranges in build because we do not want to have > 3 different minor versions installed without need. Using ranges in > both places acutally might speed up running integration tests because > metadata is already fetched and present in local repository. Otherwise > our integration tests bootstrap takes 2 minutes just to scan remotes > for new versions (see above pax url issue). > > Cheers, > Lukasz > > 2017-02-01 12:02 GMT+01:00 Grzegorz Grzybek <gr.grzy...@gmail.com>: > > Hello, > > > > Current pax-url-aether has already some custom services DInjected into > > RepositorySystem, like here[1]. Also in Fabric8v1 and in Karaf I did some > > tricks to implement non-canonical "update releases" scenario[2]. > > So I think adding configuration options for pax-url to modify the way > > RepositorySystem is configured should not be a problem. > > > > What is the ultimate problem you want to solve? Is it (at lowest level) > the > > ability to handle the below URLs?: > > > > osgi:install mvn:groupId/artifactId/[lowerBound, upperBound) > > > > Currently (pax-url 2.5.2) LATEST, RELEASE and SNAPSHOT versions should be > > handled correctly[3]: > > - LATEST instructs AetherBasedResolver to fetch group/artifact > > metadata.xml and pick latest release OR snapshot > > - RELEASE instructs AetherBasedResolver to fetch group/artifact > > metadata.xml and pick latest release > > - SNAPSHOT instructs AetherBasedResolver to fetch group/artifact/version > > metadata.xml and pick latest snapshot > > > > e.g., in Fabric8v1 I added custom org.eclipse.aether.impl. > MetadataResolver > > that is able to resolve metadata ("maven-metadata.xml") even in local > > repositories into which a SNAPSHOT was installed using `mvn clean > install` > > - so the metadata is stored in "maven-metadata-local.xml" file - but the > > repo is used as remote repository (expecting to return > "maven-metadata.xml" > > file. > > > > Are you using same version ranges in POM and in features.xml? (I don't > > argue with the fact that version ranges are used at all in POM :). > > > > regards > > Grzegorz > > === > > [1]: > > https://github.com/ops4j/org.ops4j.pax.url/blob/master/pax- > url-aether/src/main/java/org/ops4j/pax/url/mvn/internal/ > AetherBasedResolver.java#L1168-L1169 > > [2]: > > https://ops4j1.jira.com/browse/PAXURL-322?focusedCommentId=37006&page= > com.atlassian.jira.plugin.system.issuetabpanels:comment- > tabpanel#comment-37006 > > [3]: http://ggrzybek.blogspot.com/2016/10/using-maven-with-osgi- > part-3.html > > > > 2017-02-01 11:44 GMT+01:00 Guillaume Nodet <gno...@apache.org>: > > > >> 2017-02-01 11:31 GMT+01:00 Łukasz Dywicki <l...@code-house.org>: > >> > >> > Thanks for your repiles. If we will manage to get pax-url accepting > >> > different version range resolving than maven default then I think we > >> > will not have any troubles with features left. What I was thinking > >> > about is moving my maven-osgi-resolver to karaf tooling and extending > >> > pax-url in the way it could pick up version range resolver > >> > implementation from fragment bundle. This way we could keep current > >> > behavior which might be used by someone but also let others use end to > >> > end range support. There are more "extension points" built into > >> > Aerther which gets normally wired by IoC. Since we can't and we do not > >> > want to embed yet-another-ioc-tool for low level stuff we would just > >> > need to make aether's ServiceLocator entries customizable. It is > >> > simple Map between role and implementation classes thus would not > >> > require anything more than bundle.findEntries. This way we could also > >> > solve pax-url troubles with wagon not loaded up properly. > >> > > >> > >> Won't that make things a bit more complicated for the karaf maven > plugin ? > >> It does not run in OSGi, so the fragment stuff won't work. If the > problem > >> is the compatibility, it may still be easier to put the code in > >> pax-url-eather, and only have a flag to turn the version resolver into > an > >> OSGi compatible one, so that the default would be unchanged. > >> I honestly don't mind, I'm just trying to find the best way to handle > that. > >> > >> > >> > @Guillaume, we don't need to handle RELEASE flag because this part is > >> > not subject of version range resolution but VersionResolver. This is > >> > piece of logic we would not (hopefully) need to amend. > >> > > >> > If you will take a look on my current implementation there is mixed > >> logic: > >> > https://github.com/splatch/maven-osgi-resolver/blob/ > >> > master/compatible/src/main/java/org/code_house/maven/ > >> > osgi/resolver/compatible/CompatibleOsgiVersionRangeResolver.java#L87 > >> > https://github.com/splatch/maven-osgi-resolver/blob/ > >> > master/strict/src/main/java/org/code_house/maven/osgi/ > resolver/strict/ > >> > StrictOsgiVersionRangeResolver.java#L80 > >> > > >> > First implementation uses Maven ordering of versions meaning it > >> > preffers releases over snapshots in selected range. Second > >> > implementation behaves as OSGi, meaning it will ignore snapshot and > >> > use regular qualifier comparision but more importantly it will also > >> > accept just 3.4.0 as a range without upper bound. > >> > > >> > Best regards, > >> > Lukasz > >> > > >> > 2017-02-01 9:44 GMT+01:00 Jean-Baptiste Onofré <j...@nanthrax.net>: > >> > > Hi Lukasz, > >> > > > >> > > Thanks for your detailed e-mail and I fully agree with you. > >> > > > >> > > I guess the first step would be to improve the version range > support in > >> > > Maven URL, and after in the feature resolver. > >> > > > >> > > Correct ? > >> > > > >> > > Regards > >> > > JB > >> > > > >> > > > >> > > On 02/01/2017 02:05 AM, Łukasz Dywicki wrote: > >> > >> > >> > >> Dear receivers, > >> > >> I would like to summarize my research and fight to align version > range > >> > >> handling in different parts of karaf related projects. As some of > you > >> > >> might not know version ranges are working differently depending on > >> > >> context we are working in. In general most of logic stays the same > >> > >> while there are some edge cases which breaks up everything. But > let me > >> > >> start from begining. > >> > >> > >> > >> Karaf is OSGi related project which keeps very nice integration > with > >> > >> maven based repositories thanks to pax-url. Both environments do > >> > >> support ranges in quite different way, an example of maven range > >> > >> understanding is described in maven enforcer plugin documentation > [1]. > >> > >> Reason why ranges are working differently here and there is a maven > >> > >> snapshot version and understanding of released version. Osgi > framework > >> > >> does not distinguish any of these. It has knowledge of major, minor > >> > >> and micro parts of an version and uses them for comparision but the > >> > >> qualifier is just a text which might be used for sorting artifacts > >> > >> with same number. This means that for Maven 3.0-SNAPSHOT version is > >> > >> lower than 3.0. In maven there is also knowledge of alpha, beta, > rc, > >> > >> cr, milestone, ga and sp (service pack) release types [2]. > >> > >> > >> > >> Now lets come to places which are using or might be using version > >> > >> ranges in typical Karaf based project: > >> > >> - OSGi framework for wiring in packages > >> > >> - pax-url-mvn for installing maven artifacts > >> > >> - karaf feature core for choosing dependant features > >> > >> - maven for including dependant artifacts (ie. feature sets/KARs > etc) > >> > >> - karaf-maven-plugin for building assemblies > >> > >> > >> > >> When any of range definitions is crossing osgi-maven world problems > >> > >> starts to happen. For example range such [2.18, 2.19) in maven will > >> > >> accept 2.19.0-SNAPSHOT while in OSGi it will not. This lead to > >> > >> situations that these two code parts behave completely differently > >> > >> (assuming that camel-core feature is just one bundle): > >> > >> <bundle>mvn:org.apache.camel/camel-core/${camel.version}</bundle> > >> > >> <feature version="${camel.version}">camel-core</feature> > >> > >> This will behave like above but not like bundle statement: > >> > >> > >> > >> <repository>mvn:org.apache.camel.karaf/features/${camel. > >> > version}/xml/features</repository> > >> > >> > >> > >> There are some attempts to work around that by using versions > starting > >> > >> from ie 2.18.1 so version beginning works just fine but still > there is > >> > >> problem of range end. To exclude 2.19-SNAPSHOT in maven you must > use > >> > >> "2.19.min" which in osgi will acceptversion 2.19.. Obviously there > is > >> > >> also no way to influence 3rd party so they do not release version > >> > >> 4.1.0 but 4.1.1 just for our environment pleasure. > >> > >> > >> > >> For me it's quite big issue because hitting us on daily basis. We > have > >> > >> quite few modules (around 400) which are usualy moving together but > >> > >> they should be keeping contract/interfaces on micro versions. This > >> > >> inconsistency lives in Karaf and Pax Url since very long time and > >> > >> current project infrastructure is not ready to changing that. From > >> > >> other hand keeping this inconsistent will lead to ultimate fail > some > >> > >> day and users frustration as well (see KARAF-4105 [3]). Worth to > point > >> > >> that this issue pointed out brieefly this issue but didn't solve > cause > >> > >> but aligned just one place to maven's logic while keeping all > others > >> > >> the same. > >> > >> > >> > >> I took my chance and managed to get maven understanding osgi > version > >> > >> ranges thanks to core extensions mechanism they have [4]. I also > >> > >> managed to correct shaded aether inside pax-url [5] so it use > version > >> > >> ranges in same way as maven. What I completely failed is making a > >> > >> custom distro built with my pax-url. Since pax-url-mvn is a startup > >> > >> bundle I can't use overrides for changing it's version and I can't > >> > >> influence its classes using fragment bundle (yet). To get my own > >> > >> pax-url I would ned to get rid of framework, but then I have to > copy > >> > >> bunch of resources. It would be fine for temporary prosthesis but I > >> > >> can't rely on it forever. I also got into troubles with > >> > >> karaf-maven-plugin when setting extra dependency with "my own > aether". > >> > >> > >> > >> As you now know - there is lots of troubles with version ranges > making > >> > >> their usage in end-to-end build very difficult. I would love to get > >> > >> this solved as soon as possible in 4.1 without holding current > >> > >> release. Aligning all these version range handling is definitelly > >> > >> doable because from Maven/Aether perspective there is an SPI for > that. > >> > >> We just need to deliver it our own VersionRangeResolver interface > [6]. > >> > >> Open question is shall we keep ordering of versions same as maven > >> > >> breaking up a little osgi range understanding here. > >> > >> > >> > >> [1] > >> > >> http://maven.apache.org/components/enforcer/enforcer- > >> > rules/versionRanges.html > >> > >> [2] > >> > >> https://github.com/eclipse/aether-core/blob/1.0.x/aether- > >> > util/src/main/java/org/eclipse/aether/util/version/ > >> > GenericVersion.java#L183 > >> > >> [3] https://issues.apache.org/jira/browse/KARAF-4105 > >> > >> [4] http://markmail.org/message/z6x27umabwqhdjvy > >> > >> [5] > >> > >> https://github.com/splatch/maven-osgi-resolver/blob/ > >> > master/compatible-pax/pom.xml > >> > >> [6] > >> > >> https://github.com/splatch/maven-osgi-resolver/blob/ > >> > master/compatible-locator/src/main/java/org/apache/maven/ > >> > repository/internal/MavenRepositorySystemUtils.java#L78 > >> > >> > >> > >> Kind regards, > >> > >> Lukasz > >> > >> -- > >> > >> Apache Karaf Committer & PMC > >> > >> Twitter: @ldywicki > >> > >> Blog: http://dywicki.pl > >> > >> Code-House - http://code-house.org > >> > >> > >> > > > >> > > -- > >> > > Jean-Baptiste Onofré > >> > > jbono...@apache.org > >> > > http://blog.nanthrax.net > >> > > Talend - http://www.talend.com > >> > > >> > >> > >> > >> -- > >> ------------------------ > >> Guillaume Nodet > >> > -- ------------------------ Guillaume Nodet
