I managed to get the CI jobs to complete now with setting higher memory. The snapshots are now deployed and the CI jobs for osgi is triggered
Triggering a new build of Camel.trunk.itest.karaf Triggering a new build of Camel.trunk.itest.osgi Finished: SUCCESS On Sat, Mar 26, 2016 at 12:36 PM, Claus Ibsen <claus.ib...@gmail.com> wrote: > Oh I forgot the CI jobs are here > https://builds.apache.org/view/A-D/view/Camel/ > > I got the link from Mueller. Beforehand it was hard/impossible to find > Camel from > https://builds.apache.org/ > > So I was for a longer time not able to see all these jobs, and just > got the CI emails to look at. > > But the OSGi jobs rely on the SNAPSHOT of Camel has been built prior > to that, and that job is unreliable. I think we need to look at making > the -notest also publish the SNAPSHOT to the snapshot mvn repo, so the > CI jobs can pickup these new JARs. > > If we get that more reliable then Camel end users can also easier use > SNAPSHOT instead of having to build locally to be sure they get a > clean build of it, otherwise you can end up with mixed builds or > stale/old snapshots. > > > > On Sat, Mar 26, 2016 at 12:33 PM, Claus Ibsen <claus.ib...@gmail.com> wrote: >> On Sat, Mar 26, 2016 at 12:12 PM, Raul Kripalani <ra...@apache.org> wrote: >>> Agree with your points. >>> >>> 1. If you use lambdas on 2.18, you can't backport the code. >>> >>> 2. I'll spend some time this weekend installing the bundles on Karaf. Karaf >>> 4 is OK, or are we sticking with older versions for 2.x? >>> >> >> Karaf 2.x is dropped. Use Karaf 4.x as the primary karaf version. >> We may want to add a -Pkaraf3 profile to run against older Karaf 3.x >> containers. >> >> >>> 3. I'll also run our OSGi itests, but it may be worth having Jenkins do all >>> this. Could you configure a job in JKS to build this branch? I don't have >>> job admin rights I believe. >>> >> >> We already have some CI jobs to run 2 osgi based tests >> >> - tests/camel-itest-karaf >> - tests/camel-itest-osgi >> >> The former tests that the Camel component can be installed, it does >> this using pax-exam to bootup karaf and install the camel-xxx feature >> and find the component and then tear down the test. >> >> You can run these tests with the -Pkaraf4 profile. >> But mind I had trouble with these tests may stop after 20 or so tests >> and leave some karaf JVMs dangling. There is a kill-karaf.sh script to >> kill em. >> >> The camel-itest-osgi is likely in some state where some tests would >> fail even for older branches. But these are unit tests that run Camel >> apps in karaf and do some testing stuff like we do in regular >> component tests. This has not yet been migrated to karaf 4, and run on >> karaf 2.x. But you can give it a go and see how bad/good it is. It >> used to be able to run almost all tests and pass. And we had also had >> periods where everything was green. >> >> However people dont write so many osgi tests so this module do not >> have so much love. And there are plenty of components that are not >> being tested. >> >> >>> 4. It's necessary to configure the maven-jar-plugin to explicitly use a >>> manifest file, otherwise the plugin generates its own. Originally I tried >>> adding the config to the build plugins in the parent POM, but it failed for >>> artifacts of type=maven-plugin. Hence I placed it in pluginManagement and >>> then incorporated it in those intermediate POMs (components, examples, >>> etc.) whose children built normal JARs. However, I ended up tracing the >>> issue to a conflict introduced by the config of the maven-jar-plugin that >>> maven-plugin injects, and solved it with a couple >>> of combine.self="override". Hence, I think I might be able to push our >>> maven-jar-plugin config back to the parent POM and to the build section. >>> >> >> Ah okay. I am okay with we can do "anything" we need to build all the >> Camel components and whatnot. >> >> Its more what the impact is for end users. If we can make out examples >> and maven archetypes simpler and easier that would be cool. But mind >> that not all examples are osgi examples. >> >> But it looks promising your work - thanks for starting this hard work. >> >> >> >>> Cheers, >>> >>> *Raúl Kripalani* >>> PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and >>> Messaging Engineer >>> http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani >>> Blog: raul.io | twitter: @raulvk <https://twitter.com/raulvk> >>> >>> On Sat, Mar 26, 2016 at 10:12 AM, Claus Ibsen <claus.ib...@gmail.com> wrote: >>> >>>> Hi >>>> >>>> Good to hear that we may drop the <bundle> extension, nice work. >>>> >>>> For camel-core I have thought of build the manifest.mf by hand so we >>>> can better control it. It seems the bundle plugin has its issues with >>>> different versions to build it, as you say. >>>> >>>> Have you tried testing any of these JARs in OSGi? It would be good if >>>> you try that and install some of the examples in a karaf 4 container >>>> and see how it runs. >>>> >>>> For lambda code in the existing source code then we cannot do that for >>>> code that need to be backported to 2.17 and 2.16 branches. We can only >>>> use lambdas in new code that is not to be backported. >>>> >>>> For Camel 3.0 we can then covert the code to use lambads all over >>>> where applicable. There is likely some tools in IDEA or Eclipse etc >>>> that can assist with that. >>>> >>>> >>>> And Camel end users can use lamdas today with their own code, also >>>> with older Camel versions on java 8. >>>> >>>> It may be that those people are using a new bundle plugin, or more >>>> likely not using OSGi at all. >>>> >>>> >>>> I would like to see more OSGi testing on this branch before any merge. >>>> OSGi is complex and pain to maintain and build modules. So we need to >>>> be sure we are on a path that we can do this. There is 250+ maven >>>> modules that are build as OSGi so it better have to work for us. >>>> >>>> But I really like that the bundle plugin is just generating a >>>> MANIFEST.MF file that the JAR plugin just slurps in. Can it not be >>>> configured somehow so the generated MANIFEST.MF is not replace so end >>>> users do not need to do anything with a JAR plugin? >>>> >>>> >>>> >>>> >>>> >>>> On Sat, Mar 26, 2016 at 1:09 AM, Raul Kripalani <ra...@apache.org> wrote: >>>> > There is good news, bad news, and good news again ;-) >>>> > >>>> > *Good news:* Now that Camel 2.18 is officially on JDK8, we can use >>>> lambdas >>>> > in our code. Woohoo! >>>> > >>>> > *Bad news:* Our current Maven build breaks when doing so. >>>> > >>>> > This is what happens: >>>> > >>>> > 1. maven-bundle-plugin version 2.3.x cannot parse JDK8 lambdas. [1] >>>> > >>>> > 2. Upgrading it to the latest version (3.0.1) makes the plugin fail >>>> > miserably (at least on camel-core). >>>> > >>>> > 3. Using the latest 2.x version of the maven-bundle-plugin (2.5.4) >>>> makes >>>> > the maven-shade-plugin fail with this error in turn: >>>> > >>>> > [ERROR] The project main artifact does not exist. This could have the >>>> > following >>>> > [ERROR] reasons: >>>> > [ERROR] - You have invoked the goal directly from the command line. This >>>> is >>>> > not >>>> > [ERROR] supported. Please add the goal to the default lifecycle via an >>>> > [ERROR] <execution> element in your POM and use "mvn package" to have >>>> it >>>> > run. >>>> > [ERROR] - You have bound the goal to a lifecycle phase before "package". >>>> > Please >>>> > [ERROR] remove this binding from your POM such that the goal will be >>>> run >>>> > in >>>> > [ERROR] the proper phase. >>>> > >>>> > 4. Upgrading the shade plugin doesn't help. >>>> > >>>> > 5. The situation is dirty :P >>>> > >>>> > The problem is the usage of the 'bundle' custom packaging type enabled by >>>> > extensions='true' in the maven-bundle-plugin. >>>> > >>>> > *Good news again:* >>>> > >>>> > We can avoid the problem by reverting back to the standard 'jar' >>>> packaging >>>> > type, which even puts us in a less brittle situation wrt. to future Maven >>>> > plugins we may want to add/upgrade. This implies: >>>> > >>>> > 1. Calling the maven-bundle-plugin:manifest goal (instead of the >>>> bundle >>>> > goal) to only generate the MANIFEST.MF. >>>> > >>>> > 2. Adding it to the JAR via an option in the the maven-jar-plugin: >>>> > >>>> > <plugin> >>>> > <groupId>org.apache.maven.plugins</groupId> >>>> > <artifactId>maven-jar-plugin</artifactId> >>>> > <version>${maven-jar-plugin-version}</version> >>>> > <configuration> >>>> > <archive> >>>> > >>>> > >>>> <manifestFile>${project.build.outputDirectory}/META-INF/MANIFEST.MF</manifestFile> >>>> > </archive> >>>> > </configuration> >>>> > </plugin> >>>> > >>>> > I've made the change. But since it's a big one, I didn't push to master >>>> but >>>> > to a branch for us to review first: jdk8-lambdas. >>>> > >>>> > The build passes (yay!) but OSGi battle testing is a must to ensure there >>>> > are no regressions. >>>> > >>>> > Could you have a quick look? If no feedback is received until Monday EOD, >>>> > I'll merge to master. If feedback is positive, we can merge earlier to >>>> > enable people to develop with lambdas finally. >>>> > >>>> > --- >>>> > >>>> > Just in case you're not aware, the OSGi legend Neil Bartlett has just >>>> > released a new bnd-maven-plugin [2] that claims to overcome such >>>> > dysfunctions, as well as others, of the Felix plugin. He complained about >>>> > how the custom packaging type breaks integration with other plugins >>>> > downstream (what we're experiencing). >>>> > >>>> > I see no reason to migrate to his bnd-maven-plugin, but we should keep it >>>> > in mind for the future. >>>> > >>>> > [1] https://issues.apache.org/jira/browse/FELIX-4005 >>>> > [2] http://njbartlett.name/2015/03/27/announcing-bnd-maven-plugin.html >>>> > >>>> > Cheers, >>>> > >>>> > *Raúl Kripalani* >>>> > PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and >>>> > Messaging Engineer >>>> > http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani >>>> > Blog: raul.io | twitter: @raulvk <https://twitter.com/raulvk> >>>> >>>> >>>> >>>> -- >>>> Claus Ibsen >>>> ----------------- >>>> http://davsclaus.com @davsclaus >>>> Camel in Action 2: https://www.manning.com/ibsen2 >>>> >> >> >> >> -- >> Claus Ibsen >> ----------------- >> http://davsclaus.com @davsclaus >> Camel in Action 2: https://www.manning.com/ibsen2 > > > > -- > Claus Ibsen > ----------------- > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2 -- Claus Ibsen ----------------- http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
