Re: Bug in Karaf feature resolver?
Hi JB, Making the SymbolicName unique in the 2 bundles didn't make a difference. How does the resolver consider two bundles to be the same if the URI and SymbolicName (and several MANIFEST.MF headers) are different? Is there something that I can do to force the bundles to appear unique to the resolver, like adding a Provide-Capability header? It sounds like you're saying that the feature resolver does things in this order: 1. Read feature 2. Compose bundle capabilities, exports 3. Evaluate conditions 4. Install bundles I guess it has to do things in that order since conditionals can use 'req:' statements. However, it leads to the case I'm in where the resolver decides that 2 different bundles are the "same" and they get de-duplicated in (2) and then skipped in (3). Is my description of how things work accurate so far? It seems like you could do: 1. Read feature 2. Evaluate feature conditionals 3. Compose bundle capabilities, exports 4. Evaluate req:... conditionals 5. Install bundles I don't know exactly how the feature resolver is working so this could be a shot in the dark. You said in one message that I could work around all of this by using prerequisite features, would that look like: mvn:hello/world/1.0 mvn:hello/world/1.0/jar/gemini aries-blueprint hello-aries gemini-blueprint hello-gemini That doesn't look any different to me (just another layer of indirection) but I don't know what the 'prerequisite' flag means in this case. Seth Leger The OpenNMS Group On 1/29/18 3:03 PM, Jean-Baptiste Onofré wrote: > Exactly, the resolver considers it's the same bundle ;) > > Regards > JB > > On 01/29/2018 08:49 PM, Seth Leger wrote: >> Hi Jean-Baptiste, >> >> The bundles are basically the same: they do have the same version, >> SymbolicName, etc. That might be the problem... maybe I need to make the >> SymbolicName unique. >> >> What I'm basically trying to do is make a conditional where: >> >> >> >> aries-blueprint >> mvn:hello/world/1.0 >> >> >> gemini-blueprint >> mvn:hello/world/1.0/jar/gemini >> >> >> >> and either aries-blueprint or gemini-blueprint is installed in the >> container. The bundle with the 'gemini' classifier has a couple of extra >> Spring files to allow it to run under gemini-blueprint. >> >> However, this is not working because if aries-blueprint is not installed >> (making its condition false), the feature resolver seems to prevent >> mvn:hello/world/1.0/*/* from being installed so >> mvn:hello/world/1.0/jar/gemini isn't installed. >> >> Let me try updating the SymbolicName and see if that fixes it. In other >> words, I'll try: >> >> mvn:hello/world/1.0 -> hello.world >> mvn:hello/world/1.0/jar/gemini -> hello.world.gemini >> >> Seth Leger >> The OpenNMS Group >> >> >> On 1/29/18 2:27 PM, Jean-Baptiste Onofré wrote: >>> By the way, the bundles are different (in term of name, export, etc) ? >>> >>> On 01/29/2018 08:15 PM, Seth Leger wrote: >>>> On 1/29/18 12:49 PM, Jean-Baptiste Onofré wrote: >>>>> If you try to install the bundle by hand using: >>>>> >>>>> bundle:install -s mvn:hello/world/1.0/jar/special >>>>> >>>>> does it work ? >>>> >>>> Yes. And if I put the bundle in the feature like this (without the other >>>> artifact with the same groupId/artifactId/version): >>>> >>>> >>>> mvn:hello/world/1.0/jar/special >>>> >>>> >>>> that works as expected as well. >>>> >>>>> I'm surprised as we use such kind of URL in features, like in Camel for >>>>> instance. >>>> >>>> I don't see any other features in Karaf's standard features, Karaf's >>>> Spring features, or Camel's features that have two bundles with >>>> identical groupId/artifactId/version inside the same feature. The >>>> closest is the 'camel-kubernetes' which has several bundles with >>>> classifiers but they all have unique artifactIds: >>>> >>>> https://github.com/apache/camel/blob/camel-2.20.2/platforms/karaf/features/src/main/resources/features.xml#L1306 >>>> >>>> That's why I think it's a feature resolver issue and not an Aether issue. >>>> >>>> Seth Leger >>>> The OpenNMS Group >>>> >>> >> >
Re: Bug in Karaf feature resolver?
Hi Jean-Baptiste, The bundles are basically the same: they do have the same version, SymbolicName, etc. That might be the problem... maybe I need to make the SymbolicName unique. What I'm basically trying to do is make a conditional where: aries-blueprint mvn:hello/world/1.0 gemini-blueprint mvn:hello/world/1.0/jar/gemini and either aries-blueprint or gemini-blueprint is installed in the container. The bundle with the 'gemini' classifier has a couple of extra Spring files to allow it to run under gemini-blueprint. However, this is not working because if aries-blueprint is not installed (making its condition false), the feature resolver seems to prevent mvn:hello/world/1.0/*/* from being installed so mvn:hello/world/1.0/jar/gemini isn't installed. Let me try updating the SymbolicName and see if that fixes it. In other words, I'll try: mvn:hello/world/1.0 -> hello.world mvn:hello/world/1.0/jar/gemini -> hello.world.gemini Seth Leger The OpenNMS Group On 1/29/18 2:27 PM, Jean-Baptiste Onofré wrote: > By the way, the bundles are different (in term of name, export, etc) ? > > On 01/29/2018 08:15 PM, Seth Leger wrote: >> On 1/29/18 12:49 PM, Jean-Baptiste Onofré wrote: >>> If you try to install the bundle by hand using: >>> >>> bundle:install -s mvn:hello/world/1.0/jar/special >>> >>> does it work ? >> >> Yes. And if I put the bundle in the feature like this (without the other >> artifact with the same groupId/artifactId/version): >> >> >> mvn:hello/world/1.0/jar/special >> >> >> that works as expected as well. >> >>> I'm surprised as we use such kind of URL in features, like in Camel for >>> instance. >> >> I don't see any other features in Karaf's standard features, Karaf's >> Spring features, or Camel's features that have two bundles with >> identical groupId/artifactId/version inside the same feature. The >> closest is the 'camel-kubernetes' which has several bundles with >> classifiers but they all have unique artifactIds: >> >> https://github.com/apache/camel/blob/camel-2.20.2/platforms/karaf/features/src/main/resources/features.xml#L1306 >> >> That's why I think it's a feature resolver issue and not an Aether issue. >> >> Seth Leger >> The OpenNMS Group >> >
Re: Bug in Karaf feature resolver?
On 1/29/18 12:49 PM, Jean-Baptiste Onofré wrote: > If you try to install the bundle by hand using: > > bundle:install -s mvn:hello/world/1.0/jar/special > > does it work ? Yes. And if I put the bundle in the feature like this (without the other artifact with the same groupId/artifactId/version): mvn:hello/world/1.0/jar/special that works as expected as well. > I'm surprised as we use such kind of URL in features, like in Camel for > instance. I don't see any other features in Karaf's standard features, Karaf's Spring features, or Camel's features that have two bundles with identical groupId/artifactId/version inside the same feature. The closest is the 'camel-kubernetes' which has several bundles with classifiers but they all have unique artifactIds: https://github.com/apache/camel/blob/camel-2.20.2/platforms/karaf/features/src/main/resources/features.xml#L1306 That's why I think it's a feature resolver issue and not an Aether issue. Seth Leger The OpenNMS Group
Bug in Karaf feature resolver?
Hi everyone, Running Karaf 4.1.2 and I think I'm running into a bug in the Karaf feature resolver. If I make a feature with 2 bundles generated by the same project (one with a classifier), the resolver ignores the artifact with the classifier and only installs the main artifact. For example: mvn:hello/world/1.0 mvn:hello/world/1.0/jar/special karaf@root> feature:install -t -v hello ... Bundles to install: mvn:hello/world/1.0 ... The mvn:hello/world/1.0/jar/special JAR appears to be ignored. Maybe the feature resolver isn't taking the classifier into account somewhere? Does this sound like a bug that needs to be filed in JIRA? Maybe I am missing something in my feature definition. Seth Leger The OpenNMS Group
Re: Karaf console and Embed Mongo...
Yep, I had the same problem with JNA. Glad you tracked it down. https://issues.apache.org/jira/browse/KARAF-5251 Seth Leger The OpenNMS Group On 1/22/18 10:13 AM, Ryan Moquin wrote: > Thanks Seth, I feel stupid for completely forgetting to check if the > problem is because of the karaf bundles getting refreshed since that > turned out to be the reason. In this case though, it's not Mina, it's > because of JLine and the JNA library (which would be a good candidate to > preinstall.) It stinks to see all these get refreshed because of a > dependency getting added that happens to be an option dependency of a > system bundle. It seems like this happens quite a lot (I find the > log4j2 bundle seems to end up getting refreshed a lot as well) and makes > me feel there has to be some sort of intuitive way to prevent it, > especially because it feels like it could in general hurt Karaf's > adoption when used with an OOTB distribution (I can work around it with > a custom distro.) Out of curiousity, maybe at least a small enhancement > which would at least prevent this issue from confusing people (even > those who are aware of it but are asleep at the wheel that day) could be > to show a warning in the console that some system bundles will be > refreshed which can cause some weird stuff. > > In case interested, this is where the problems are (since mongo embed > needs the com.sun.jna bundle): > > org.jline/3.4.0 (Should be wired to: com.sun.jna/4.5.1 (through > [org.jline/3.4.0] osgi.wiring.package; > filter:="(osgi.wiring.package=com.sun.jna)"; resolution:=optional)) > org.ops4j.pax.logging.pax-logging-log4j2/1.10.1 (Should be wired to: > org.apache.commons.compress/1.10.0 (through > [org.ops4j.pax.logging.pax-logging-log4j2/1.10.1] osgi.wiring.package; > filter:="(osgi.wiring.package=org.apache.commons.compress.compressors)"; > resolution:=optional)) > > Thanks again Seth, this was driving me nuts, I think my brain was > telling me to ask here because I've seen this before (just forgot about > it for some reason). I'll make a change and am sure it will resolve. > Also, I moved to Karaf 4.1.4 since I started having these problems with > 4.2.x and decided at that rate, might as well go to 4.1.4. Maybe I'll > just switch back to working with 4.2.x. Choices. choices. > > Ryan > > On Mon, Jan 22, 2018 at 12:25 AM Seth Leger <s...@opennms.org > <mailto:s...@opennms.org>> wrote: > > Hi Ryan, > > You are probably running into an issue where some dependency of Mongo is > triggering a refresh of the system bundles inside Karaf. Mongo probably > isn't "hijacking" jline; it's just causing jline + karaf shell to > refresh which kills your client shell. There are several ways to do this > on Karaf 4.1, I documented one way to do it (with Apache MINA) in > this bug: > > https://issues.apache.org/jira/browse/KARAF-5384 > > If you use: > > feature:install -t -v your-problematic-feature > > then you should be able to track down which bundle is triggering the > refresh. Also, what version of Karaf are you using? > > Seth Leger > The OpenNMS Group > > > On 1/21/18 12:23 PM, Ryan Moquin wrote: > > This might be better to ask of the Embed Mongo team, but I am not > > entirely sure. I've been experimenting with an Embedded Mongo running > > inside Karaf, which I've gotten to work successfully with one caveat > > that is driving me crazy trying to troubleshoot (I think I understand > > what's happening, but not exactly what makes it happen). If I have a > > feature which installs a bundle that created and starts a Flapdoodle > > Embed Mongo instance, then the Karaf console will "hang" since I think > > the Embed Mongo code, kind of "hijacking" the Karaf JLine input stream > > (it attaches some input streams for the Mongo process.) I did switch > > the outputstreams to use SLF4J which helped a bit, but there is > one last > > small issue I can't figure out. The reason I'm wondering if > someone on > > this list might have some ideas is because if when I install that > > feature, I force kill the Karaf process and start it back up, then the > > Embed Mongo process ends up running in the background without > > "hijacking" the Karaf jline console. > > > > I guess I'm wondering if there is a way to initially install my embed > > mongo feature and prevent it from grabbing the Karaf JLine stream if I > > understand how the feature startup is different than than when > > instal
Re: Karaf console and Embed Mongo...
Hi Ryan, You are probably running into an issue where some dependency of Mongo is triggering a refresh of the system bundles inside Karaf. Mongo probably isn't "hijacking" jline; it's just causing jline + karaf shell to refresh which kills your client shell. There are several ways to do this on Karaf 4.1, I documented one way to do it (with Apache MINA) in this bug: https://issues.apache.org/jira/browse/KARAF-5384 If you use: feature:install -t -v your-problematic-feature then you should be able to track down which bundle is triggering the refresh. Also, what version of Karaf are you using? Seth Leger The OpenNMS Group On 1/21/18 12:23 PM, Ryan Moquin wrote: > This might be better to ask of the Embed Mongo team, but I am not > entirely sure. I've been experimenting with an Embedded Mongo running > inside Karaf, which I've gotten to work successfully with one caveat > that is driving me crazy trying to troubleshoot (I think I understand > what's happening, but not exactly what makes it happen). If I have a > feature which installs a bundle that created and starts a Flapdoodle > Embed Mongo instance, then the Karaf console will "hang" since I think > the Embed Mongo code, kind of "hijacking" the Karaf JLine input stream > (it attaches some input streams for the Mongo process.) I did switch > the outputstreams to use SLF4J which helped a bit, but there is one last > small issue I can't figure out. The reason I'm wondering if someone on > this list might have some ideas is because if when I install that > feature, I force kill the Karaf process and start it back up, then the > Embed Mongo process ends up running in the background without > "hijacking" the Karaf jline console. > > I guess I'm wondering if there is a way to initially install my embed > mongo feature and prevent it from grabbing the Karaf JLine stream if I > understand how the feature startup is different than than when > installing a feature (around the Karaf console input). I hope what I'm > asking actually makes sense to someone. Basically, the feature command > doesn't return control to the Karaf console after installing that > feature due to what the Embedded mongo process does under the covers > I've been debugging the embed mongo code to see if I can figure out how > this conflict is happening.. it appears one of the threads that reads an > inputstream from mongod is the culprit, but this is only a problem when > installing the feature, not running karaf after the feature has been > installed. > > Thanks for any advice! > > Ryan
Re: KARAF-5497 still an issue in Karaf 4.1.4?
This seems to be happening because of the MINA refresh that I reported in: https://issues.apache.org/jira/browse/KARAF-5384 You can see MINA triggering the following reload: MINA -> org.apache.sshd.core/1.6.0 -> org.jline/3.4.0 -> org.apache.karaf.shell.core/4.1.4 -> everything else Seth Leger The OpenNMS Group https://www.opennms.com/ On 1/2/18 11:19 AM, Ryan Moquin wrote: > Hi, I've been in a position to be able to start developing on Karaf > again but having a rough time getting camel-cxf working properly on > Karaf 4.1.3, 4.1.4 and 4.2.0.M1. Ignoring the fact that I am having > trouble getting my rest services to work for right now, if I run Karaf > 4.1.4 and try to install the cxf feature, then Karaf will crash due to > Karaf system bundles getting refreshed. The same happens with camel-cxf > (for some reason, camel-cxf as installs 2 different versions of > jetty...). This problem doesn't seem to occur in Karaf 4.2.0.M1. I'd > like to use a released Karaf for my development, but I seem to only be > able to use 4.2.0.M1. Here is the output from a cleanly started 4.1.4 > Karaf simulating installing CXF 3.2.1 (it happens with 3.2.0 as well): > > karaf@root()> feature:install --simulate cxf > > > 2018-01-02 11:03:12,636 | INFO | l --simulate cxf | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | Adding features: cxf/[3.2.1,3.2.1] > 2018-01-02 11:03:13,601 | INFO | tures-1-thread-1 | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | Changes to perform: > 2018-01-02 11:03:13,603 | INFO | tures-1-thread-1 | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | Region: root > 2018-01-02 11:03:13,604 | INFO | tures-1-thread-1 | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | Bundles to install: > 2018-01-02 11:03:13,605 | INFO | tures-1-thread-1 | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | mvn:com.fasterxml.woodstox/woodstox-core/5.0.3 > 2018-01-02 11:03:13,605 | INFO | tures-1-thread-1 | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | mvn:com.google.guava/guava/20.0 > 2018-01-02 11:03:13,606 | INFO | tures-1-thread-1 | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | mvn:io.dropwizard.metrics/metrics-core/3.2.0 > 2018-01-02 11:03:13,606 | INFO | tures-1-thread-1 | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | mvn:javax.annotation/javax.annotation-api/1.3 > 2018-01-02 11:03:13,608 | INFO | tures-1-thread-1 | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | mvn:javax.mail/mail/1.4.5 > 2018-01-02 11:03:13,610 | INFO | tures-1-thread-1 | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | mvn:javax.servlet/javax.servlet-api/3.1.0 > 2018-01-02 11:03:13,611 | INFO | tures-1-thread-1 | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | mvn:javax.websocket/javax.websocket-api/1.1 > 2018-01-02 11:03:13,612 | INFO | tures-1-thread-1 | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | mvn:joda-time/joda-time/2.9.4 > 2018-01-02 11:03:13,612 | INFO | tures-1-thread-1 | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | mvn:net.sf.ehcache/ehcache/2.10.4 > 2018-01-02 11:03:13,615 | INFO | tures-1-thread-1 | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | > mvn:org.apache.aries.spifly/org.apache.aries.spifly.dynamic.bundle/1.0.2 > 2018-01-02 11:03:13,616 | INFO | tures-1-thread-1 | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | mvn:commons-codec/commons-codec/1.10 > 2018-01-02 11:03:13,618 | INFO | tures-1-thread-1 | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | mvn:org.apache.commons/commons-lang3/3.6 > 2018-01-02 11:03:13,618 | INFO | tures-1-thread-1 | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | mvn:org.apache.cxf/cxf-core/3.2.1 > 2018-01-02 11:03:13,619 | INFO | tures-1-thread-1 | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | mvn:org.apache.cxf/cxf-rt-bindings-coloc/3.2.1 > 2018-01-02 11:03:13,622 | INFO | tures-1-thread-1 | > FeaturesServiceImpl | 10 - org.apache.karaf.features.core - > 4.1.4 | mvn:org.apache.cxf/cxf-rt-bindings-corba/3.2.1 > 2018-01-02 11:03:13,623 | INFO | tures-1-threa
Re: Confused about feature dependency="true"
On 12/8/17 10:47 AM, lechlukasz wrote: > I'm able to do a 'fake' release to our local maven repository. I didn't > thought there's some logic in feature refreshing bound to that. Thanks for > explaining. Is this true? Does Karaf (or something inside Karaf) assume that SNAPSHOT versions should automatically be refreshed when calculating which bundles to refresh? Seth Leger The OpenNMS Group
Re: Fragment-Bundle to pax-logging not getting resolved
Hi Eduard, We are seeing the same issue with fragment bundles in OpenNMS when using Karaf 4.1.2: https://issues.apache.org/jira/browse/KARAF-5446 It might be productive to update our issue with information from your test case. Seth Leger The OpenNMS Group On 11/6/17 7:08 AM, Eduard Vodicka wrote: > The problem: > When I drop this bundle to Karaf's deploy folder, everything works as > expected. It gets resolved and is marked as a fragment bundle. When I > now restart Karaf, the bundle remains in state Installed. It works again > when I delete the bundle cache. > The logs show no indication that something went wrong when resolving the > bundle.
Re: Karaf Bundle stuck in “Starting”-status when trying to install feature programmatically
Hi Marius, One thing to keep in mind is that FeatureServiceImpl was changed to be single-threaded in Karaf 4.0.6. If you are invoking it programmatically, you need to do feature installs in a separate thread (since the single thread will be tied up installing your PluginManager and it will deadlock). This sounds similar to the problem that you're having. https://issues.apache.org/jira/browse/KARAF-3798 We ran into this in OpenNMS and used CompletableFuture as a workaround: https://github.com/OpenNMS/opennms/commit/5cbfaf5b4f455225184950b238ccdea5e94f744b Seth Leger The OpenNMS Group On 11/6/17 2:51 AM, Marius Dienel wrote: > Hey Guys, > > > we're having a problem with Karaf Version > 4.0.4. We are currently > using 4.0.9, but are thinking of upgrading to 4.0.10 soon. In our > application we have a lot of features. One of them installs a bundle > (PluginManager), which itself adds, installs and starts one or multiple > features on startup (plugins for our application). Users are able to > install the plugins via the GUI and the files (.jar) won't get deleted > after building karaf/assembly. > > For each of those features a feature repository is created, if there is > not already one from earlier startups. Everything works fine, if there > already is a feature repository, but after a new build, when the feature > repository does not exist anymore the installation of the features does > not work, which leads to the bundle (PluginManager) being stuck in > "GracePeriod"-status, for versions 4.0.5,4.0.6,4.0.7 or in > "Starting"-status for versions 4.0.8,4.0.9,4.0.10. Everything works as > expected with version 4.0.4. > > The workflow to install the features/plugins is following: > > FeatureService.addRepository(featureUri) --> > FeatureService.getRepository(symbolicName) --> Repository.getFeatures() > --> FeatureService.installFeatures(featureSet, options) > > After stepping over installFeatures in debug-mode the debugger is stuck > and I cannot continue. The biggest problem is that there is no error or > exception in karaf.log. Karaf is still starting correctly and everything > except the plugins part is usable. > > Using 4.1.X is not an option right now, since there are dependency > problems when upgrading. > > Thank you in advance for your help. > > Greetings Marius > > > > - > doubleSlash gehört zu "Deutschlands besten Arbeitgebern 2017" > <https://www.doubleslash.de/unternehmen/presse-uebersicht/pressemitteilungen/detail/dreifachauszeichnung-fuer-doubleslash-als-einer-der-besten-arbeitgeber/> > <http://blog.doubleslash.de/>--- > *Marius Dienel > *Auszubildender > doubleSlash Net-Business GmbH > Otto-Lilienthal-Str. 2 > D-88046 Friedrichshafen > > Fon: +49 7541 / 70078-211 > Fax: +49 7541 / 70078-111 > marius.die...@doubleslash.de > https://doubleSlash.de > <http://doubleslash.de/>--- > > doubleSlash Net-Business GmbH > Geschäftsführung: Konrad Krafft, Andreas Strobel > Sitz, Registergericht: Friedrichshafen, Amtsgericht Ulm HRB 631718 > ---
Karaf git repo gone?
Hi everyone, The Karaf git repo at: https://git-wip-us.apache.org/repos/asf?p=karaf.git appears to have been removed. Was this intentional? It makes tracking down things in JIRA very difficult because all of the automated git repo links are now broken. Seth Leger The OpenNMS Group, Inc.
