I now have all the basic functionality I wanted working at least to some extent:
packagings: - feature packaging to just generate a feature.xml - kar packaging to in addition generate a .kar that can include resources and bundles (along with the feature.xml generated as above) - karaf-assembly packaging to take one or more .kars, unpack the resources, unpack the bundles (and feature files) to the local-repo), and package up the server as tar.gz and zip mojos: ArchiveKarMojo packs up a kar file using project resources and generated feature xml ArchiveServerMojo packs up a server GenerateFeaturesXmlMojo2 generates a features.xml containing a single feature with bundles from the (transitive) maven dependencies InstallKarsMojo unpack cars listed as maven dependencies to create a server image putting it to work: features/assembly/framework has the base karaf server resources and builds a kar containing these and the jars and bundles needed to start a minimal server assemblies/apache-karaf assembles and packages a server containing the framework kar. If you unpack the tar.gz, it actually starts. The code is mostly copied from geronimo and is pretty messy, with giant chunks commented out. What's left is pretty simple however. Comments on guillaumes comments inline below: more comments very welcome. On Jan 20, 2011, at 12:20 AM, Guillaume Nodet wrote: > On Thu, Jan 20, 2011 at 02:17, David Jencks <[email protected]> wrote: >> I've set up >> >> https://[email protected]/djencks/karaf.git >> >> So far I've... >> - set up a jaxb tree to read/write feature.xml. Presumably this would be >> better inside the feature bundle itself but that also presumably requires >> some discussion. > > We were envisioning using jaxb when we switch to JDK >= 1.6, i.e. for > 3.x branch. At the moment I have the jaxb classes in the feature plugin. I wonder if it's a reasonable restriction even for 2.2 that if you want to use this new way of generating features and kar files, you need to build on java 6. It's been a while but can't you also add the jaxb goo as maven dependencies perhaps in a profile to allow building on java 5? Eventually I assume they should go into the feature service itself. > >> - added a GenerateFeaturesXmlMojo2 that generates feature.xml based on maven >> dependencies similar to how geronimo deals with them. It (supposedle) stops >> when one of the dependencies is a maven project that generates a feature. >> (not tested yet) >> - added an ArchiveKarMojo that takes the feature.xml and generates a kar >> file with the specified bundles inside in a maven repo-like structure. A >> couple obvious problems with this are that the feature.xml and legal files >> from remote-resources-plugin are not yet included. >> - added feature and kar maven packaging types > > Can't those be generated from the same project ? Or do you see > drawbacks to that? There are two related packagings and two mojos. The feature packaging only uses the feature mojo and stops there. The kar packaging uses the feature mojo to generate the feature xml and continues with the ArchiveKarMojo to package it and other goo into a kar. > >> >> This is all very experimental and there are still a lot of capabilities of >> features and the existing feature plugin that I don't understand. > > Yeah, I think we need to find some way to add them back step by step. > The two most important things that come to my mind are: > > 1/ the use of configurations: those are feed to ConfigAdmin before > starting the bundles Unless you want to do something like convert maven properties to config admin properties (I don't recommend this, it's too hard to get rid of the ones you don't want) I can think of two plausible approaches: 1. allow starting with a base features.xml in src/main/feature/feature.xml and add the bundles to it. This is easy. 2. configure this stuff in the maven feature-maven-plugin configuration. This is kind of yucky because maven uses a pretty non standard xml system so you have to write another model and translate to the jaxb one. I suspect (1) will be most plausible. > > 2/ the use of OBR and flagging some bundles as dependencies > If you're familiar with the way Aries EBAs work, it's the same concept. > If obr is installed and the feature has resolver="obr" or > resolver="(obr)" (the > parenthesis mean that the obr resolver is optional), an OBR > repository will be > generated using the bundles in the feature, and OBR will be > asked to resolve > bundles that are not flagged with dependency="true". This > avoids duplication > of the bundles in the framework instead of reusing those already > installed. the resolver="OBR" should be easy to specify in the maven plugin configuration. Perhaps we can map the maven dependency scope to the feature bundle dependency attribute? <scope>compile</scope> ----> dependency="true" <scope>runtime</scope> ----> dependency="false" (or vice-versa, not sure I understand which way makes more sense) Another current limitation is that the GenerateFeaturesXmlMojo2 really only looks at maven dependencies. What it does is traverse the entire (appropriately scoped) dependency tree. It keeps track of transitive project dependencies and also dependencies that are dependencies of other .kar or features projects. Then it removes all the dependencies found in other feature projects from the set of transitive project dependencies. This avoids duplication but means (if you want this pruning to work) that all the features.xml need to be generated using this plugin from maven dependencies. I don't know if this will be a noticeable limitation since there is the option of asking obr to resolve the dependency rather than directly installing it. thanks! david jencks > >> I have a sandbox project at geronimo I've been using to try these out. >> >> https://svn.apache.org/repos/asf/geronimo/sandbox/djencks/txmanager >> >> thanks >> david jencks >> >> >> On Jan 18, 2011, at 12:35 AM, Jean-Baptiste Onofré wrote: >> >>> David, >>> >>> No problem if you want to fork on github to work around these topics. >>> I will help you on that. >>> >>> Thanks >>> Regards >>> JB >>> >>> On 01/18/2011 09:15 AM, Guillaume Nodet wrote: >>>> On Tue, Jan 18, 2011 at 07:22, Jean-Baptiste Onofré<[email protected]> >>>> wrote: >>>> >>>>> Hi David, >>>>> >>>>> my comments inline: >>>>> >>>>> 1. maven-generated features.xml and maven feature packaging type. >>>>>> Geronimo has some code that basically generates a features.xml that >>>>>> includes the (appropriately scoped) pom dependencies and transitive >>>>>> dependencies, stopping when a dependency is a feature. It even works >>>>>> with >>>>>> maven 3 :-) (AFAICT the current generate-features-xml mojo does not). I >>>>>> haven't been able to figure out what the current geronimo-feature-xml >>>>>> mojo >>>>>> tries to do but it appears to be something else. So, the output from >>>>>> such a >>>>>> maven project would be a features.xml file with a single feature >>>>>> including >>>>>> all the transitive dependencies. This is a totally non-code project. A >>>>>> feature maven packaging type would then deploy this as the sole generated >>>>>> artifact for this project. Then you can refer to this project as a >>>>>> dependency and it all fits into the maven infrastructure. >>>>>> >>>>> We have the features maven plugin: >>>>> >>>>> http://karaf.apache.org/manual/2.1.99-SNAPSHOT/developers-guide/features-maven-plugin.html >>>>> >>>>> Using the features plugin, you can: >>>>> - features:generate-features-xml to generate the features XML >>>>> - features:add-features-to-repo which reads a features descriptor and >>>>> resolves artifacts to the local repo >>>>> >>>>> FYI, we raised a JIRA to rename this maven plugin with a more generic >>>>> name: >>>>> karaf. >>>>> The purpose is to provide new goals such as karaf:branding, karaf:run, >>>>> karaf:distribution, etc. >>>>> Of course, the existing goals will still exist: >>>>> karaf:generate-features-xml, karaf:add-features-to-repo. >>>>> >>>>> >>>> I think the point here is that if we were to head toward one maven project >>>> == one feature, handling of maven dependencies would be much easier. One >>>> problem we had (and that's why camel and servicemix did not use our >>>> plugin), >>>> is that for non pure-OSGi projects such as Camel, the dependencies are >>>> mostly non OSGi-bundles, so having them globally is really difficult to >>>> deal >>>> with. However, if we were to have a single feature per project, it's way >>>> easier to use maven dependencies to actually reflect the content of the >>>> features. so definitely +1 for me, as I think the current goal has not >>>> proved very usable atm. >>>> >>>> >>>>> >>>>> >>>>>> 2. .kar files >>>>>> I just found out about .kar files today. IIUC this is a jar file that >>>>>> contains a feature.xml and the bundle dependencies listed in the >>>>>> feature.xml. I think an additional mojo and packaging type that >>>>>> generates >>>>>> the feature.xml as in (1) and then packages it and the bundles listed >>>>>> into >>>>>> an archive would be a good idea. >>>>>> >>>>> Agree, it's in the plan of the "new" generic karaf maven plugin: >>>>> - karaf:kar goal will take just a features descriptor (and eventually some >>>>> resources) to create the kar by packaging the features description plus >>>>> the >>>>> bundles set >>>>> I raised a JIRA about that: KARAF-401. >>>>> >>>>> >>>> +1 >>>> >>>> >>>>> >>>>> >>>>>> 3. resources in .kar files >>>>>> Geronimo .car files can include resources that get unpacked on >>>>>> installation into the server directory structure. While normally osgi >>>>>> bundles are supposed to look for their data in their own osgi-managed >>>>>> data >>>>>> area, sometimes you need something that the user can find :-) and it >>>>>> really >>>>>> belongs in a well known location not associated with a particular bundle. >>>>>> In particular, in geronimo we use this idea in server assembly. We >>>>>> have a >>>>>> .car file that has the basic directory structure of a server and the >>>>>> startup >>>>>> jars packed as resources. When assembling a server, we start with this >>>>>> .car >>>>>> file and unpack the server structure out of it. >>>>>> >>>>> Yeah, as I said before, the kar goal could contains an instructions (with >>>>> includes/excludes tags) to embed resources into the generated kar archive. >>>>> >>>>> >>>> Note that we recently added a<configfile> element to the<feature> element >>>> which is related to that. One example is in the http feature in >>>> >>>> http://svn.apache.org/repos/asf/karaf/trunk/features/assembly/standard/src/main/resources/features.xml >>>> Though, in the case of kar files, those files should definitely be >>>> embedded. >>>> >>>> >>>>> >>>>> >>>>>> 4. server assembly >>>>>> (cf. KARAF-56) Karaf really needs a really simple way to assemble a >>>>>> custom server using maven. Based on geronimo's experience, one way to do >>>>>> this would be using a maven plugin that installed features and .kar >>>>>> files, >>>>>> using the resource-in-kar idea from (3). You'd use a server-assembly >>>>>> packaging and list all the features and kar files you want as >>>>>> dependencies, >>>>>> and the plugin would install them and pack up the result. >>>>>> >>>>> Agree. I'm working about this. >>>>> The first step is to rename the features maven plugin into karaf maven >>>>> plugin to avoid to lost the users with several plugin name. >>>>> On the karaf maven plugin, the dist goal will create and package a custom >>>>> karaf container. >>>>> >>>>> >>>> That was the plan. We just haven't got any time to get to it :-( >>>> >>>> >>>>> >>>>> >>>>>> >>>>>> Comments >>>>>> >>>>>> When assembling an osgi server, there's a battle between osgi/obr "I need >>>>>> these packages/services" and maven/file-system/require-bundle "I want >>>>>> this >>>>>> artifact". When a feature lists an artifact to install, its definitely >>>>>> on >>>>>> the "this artifact" side of the line. What do you do when you get to >>>>>> something you need but don't want to include in the feature? you can >>>>>> either >>>>>> specify the external requirements as Import-Package and some kind of >>>>>> require-service specifications or specify e.g. other features whose >>>>>> bundles >>>>>> provide the requirements. Geronimo has used something similar to the >>>>>> "specify other features" approach. Aries EBAs specify osgi package >>>>>> requirements. If you want to assemble a server with the minimum input >>>>>> information, relying on feature dependencies is very simple. However it >>>>>> makes it harder to use alternate sources of packages and services. One >>>>>> idea >>>>>> I had was to associate an OBR repository.xml with each feature that >>>>>> includes >>>>>> the OBR info for the bundles in the >>>>>> >>>>> feature. Then you can create a "virtual" OBR from a set of features by >>>>> including all the repository.xmls from the features. If a feature >>>>> specifies >>>>> its external requirements both as osgi requirements and as feature >>>>> dependencies, an installer/assembler could check if each feature >>>>> dependency >>>>> is needed to satisfy the osgi dependencies before installing it. >>>>> We added the obr resolver in a feature. >>>>> If we package an OBR repository.xml in a feature, I could be append to the >>>>> Karaf instance OBR. >>>>> From my point of view, the OBR should be manage: >>>>> - by the Karaf instance itself >>>>> - by the provisioning tool >>>>> As we can consider the feature as a provisioning tool, it makes sense that >>>>> it can provide an OBR repository.xml. >>>>> >>>> >>>> I don't think we need to package an OBR respository. Right now, it's >>>> generated based on the content of the features descriptor with all the >>>> bundles, and those marked as 'dependency' are not asked to be resolved, so >>>> OBR will only include those if they are actually needed. I'm still not >>>> convinced by requiring an OBR repository, as it's an extra thing users have >>>> to manage, and people may not want to do that. In addition OBR >>>> repositories >>>> consume quite a lot of memory when loaded, so I'd like to avoid requiring >>>> those. However, using OBR to avoid installing unnecessary bundles is >>>> quite >>>> neat. >>>> >>>> David, as I said on IRC, feel free to work on that on a github fork or in a >>>> branch in the sandbox, up to you. If you can get to have that working, >>>> that would be awesome !!! >>>> >>>> >>>>> Regards >>>>> JB >>>>> >>>>> >>>>> >>>>>> >>>>>> ---- >>>>>> >>>>>> I'd really appreciate comments on these ideas. I'm going to start seeing >>>>>> if I can munge the geronimo code to do some of this either in a karaf >>>>>> sandbox or in git. Having some code to try out may be a lot clearer >>>>>> than my >>>>>> sometimes convoluted explanations... >>>>>> >>>>>> thanks >>>>>> david jencks >>>>>> >>>>>> >>>> >>>> >> >> > > > > -- > Cheers, > Guillaume Nodet > ------------------------ > Blog: http://gnodet.blogspot.com/ > ------------------------ > Open Source SOA > http://fusesource.com
