Just one comment on OBR. I'll try to reply to the other points later. On Tue, Jan 25, 2011 at 02:20, David Jencks <[email protected]> wrote: > Thanks for the comments -- more inline. > <snipped> >>> >>> Another current limitation is that the GenerateFeaturesXmlMojo2 really only >>> looks at maven dependencies. What it does is traverse the entire >>> (appropriately scoped) dependency tree. It keeps track of transitive >>> project dependencies and also dependencies that are dependencies of other >>> .kar or features projects. Then it removes all the dependencies found in >>> other feature projects from the set of transitive project dependencies. >>> This avoids duplication but means (if you want this pruning to work) that >>> all the features.xml need to be generated using this plugin from maven >>> dependencies. I don't know if this will be a noticeable limitation since >>> there is the option of asking obr to resolve the dependency rather than >>> directly installing it. >> >> Leveraging OBR here would definitely make sense and maybe it can be >> done by computing the OBR repository for all transitive dependencies >> and asking it to resolve the direct mandatory dependencies. OBR will >> output a list of bundles that should be installed in order to solve >> those direct mandatory dependencies, then we remove any bundle that >> are mandatory bundles for dependant kars/features and flag them as >> being dependencies in the feature. > > Confused again :-). I should warn everyone that I took out the code that > tries to figure out package imports and exports because I couldn't understand > how it is supposed to work. Here I'm having a related problem. I think of > OBR as naturally having all bundles ever created in it so if you ask it to > satisfy package constraints it will come up with an infinite list of > acceptable bundles. However with maven dependencies you have specified which > bundles you want to use for your dependencies. On the other hand the maven > dependencies are hard to override with a different choice. So whenever > someone says they want to use OBR to figure out which bundles to install I > want to know how OBR has been configured to come up with a reasonable list of > acceptable bundles. So far the only solution that makes much sense to me is > to generate OBR repository.xmls for each feature or kar and construct and OBR > that knows about the bundles specified in the features/kars you have actually > installed. I'd really like to understand other peoples point of view on > this, I've been discussing it for at least a year with everyone I can without > much resolution.
I totally agree with your point of view. The OBR repository computation from the feature xml is what we do at runtime when the feature specify an obr resolver: when installing a given feature, we first install the dependent features, then we create an OBR repository from the list of bundles listed in the feature and ask OBR to resolve the bundles not flagged as dependencies. That way, OBR is used mostly to avoid installing bundles that fulfill requirements that can be solved using already installed bundles. The goal here would be to compute this list in an efficient way using OBR, so that if that feature would be installed in an empty framework, all bundles would be installed. > > many thanks! > david jencks > > >> >> >>> >>> thanks! >>> david jencks >>> >>>> >>>>> I have a sandbox project at geronimo I've been using to try these out. >>>>> >>>>> https://svn.apache.org/repos/asf/geronimo/sandbox/djencks/txmanager >>>>> >>>>> thanks >>>>> david jencks >>>>> >>>>> >>>>> On Jan 18, 2011, at 12:35 AM, Jean-Baptiste Onofré wrote: >>>>> >>>>>> David, >>>>>> >>>>>> No problem if you want to fork on github to work around these topics. >>>>>> I will help you on that. >>>>>> >>>>>> Thanks >>>>>> Regards >>>>>> JB >>>>>> >>>>>> On 01/18/2011 09:15 AM, Guillaume Nodet wrote: >>>>>>> On Tue, Jan 18, 2011 at 07:22, Jean-Baptiste Onofré<[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi David, >>>>>>>> >>>>>>>> my comments inline: >>>>>>>> >>>>>>>> 1. maven-generated features.xml and maven feature packaging type. >>>>>>>>> Geronimo has some code that basically generates a features.xml that >>>>>>>>> includes the (appropriately scoped) pom dependencies and transitive >>>>>>>>> dependencies, stopping when a dependency is a feature. It even works >>>>>>>>> with >>>>>>>>> maven 3 :-) (AFAICT the current generate-features-xml mojo does not). >>>>>>>>> I >>>>>>>>> haven't been able to figure out what the current geronimo-feature-xml >>>>>>>>> mojo >>>>>>>>> tries to do but it appears to be something else. So, the output from >>>>>>>>> such a >>>>>>>>> maven project would be a features.xml file with a single feature >>>>>>>>> including >>>>>>>>> all the transitive dependencies. This is a totally non-code project. >>>>>>>>> A >>>>>>>>> feature maven packaging type would then deploy this as the sole >>>>>>>>> generated >>>>>>>>> artifact for this project. Then you can refer to this project as a >>>>>>>>> dependency and it all fits into the maven infrastructure. >>>>>>>>> >>>>>>>> We have the features maven plugin: >>>>>>>> >>>>>>>> http://karaf.apache.org/manual/2.1.99-SNAPSHOT/developers-guide/features-maven-plugin.html >>>>>>>> >>>>>>>> Using the features plugin, you can: >>>>>>>> - features:generate-features-xml to generate the features XML >>>>>>>> - features:add-features-to-repo which reads a features descriptor and >>>>>>>> resolves artifacts to the local repo >>>>>>>> >>>>>>>> FYI, we raised a JIRA to rename this maven plugin with a more generic >>>>>>>> name: >>>>>>>> karaf. >>>>>>>> The purpose is to provide new goals such as karaf:branding, karaf:run, >>>>>>>> karaf:distribution, etc. >>>>>>>> Of course, the existing goals will still exist: >>>>>>>> karaf:generate-features-xml, karaf:add-features-to-repo. >>>>>>>> >>>>>>>> >>>>>>> I think the point here is that if we were to head toward one maven >>>>>>> project >>>>>>> == one feature, handling of maven dependencies would be much easier. >>>>>>> One >>>>>>> problem we had (and that's why camel and servicemix did not use our >>>>>>> plugin), >>>>>>> is that for non pure-OSGi projects such as Camel, the dependencies are >>>>>>> mostly non OSGi-bundles, so having them globally is really difficult to >>>>>>> deal >>>>>>> with. However, if we were to have a single feature per project, it's >>>>>>> way >>>>>>> easier to use maven dependencies to actually reflect the content of the >>>>>>> features. so definitely +1 for me, as I think the current goal has not >>>>>>> proved very usable atm. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> 2. .kar files >>>>>>>>> I just found out about .kar files today. IIUC this is a jar file that >>>>>>>>> contains a feature.xml and the bundle dependencies listed in the >>>>>>>>> feature.xml. I think an additional mojo and packaging type that >>>>>>>>> generates >>>>>>>>> the feature.xml as in (1) and then packages it and the bundles listed >>>>>>>>> into >>>>>>>>> an archive would be a good idea. >>>>>>>>> >>>>>>>> Agree, it's in the plan of the "new" generic karaf maven plugin: >>>>>>>> - karaf:kar goal will take just a features descriptor (and eventually >>>>>>>> some >>>>>>>> resources) to create the kar by packaging the features description >>>>>>>> plus the >>>>>>>> bundles set >>>>>>>> I raised a JIRA about that: KARAF-401. >>>>>>>> >>>>>>>> >>>>>>> +1 >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> 3. resources in .kar files >>>>>>>>> Geronimo .car files can include resources that get unpacked on >>>>>>>>> installation into the server directory structure. While normally osgi >>>>>>>>> bundles are supposed to look for their data in their own osgi-managed >>>>>>>>> data >>>>>>>>> area, sometimes you need something that the user can find :-) and it >>>>>>>>> really >>>>>>>>> belongs in a well known location not associated with a particular >>>>>>>>> bundle. >>>>>>>>> In particular, in geronimo we use this idea in server assembly. We >>>>>>>>> have a >>>>>>>>> .car file that has the basic directory structure of a server and the >>>>>>>>> startup >>>>>>>>> jars packed as resources. When assembling a server, we start with >>>>>>>>> this .car >>>>>>>>> file and unpack the server structure out of it. >>>>>>>>> >>>>>>>> Yeah, as I said before, the kar goal could contains an instructions >>>>>>>> (with >>>>>>>> includes/excludes tags) to embed resources into the generated kar >>>>>>>> archive. >>>>>>>> >>>>>>>> >>>>>>> Note that we recently added a<configfile> element to the<feature> >>>>>>> element >>>>>>> which is related to that. One example is in the http feature in >>>>>>> >>>>>>> http://svn.apache.org/repos/asf/karaf/trunk/features/assembly/standard/src/main/resources/features.xml >>>>>>> Though, in the case of kar files, those files should definitely be >>>>>>> embedded. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> 4. server assembly >>>>>>>>> (cf. KARAF-56) Karaf really needs a really simple way to assemble a >>>>>>>>> custom server using maven. Based on geronimo's experience, one way to >>>>>>>>> do >>>>>>>>> this would be using a maven plugin that installed features and .kar >>>>>>>>> files, >>>>>>>>> using the resource-in-kar idea from (3). You'd use a server-assembly >>>>>>>>> packaging and list all the features and kar files you want as >>>>>>>>> dependencies, >>>>>>>>> and the plugin would install them and pack up the result. >>>>>>>>> >>>>>>>> Agree. I'm working about this. >>>>>>>> The first step is to rename the features maven plugin into karaf maven >>>>>>>> plugin to avoid to lost the users with several plugin name. >>>>>>>> On the karaf maven plugin, the dist goal will create and package a >>>>>>>> custom >>>>>>>> karaf container. >>>>>>>> >>>>>>>> >>>>>>> That was the plan. We just haven't got any time to get to it :-( >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Comments >>>>>>>>> >>>>>>>>> When assembling an osgi server, there's a battle between osgi/obr "I >>>>>>>>> need >>>>>>>>> these packages/services" and maven/file-system/require-bundle "I want >>>>>>>>> this >>>>>>>>> artifact". When a feature lists an artifact to install, its >>>>>>>>> definitely on >>>>>>>>> the "this artifact" side of the line. What do you do when you get to >>>>>>>>> something you need but don't want to include in the feature? you can >>>>>>>>> either >>>>>>>>> specify the external requirements as Import-Package and some kind of >>>>>>>>> require-service specifications or specify e.g. other features whose >>>>>>>>> bundles >>>>>>>>> provide the requirements. Geronimo has used something similar to the >>>>>>>>> "specify other features" approach. Aries EBAs specify osgi package >>>>>>>>> requirements. If you want to assemble a server with the minimum input >>>>>>>>> information, relying on feature dependencies is very simple. However >>>>>>>>> it >>>>>>>>> makes it harder to use alternate sources of packages and services. >>>>>>>>> One idea >>>>>>>>> I had was to associate an OBR repository.xml with each feature that >>>>>>>>> includes >>>>>>>>> the OBR info for the bundles in the >>>>>>>>> >>>>>>>> feature. Then you can create a "virtual" OBR from a set of features by >>>>>>>> including all the repository.xmls from the features. If a feature >>>>>>>> specifies >>>>>>>> its external requirements both as osgi requirements and as feature >>>>>>>> dependencies, an installer/assembler could check if each feature >>>>>>>> dependency >>>>>>>> is needed to satisfy the osgi dependencies before installing it. >>>>>>>> We added the obr resolver in a feature. >>>>>>>> If we package an OBR repository.xml in a feature, I could be append to >>>>>>>> the >>>>>>>> Karaf instance OBR. >>>>>>>> From my point of view, the OBR should be manage: >>>>>>>> - by the Karaf instance itself >>>>>>>> - by the provisioning tool >>>>>>>> As we can consider the feature as a provisioning tool, it makes sense >>>>>>>> that >>>>>>>> it can provide an OBR repository.xml. >>>>>>>> >>>>>>> >>>>>>> I don't think we need to package an OBR respository. Right now, it's >>>>>>> generated based on the content of the features descriptor with all the >>>>>>> bundles, and those marked as 'dependency' are not asked to be resolved, >>>>>>> so >>>>>>> OBR will only include those if they are actually needed. I'm still not >>>>>>> convinced by requiring an OBR repository, as it's an extra thing users >>>>>>> have >>>>>>> to manage, and people may not want to do that. In addition OBR >>>>>>> repositories >>>>>>> consume quite a lot of memory when loaded, so I'd like to avoid >>>>>>> requiring >>>>>>> those. However, using OBR to avoid installing unnecessary bundles is >>>>>>> quite >>>>>>> neat. >>>>>>> >>>>>>> David, as I said on IRC, feel free to work on that on a github fork or >>>>>>> in a >>>>>>> branch in the sandbox, up to you. If you can get to have that working, >>>>>>> that would be awesome !!! >>>>>>> >>>>>>> >>>>>>>> Regards >>>>>>>> JB >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> ---- >>>>>>>>> >>>>>>>>> I'd really appreciate comments on these ideas. I'm going to start >>>>>>>>> seeing >>>>>>>>> if I can munge the geronimo code to do some of this either in a karaf >>>>>>>>> sandbox or in git. Having some code to try out may be a lot clearer >>>>>>>>> than my >>>>>>>>> sometimes convoluted explanations... >>>>>>>>> >>>>>>>>> thanks >>>>>>>>> david jencks >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Cheers, >>>> Guillaume Nodet >>>> ------------------------ >>>> Blog: http://gnodet.blogspot.com/ >>>> ------------------------ >>>> Open Source SOA >>>> http://fusesource.com >>> >>> >> >> >> >> -- >> Cheers, >> Guillaume Nodet >> ------------------------ >> Blog: http://gnodet.blogspot.com/ >> ------------------------ >> Open Source SOA >> http://fusesource.com > > -- Cheers, Guillaume Nodet ------------------------ Blog: http://gnodet.blogspot.com/ ------------------------ Open Source SOA http://fusesource.com
