I can also try add these bundles to an actual p2 test case and will share it somehow here :)
From: [email protected] [mailto:[email protected]] On Behalf Of Kapukaranov, Borislav Sent: Tuesday, July 19, 2011 6:18 PM To: P2 developer discussions Subject: Re: [p2-dev] Publishing units with whole manifest Virgo's repository environment is as controlled as Eclipse's and people do have the chance or at least we would like to give them the chance to install things from various repos. Only Virgo's own repository structure is controlled in a sense that it needs to cover any isolation scenarios that we might have with the regions and also there is a defined repository structure for backwards compatibility. The control over bundles(with plans) is just like p2's features, with the only difference we add scoping, to some plans, but still the users controls how and if to group artifacts - again the parallel with the p2 features, where the user controls them. TBH I like 2) from the previous mail more than the "easy" solution :) That much downloading sounds heavy in contrast with lazy manifest loading. I agree that tools are much more "uses"-friendly than people. Here's an example, install all bundles and refresh them. Bundle D will still be in "installed" state and if you "diag" it, uses conflict error will be dumped. http://www.mediafire.com/?zsbdur1nxjaabhb Its exactly the same example as defined in the OSGI core 4.3 spec, figure 3.12. Best Regards Borislav From: [email protected] [mailto:[email protected]] On Behalf Of Pascal Rapicault Sent: Tuesday, July 19, 2011 2:29 PM To: P2 developer discussions Subject: Re: [p2-dev] Publishing units with whole manifest On 2011-07-19, at 10:06 AM, Kapukaranov, Borislav wrote: Hi Pascal, Thanks for the answer :) I was intending to fail early therefore perform the uses resolution before downloading. Still the technical solution isn't fixed and it's good to have options in mind, so I'm curious in the easy solution, can you share it? :) I agree with you on 2 and 4. BTW do you know when the manifest is currently loaded? In 3.7, the manifest is no longer used. The bsn was preserved in the generated metadata such that metadata generated with 3.7 would still be usable by 3.6 installs (for example in the case of upgrade from 3.6 to 3.7) Back in 3.6 (see R3_6_0 version of the eclipse touchpoint), the manifest was used to construct a bundle info that was then passed to framework admin. On the Virgo aspect: 1) I'm not sure if I understood the question entirely but currently in a standard Virgo, there is only Equinox resolution and we've seen it fail because of "uses". We even have "uses" diagnostics in the Web Admin console to help users as its happening more often than expected. All in all uses failure isn't happening on every app deployment but it should be considered common. For example almost every Spring bundle I've checked in Virgo's packaging has a lot of "uses" in their MANIFESTs, so chances are every Spring app might also utilize this. "Uses" is an important aspect of deployment in Virgo and I would really like to benefit from the metadata separation in p2 to fail early when such problems are detected without downloading artifacts. This is interesting. The reason why I was suggesting to fail later than earlier for this scenario is that based on my understanding, server environments were much more controlled (controlled repo, controlled bundles through plans, etc.) and as such the chances to run into a resolution error at install time were much lower in comparison to a typical Eclipse SDK scenario where ppl install things from various repos. So the "easy" solution I had in mind consisted in letting the download occur (and maybe a bit more) and perform the osgi level resolution using the manifests that are in the bundles that got downloaded. This validation could be done as part of the commit phase of the touchpoint. From what I have seen, the uses clause are much more frequent when people are generating the manifest (e.g. using bnd) rather than authoring them. Btw, could you please construct a test case where p2 would install a set of bundles that would then fail to resolver at runtime because of uses clause. I have been after such use cases for a while but nobody ever provided one to me and I have been lazy to try to create one :) 2) If by resolver you mean Equinox's, there is a method to set a resolver hook to the resolver. After that any resolution performed with that resolver instance respects the current region configuration. Such resolver is used to catch "uses" conflicts. In the p2 resolver's case we did it by exposing the stuff visible from the kernel region as IU in the user region's profile (as we discussed some time ago, there are two profiles for the two regions) -this IU is generated on Virgo's start-up according to the currently defined visibility configuration. So far I haven't seen any problems with this. Best Regards Borislav From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Pascal Rapicault Sent: Tuesday, July 19, 2011 2:02 AM To: P2 developer discussions Subject: Re: [p2-dev] Publishing units with whole manifest Hey, On 2011-07-12, at 7:29 PM, Borislav Kapukaranov wrote: Hi, The code still has a reduced set of "interestingKeys" in BundlesAction:411-426 (toManifestString() method). It seems this change is fairly old. Is it possible that tycho uses older p2.publisher version? Or indeed this is configurable somehow and the touchpoint data is modified at a later point? From memory Tycho has not yet been updated to use p2 3.7. What you've seen in BundlesAction is everything when it comes to generate an IU from a bundle. There is no flag to keep on including a complete manifest in the IU. I'm curious about that because when integrating Virgo with p2 it became clear that the manifest of some artifacts or at least parts of it needs to be published as well in order to transfer the "uses" package data it contains and extract it later when deploying but prior executing the provisioning plan to feed it in an Equinox resolver. To be sure, do you intend to do this resolution before downloading the bundles? If you don't, I have an easy answer, if you do, it is another story :) Stepping back on why we stopped including the complete manifest in 3.7, this was mostly done to improve the memory consumption when loading large repos. Given that other changes have been done, it would worth checking what would be the cost of reintroducing the manifests. Aside from this I can think of several solutions: 1) adding a flag to the publisher for the IUs to always include the manifests, but this could cause an issue for the consumer since some IUs in the dependency chain of the one he wants to consume may not have the required manifest information. 2) adding back the manifest in every IU, but improve the parsing code to skip over the attribute and only load it on demand 3) enhancing the p2 requirement model to include the uses flag (so you would have the data in there and could continue on what I assume is inferring the manifest from the IU requirements) and having it be ignored by the p2 resolver for now. 4) change the code in BundlesAction to include the complete manifest if it has a uses clause. At this point, I think that 2 and 4 are the most viable. 3 would work, but is heavier to coordinate since it will require a metadata change unless we decide to play some trick with how the uses clause is expressed (e.g. we could have it be a provided capability instead of a requirement but this is really ugly :) ). Now two questions on the virgo aspect: - How often does the resolution fail when the app is actually deployed? - How will you handle the regions while invoking the resolver? HTH PaScaL So if there is a standard way to configure publishing the whole manifest it would be helpful to understand how its done. If there isn't then what do you think is the best way to transfer extra manifest data from publishing to the resolution phase? An idea can be to publish properties to the IUs containing just the "uses" data or maybe a new "uses" xml element. Obviously reverting to publish the whole manifest again is a step backwards. Thanks, Borislav 2011/7/12 Борислав Капукаранов <[email protected]<mailto:[email protected]>> Hi, While browsing Eclipse's hudson server I stumbled upon some maven/tycho build jobs that publish p2 repositories with units containing the whole artifact manifest. For example check out the content.xml here: https://hudson.eclipse.org/hudson/job/gemini-management/ws/gemini.mgmt.repository/target/site/ The pom.xml used to build this site is here: https://hudson.eclipse.org/hudson/job/gemini-management/ws/gemini.mgmt.repository/pom.xml Another example is this jetty repo: https://hudson.eclipse.org/hudson/view/Jetty-RT/job/jetty-wtp/ws/org.eclipse.jst.server.jetty.site/target/packed/repository/ I thought publishing the whole manifest is removed from p2 due to performance issues. I'm pretty sure I saw it in the code too. Is it still possible to activate it somehow, considering these repositories I found? Thanks Borislav _______________________________________________ p2-dev mailing list [email protected]<mailto:[email protected]> https://dev.eclipse.org/mailman/listinfo/p2-dev _______________________________________________ p2-dev mailing list [email protected]<mailto:[email protected]> https://dev.eclipse.org/mailman/listinfo/p2-dev
_______________________________________________ p2-dev mailing list [email protected] https://dev.eclipse.org/mailman/listinfo/p2-dev
