That would be ideal, or at least a p2 repo :) On 2011-07-19, at 5:57 PM, Kapukaranov, Borislav wrote:
> I can also try add these bundles to an actual p2 test case and will share it > somehow here J > > From: [email protected] [mailto:[email protected]] On > Behalf Of Kapukaranov, Borislav > Sent: Tuesday, July 19, 2011 6:18 PM > To: P2 developer discussions > Subject: Re: [p2-dev] Publishing units with whole manifest > > Virgo’s repository environment is as controlled as Eclipse’s and people do > have the chance or at least we would like to give them the chance to install > things from various repos. > Only Virgo’s own repository structure is controlled in a sense that it needs > to cover any isolation scenarios that we might have with the regions and also > there is a defined repository structure for backwards compatibility. > The control over bundles(with plans) is just like p2’s features, with the > only difference we add scoping, to some plans, but still the users controls > how and if to group artifacts - again the parallel with the p2 features, > where the user controls them. > > TBH I like 2) from the previous mail more than the “easy” solution J That > much downloading sounds heavy in contrast with lazy manifest loading. > > I agree that tools are much more “uses”-friendly than people. > Here’s an example, install all bundles and refresh them. Bundle D will still > be in “installed” state and if you “diag” it, uses conflict error will be > dumped. > http://www.mediafire.com/?zsbdur1nxjaabhb > Its exactly the same example as defined in the OSGI core 4.3 spec, figure > 3.12. > > Best Regards > Borislav > > From: [email protected] [mailto:[email protected]] On > Behalf Of Pascal Rapicault > Sent: Tuesday, July 19, 2011 2:29 PM > To: P2 developer discussions > Subject: Re: [p2-dev] Publishing units with whole manifest > > > On 2011-07-19, at 10:06 AM, Kapukaranov, Borislav wrote: > > > Hi Pascal, > > Thanks for the answer J > I was intending to fail early therefore perform the uses resolution before > downloading. > Still the technical solution isn’t fixed and it’s good to have options in > mind, so I’m curious in the easy solution, can you share it? J > > I agree with you on 2 and 4. > BTW do you know when the manifest is currently loaded? > In 3.7, the manifest is no longer used. The bsn was preserved in > the generated metadata such that metadata generated with 3.7 would still be > usable by 3.6 installs (for example in the case of upgrade from 3.6 to 3.7) > Back in 3.6 (see R3_6_0 version of the eclipse touchpoint), the > manifest was used to construct a bundle info that was then passed to > framework admin. > > > > On the Virgo aspect: > 1) I’m not sure if I understood the question entirely but currently in a > standard Virgo, there is only Equinox resolution and we’ve seen it fail > because of “uses”. > We even have “uses” diagnostics in the Web Admin console to help users as its > happening more often than expected. > All in all uses failure isn’t happening on every app deployment but it should > be considered common. > For example almost every Spring bundle I’ve checked in Virgo’s packaging has > a lot of “uses” in their MANIFESTs, so chances are every Spring app might > also utilize this. > “Uses” is an important aspect of deployment in Virgo and I would really like > to benefit from the metadata separation in p2 to fail early when such > problems are detected without downloading artifacts. > > > This is interesting. The reason why I was suggesting to fail > later than earlier for this scenario is that based on my understanding, > server environments were much more controlled (controlled repo, controlled > bundles through plans, etc.) and as such the chances to run into a resolution > error at install time were much lower in comparison to a typical Eclipse SDK > scenario where ppl install things from various repos. > So the "easy" solution I had in mind consisted in letting the > download occur (and maybe a bit more) and perform the osgi level resolution > using the manifests that are in the bundles that got downloaded. This > validation could be done as part of the commit phase of the touchpoint. > > From what I have seen, the uses clause are much more frequent > when people are generating the manifest (e.g. using bnd) rather than > authoring them. > Btw, could you please construct a test case where p2 would > install a set of bundles that would then fail to resolver at runtime because > of uses clause. I have been after such use cases for a while but nobody ever > provided one to me and I have been lazy to try to create one :) > > > 2) If by resolver you mean Equinox’s, there is a method to set a > resolver hook to the resolver. > After that any resolution performed with that resolver instance respects the > current region configuration. Such resolver is used to catch “uses” conflicts. > In the p2 resolver’s case we did it by exposing the stuff visible from the > kernel region as IU in the user region’s profile (as we discussed some time > ago, there are two profiles for the two regions) –this IU is generated on > Virgo’s start-up according to the currently defined visibility configuration. > So far I haven’t seen any problems with this. > > Best Regards > Borislav > > > From: [email protected] [mailto:[email protected]] On > Behalf Of Pascal Rapicault > Sent: Tuesday, July 19, 2011 2:02 AM > To: P2 developer discussions > Subject: Re: [p2-dev] Publishing units with whole manifest > > Hey, > > On 2011-07-12, at 7:29 PM, Borislav Kapukaranov wrote: > > > > Hi, > > The code still has a reduced set of "interestingKeys" in > BundlesAction:411-426 (toManifestString() method). > > It seems this change is fairly old. Is it possible that tycho uses older > p2.publisher version? > Or indeed this is configurable somehow and the touchpoint data is modified at > a later point? > From memory Tycho has not yet been updated to use p2 3.7. > What you've seen in BundlesAction is everything when it comes to > generate an IU from a bundle. There is no flag to keep on including a > complete manifest in the IU. > > > > > I'm curious about that because when integrating Virgo with p2 it became clear > that the manifest of some artifacts or at least parts of it needs to be > published as well in order to transfer the "uses" package data it contains > and extract it later when deploying but prior executing the provisioning plan > to feed it in an Equinox resolver. > To be sure, do you intend to do this resolution before > downloading the bundles? > If you don't, I have an easy answer, if you do, it is another story :) > > Stepping back on why we stopped including the complete manifest in 3.7, this > was mostly done to improve the memory consumption when loading large repos. > Given that other changes have been done, it would worth checking what would > be the cost of reintroducing the manifests. Aside from this I can think of > several solutions: > 1) adding a flag to the publisher for the IUs to always include > the manifests, but this could cause an issue for the consumer since some IUs > in the dependency chain of the one he wants to consume may not have the > required manifest information. > 2) adding back the manifest in every IU, but improve the parsing > code to skip over the attribute and only load it on demand > 3) enhancing the p2 requirement model to include the uses flag > (so you would have the data in there and could continue on what I assume is > inferring the manifest from the IU requirements) and having it be ignored by > the p2 resolver for now. > 4) change the code in BundlesAction to include the complete > manifest if it has a uses clause. > > At this point, I think that 2 and 4 are the most viable. 3 would work, but is > heavier to coordinate since it will require a metadata change unless we > decide to play some trick with how the uses clause is expressed (e.g. we > could have it be a provided capability instead of a requirement but this is > really ugly :) ). > > Now two questions on the virgo aspect: > - How often does the resolution fail when the app is actually deployed? > - How will you handle the regions while invoking the resolver? > > HTH > > PaScaL > > > > > So if there is a standard way to configure publishing the whole manifest it > would be helpful to understand how its done. > If there isn't then what do you think is the best way to transfer extra > manifest data from publishing to the resolution phase? > An idea can be to publish properties to the IUs containing just the "uses" > data or maybe a new "uses" xml element. Obviously reverting to publish the > whole manifest again is a step backwards. > > Thanks, > Borislav > > 2011/7/12 Борислав Капукаранов <[email protected]> > Hi, > > While browsing Eclipse's hudson server I stumbled upon some maven/tycho build > jobs that publish p2 repositories with units containing the whole artifact > manifest. > For example check out the content.xml here: > https://hudson.eclipse.org/hudson/job/gemini-management/ws/gemini.mgmt.repository/target/site/ > The pom.xml used to build this site is here: > https://hudson.eclipse.org/hudson/job/gemini-management/ws/gemini.mgmt.repository/pom.xml > > Another example is this jetty repo: > https://hudson.eclipse.org/hudson/view/Jetty-RT/job/jetty-wtp/ws/org.eclipse.jst.server.jetty.site/target/packed/repository/ > > I thought publishing the whole manifest is removed from p2 due to performance > issues. I'm pretty sure I saw it in the code too. > Is it still possible to activate it somehow, considering these repositories I > found? > > Thanks > Borislav > > _______________________________________________ > p2-dev mailing list > [email protected] > https://dev.eclipse.org/mailman/listinfo/p2-dev > > _______________________________________________ > p2-dev mailing list > [email protected] > https://dev.eclipse.org/mailman/listinfo/p2-dev > > _______________________________________________ > p2-dev mailing list > [email protected] > https://dev.eclipse.org/mailman/listinfo/p2-dev
_______________________________________________ p2-dev mailing list [email protected] https://dev.eclipse.org/mailman/listinfo/p2-dev
