Hi Raymond, The semantics described in the OSGi whitepaper are quite prescriptive and amenable to automation, which is why we can semi-automate the process in bndtools. However this pretty much assumes that APIs are pure, i.e. free from implementation dependencies. When you have dependencies across bundles on implementation packages (i.e., packages that contain concrete implementation code rather than pure interfaces) then it's much harder for tools to automatically determine compatibility, since it can vary according to the internal behaviour of the implementation code.
Since it sounds like you have a significant migration task, it's likely that you have a lot of implementation dependencies. I would tend to regard it as higher priority to refactor into pure APIs and OSGi Services before attacking the fine-grained versioning problem, since the first task may vastly reduce the surface area of your APIs and avoid the version management hell that you rightly are fearful of. In terms of annotations, yes there is some assistance available in this area. The OSGi whitepaper makes an important distinction between consumer dependencies and provider dependencies, because the rules of backwards compatibility differ between them. For example, if I add a method to an interface, I have broken any providers of that interface (because they cannot provide the new method) but not the consumers (they simply ignore the new method). This kind of change to the interface should be indicated with a bump of the *minor* version segment. Therefore bnd (or any comparable tool) needs to know what kind of range to generate. If my bundle only *consumes* the API that has changed, then a range like [1.0, 2.0) is correct. But if my bundle *provides* the interface then I must use a range like this: [1.0, 1.1). Bnd uses a directive "provide:=true" to help make this distinction. I understand that it may be difficult to move your project to bndtools. The "release" functionality that I described in my previous mail is currently difficult to use outside of bndtools, but we are working on pushing some parts of it down into bnd itself, so that it can be used from the command line or ANT etc. Since this is open source, any contributions to getting this work done would be very gratefully received ;-) Regards, Neil On Mon, Aug 1, 2011 at 3:51 AM, Raymond Auge <[email protected]>wrote: > Lindsay, > > Thanks for starting this conversation as this has been on the edge of my > mind over the past few days. > > Unlike yourself and Craig, I'm at cusp of a new venture into the world of > OSGi and although integration of it into our product (embedded) is > proceeding extremely well, the question of API versioning (export version) > is a big one. > > I see it very much like you describe it Neil, it's export version that is > tricky to manage. > > We have a rather large API to deal with (our product is about 8 million > lines of code) and comprises hundreds if not thousands of packages which are > not currently in bundle form, but that hopefully over the next several > generations will be migrated. Our first generation will simply export most > if not all of our API via the system bundle. > > While only a fraction of that is public developer API, what I want to make > sure we leverage with OSGi (aside for the obvious aspect of using it as a > plugin framework) is the potential for helping in support and maintenance > (a.k.a. bug patching and hotfix application). > > @Neil, I'm hoping this document provides some algorithmic details for > determining package version! Also, do you know if there is something like a > annotation or metadata based scheme or some such that can be used? We have a > tone of code generation that we could possibly leverage. > > Also, what king of package granularity should we be talking about? (We're > in "no version" hell right now, but I really don't want to end up at the > other extreme either. I could see this getting out of hand pretty quick with > such a large codebase.) > > Any pointers on how to manage this would be extremely helpful? Our existing > build system is ant, and I can't see us moving to bndtools (since we're > heavily entrenched). I've already built some tooling around packages and > such. At most, I could see using bnd directly. > > PS: @Lindsay, I hope this doesn't feel like a thread hijack. I think our > questions are very similar so I wanted to throw my use case into the mix in > support of your own. > > > - Raymond (twitter: rotty3000) > > > On Sun, Jul 31, 2011 at 9:51 PM, Neil Bartlett <[email protected]>wrote: > >> Hello Lindsay, >> >> These are excellent questions. I agree with you that maintaining >> versions is too hard for humans to manage, and tools need to take over >> most of this task. >> >> Before getting into what tools can do, make sure you have read the >> OSGi Semantic Versioning whitepaper[1], which sets out the canonical >> semantics for changes in major, minor, micro and qualifier version >> segments. These semantics are not enforced anywhere in OSGi, but if >> everybody follows them -- or uses tools them automatically use them -- >> then the versioning problem gets a LOT easier. >> >> Contrary to most people's intuition, deciding export versions is a lot >> harder than deciding import ranges. Indeed if the standard semantics >> are followed then import ranges can be generated automatically without >> intervention. In my own workflow, I only have to specify import ranges >> manually if working with a 3rd party library that doesn't follow the >> standard semantics. >> >> I'm glad that you have been evaluating bndtools, because one of our >> goals in building the tool is exactly this: humans should (almost) >> never have to think about versioning. As you noticed we're not quite >> there yet, but getting ever closer I believe. For example, bndtools >> includes a "release" plugin that can be used to release bundles into a >> repository. This tool examines the code changes in your bundles and >> determines what the versions of the packages (and the bundle itself) >> should be, and allows the developer to review the suggestions before >> they are committed to the repository. Finally the versions are >> persisted into source artifacts: "packageinfo" files in the source >> packages themselves, and the main bundle descriptor for the >> Bundle-Version. >> >> Bnd does indeed work out import ranges by looking at the manifests of >> referenced bundles. This works when the referenced bundle is another >> source project in the workspace, or when it is a binary bundle in the >> repository. Of course it doesn't work when using a library at build >> time that is not a bundle, or if it is a bundle but declares no export >> versions. In these cases bnd has to punt and just use a completely >> open version range. Bndtools inherits this behaviour because it wraps >> bnd. >> >> In terms of other tools, Eclipse PDE does include an API Tooling >> plug-in, but in my (biased) opinion, that tooling is not yet useful >> because it does not support package-level versioning. Perhaps somebody >> from PDE can comment on whether and when their API Tooling will >> support package versioning. >> >> You said that your workflow doesn't quite fit bndtools. I'd be very >> glad to have a conversation with you about your workflow and how it >> compares to what bndtools supports. It's actually very flexible, but >> the problem as you point out is the lack of documentation. I am >> "njbartlett" on Skype, feel free to ping me any time when I am online. >> >> Regards, >> Neil >> >> [1] OSGi Semantic Versioning (PDF): >> http://www.osgi.org/wiki/uploads/Links/SemanticVersioning.pdf >> >> >> On Sun, Jul 31, 2011 at 11:46 PM, Lindsay Smith <[email protected]> >> wrote: >> > Hi all, >> > >> > >> > >> > I’ve got some questions about development lifecycle of OSGI components, >> and >> > specifically maintaining and using package versioning correctly. All >> the >> > rationale behind having package versioning is sound in my opinion, OSGI >> has >> > the right abstraction. In my experience the hard part really is how >> this is >> > realised through day to day development practices. >> > >> > >> > >> > I architected the transition of our middleware product from a ‘standard’ >> > Java application to an OSGI-architected one. The main driver for us was >> the >> > class isolation that OSGI can provide – and this was most important for >> us >> > not for our own components, but for classes that are defined by end >> users of >> > the platform. The transition was on the whole very positive, our >> > architecture is much more solid I think, and the build process went from >> a >> > monolithic build to many individual components. >> > >> > >> > >> > Since time was tight we never introduced packaging versioning, and never >> > have. We still rely on ‘naïve’ jar versioning, and manually manage >> package >> > level issues as they arise. I suspect that many OSGI systems are the >> same. >> > >> > >> > >> > I’m attracted to having proper package versioning on our components, but >> > frankly I can’t work out how to integrate it into our development >> process. >> > Put simply, if you have to maintain the package versioning manually, >> it’s >> > never going to work. Unless tools can automate the package versions (at >> > least at the export side), I don’t think the cost will ever outweigh the >> > benefit. I feel that this points to the heart of the ‘complexity’ >> argument >> > that detractors of OSGI make – while the model is sound, unless the >> model is >> > inherent at every level of development and deployment then it cannot be >> > adopted without introducing the need for ‘expert’ (in that it’s not >> common) >> > knowledge about OSGI into the development cycle. >> > >> > >> > >> > Anyway, I have some questions about package versioning which I hope can >> > enlighten me and help me towards some kind of solution path. >> > >> > >> > >> > The Bnd tool is the canonical tool for managing package versions in the >> > manifest. I’m interested to know – how does it work out imported >> package >> > versions? Does it directly read the manifest of the bundles referenced >> by >> > your bundle? >> > >> > >> > >> > What tools are there for maintaining your package versions? Is there a >> > tool that I can use to detect API changes between two versions of a >> bundle? >> > For such a tool to work, you need something to compare it to – so what’s >> the >> > standard for the ‘last thing to compare to’? The last released version? >> > The last built version? This part is crucial – it seems to me you need >> a >> > build system that can go and find the last released version all by >> itself >> > (maven style, although my impression is that that’s a dirty word around >> > here) and fail if your package versions have not been set correctly. >> > Using the last released version makes sense to me since that’s all >> that >> > really matters – steps of interim changes are irrelevant. >> > >> > >> > >> > I am aware of the bndtools project – and this shows some real promise as >> an >> > alternate, lightweight set of tools for managing bundle development. I >> > haven’t found it useful yet, it’s still under development and I’m not >> sure >> > that our workflow fits into its model, but I check on it all the time >> for >> > progress. More docs would be great. >> > >> > >> > >> > Thanks in advance >> > >> > >> > >> > Lindsay >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > _______________________________________________ >> > OSGi Developer Mail List >> > [email protected] >> > https://mail.osgi.org/mailman/listinfo/osgi-dev >> > >> >> _______________________________________________ >> OSGi Developer Mail List >> [email protected] >> https://mail.osgi.org/mailman/listinfo/osgi-dev >> > > > > -- > Raymond Augé <http://twitter.com/#!/rotty3000> > Senior Software Architect > Liferay, Inc. <http://www.liferay.com> <https://twitter.com/#!/liferay> > -- > Liferay West Coast Symposium > September 21-22, 2011 > Register today: www.liferay.com/WCS2011 > New! Going to the Symposium? Add Portal Admin Training > Express<http://www.regonline.com/Register/Checkin.aspx?EventID=991750> > or Building Themes in > Liferay<http://www.regonline.com/Register/Checkin.aspx?EventID=991751> > > > > _______________________________________________ > OSGi Developer Mail List > [email protected] > https://mail.osgi.org/mailman/listinfo/osgi-dev >
_______________________________________________ OSGi Developer Mail List [email protected] https://mail.osgi.org/mailman/listinfo/osgi-dev
