Lindsay, Thanks for starting this conversation as this has been on the edge of my mind over the past few days.
Unlike yourself and Craig, I'm at cusp of a new venture into the world of OSGi and although integration of it into our product (embedded) is proceeding extremely well, the question of API versioning (export version) is a big one. I see it very much like you describe it Neil, it's export version that is tricky to manage. We have a rather large API to deal with (our product is about 8 million lines of code) and comprises hundreds if not thousands of packages which are not currently in bundle form, but that hopefully over the next several generations will be migrated. Our first generation will simply export most if not all of our API via the system bundle. While only a fraction of that is public developer API, what I want to make sure we leverage with OSGi (aside for the obvious aspect of using it as a plugin framework) is the potential for helping in support and maintenance (a.k.a. bug patching and hotfix application). @Neil, I'm hoping this document provides some algorithmic details for determining package version! Also, do you know if there is something like a annotation or metadata based scheme or some such that can be used? We have a tone of code generation that we could possibly leverage. Also, what king of package granularity should we be talking about? (We're in "no version" hell right now, but I really don't want to end up at the other extreme either. I could see this getting out of hand pretty quick with such a large codebase.) Any pointers on how to manage this would be extremely helpful? Our existing build system is ant, and I can't see us moving to bndtools (since we're heavily entrenched). I've already built some tooling around packages and such. At most, I could see using bnd directly. PS: @Lindsay, I hope this doesn't feel like a thread hijack. I think our questions are very similar so I wanted to throw my use case into the mix in support of your own. - Raymond (twitter: rotty3000) On Sun, Jul 31, 2011 at 9:51 PM, Neil Bartlett <[email protected]> wrote: > Hello Lindsay, > > These are excellent questions. I agree with you that maintaining > versions is too hard for humans to manage, and tools need to take over > most of this task. > > Before getting into what tools can do, make sure you have read the > OSGi Semantic Versioning whitepaper[1], which sets out the canonical > semantics for changes in major, minor, micro and qualifier version > segments. These semantics are not enforced anywhere in OSGi, but if > everybody follows them -- or uses tools them automatically use them -- > then the versioning problem gets a LOT easier. > > Contrary to most people's intuition, deciding export versions is a lot > harder than deciding import ranges. Indeed if the standard semantics > are followed then import ranges can be generated automatically without > intervention. In my own workflow, I only have to specify import ranges > manually if working with a 3rd party library that doesn't follow the > standard semantics. > > I'm glad that you have been evaluating bndtools, because one of our > goals in building the tool is exactly this: humans should (almost) > never have to think about versioning. As you noticed we're not quite > there yet, but getting ever closer I believe. For example, bndtools > includes a "release" plugin that can be used to release bundles into a > repository. This tool examines the code changes in your bundles and > determines what the versions of the packages (and the bundle itself) > should be, and allows the developer to review the suggestions before > they are committed to the repository. Finally the versions are > persisted into source artifacts: "packageinfo" files in the source > packages themselves, and the main bundle descriptor for the > Bundle-Version. > > Bnd does indeed work out import ranges by looking at the manifests of > referenced bundles. This works when the referenced bundle is another > source project in the workspace, or when it is a binary bundle in the > repository. Of course it doesn't work when using a library at build > time that is not a bundle, or if it is a bundle but declares no export > versions. In these cases bnd has to punt and just use a completely > open version range. Bndtools inherits this behaviour because it wraps > bnd. > > In terms of other tools, Eclipse PDE does include an API Tooling > plug-in, but in my (biased) opinion, that tooling is not yet useful > because it does not support package-level versioning. Perhaps somebody > from PDE can comment on whether and when their API Tooling will > support package versioning. > > You said that your workflow doesn't quite fit bndtools. I'd be very > glad to have a conversation with you about your workflow and how it > compares to what bndtools supports. It's actually very flexible, but > the problem as you point out is the lack of documentation. I am > "njbartlett" on Skype, feel free to ping me any time when I am online. > > Regards, > Neil > > [1] OSGi Semantic Versioning (PDF): > http://www.osgi.org/wiki/uploads/Links/SemanticVersioning.pdf > > > On Sun, Jul 31, 2011 at 11:46 PM, Lindsay Smith <[email protected]> > wrote: > > Hi all, > > > > > > > > I’ve got some questions about development lifecycle of OSGI components, > and > > specifically maintaining and using package versioning correctly. All the > > rationale behind having package versioning is sound in my opinion, OSGI > has > > the right abstraction. In my experience the hard part really is how this > is > > realised through day to day development practices. > > > > > > > > I architected the transition of our middleware product from a ‘standard’ > > Java application to an OSGI-architected one. The main driver for us was > the > > class isolation that OSGI can provide – and this was most important for > us > > not for our own components, but for classes that are defined by end users > of > > the platform. The transition was on the whole very positive, our > > architecture is much more solid I think, and the build process went from > a > > monolithic build to many individual components. > > > > > > > > Since time was tight we never introduced packaging versioning, and never > > have. We still rely on ‘naïve’ jar versioning, and manually manage > package > > level issues as they arise. I suspect that many OSGI systems are the > same. > > > > > > > > I’m attracted to having proper package versioning on our components, but > > frankly I can’t work out how to integrate it into our development > process. > > Put simply, if you have to maintain the package versioning manually, it’s > > never going to work. Unless tools can automate the package versions (at > > least at the export side), I don’t think the cost will ever outweigh the > > benefit. I feel that this points to the heart of the ‘complexity’ > argument > > that detractors of OSGI make – while the model is sound, unless the model > is > > inherent at every level of development and deployment then it cannot be > > adopted without introducing the need for ‘expert’ (in that it’s not > common) > > knowledge about OSGI into the development cycle. > > > > > > > > Anyway, I have some questions about package versioning which I hope can > > enlighten me and help me towards some kind of solution path. > > > > > > > > The Bnd tool is the canonical tool for managing package versions in the > > manifest. I’m interested to know – how does it work out imported package > > versions? Does it directly read the manifest of the bundles referenced > by > > your bundle? > > > > > > > > What tools are there for maintaining your package versions? Is there a > > tool that I can use to detect API changes between two versions of a > bundle? > > For such a tool to work, you need something to compare it to – so what’s > the > > standard for the ‘last thing to compare to’? The last released version? > > The last built version? This part is crucial – it seems to me you need a > > build system that can go and find the last released version all by itself > > (maven style, although my impression is that that’s a dirty word around > > here) and fail if your package versions have not been set correctly. > > Using the last released version makes sense to me since that’s all > that > > really matters – steps of interim changes are irrelevant. > > > > > > > > I am aware of the bndtools project – and this shows some real promise as > an > > alternate, lightweight set of tools for managing bundle development. I > > haven’t found it useful yet, it’s still under development and I’m not > sure > > that our workflow fits into its model, but I check on it all the time for > > progress. More docs would be great. > > > > > > > > Thanks in advance > > > > > > > > Lindsay > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > OSGi Developer Mail List > > [email protected] > > https://mail.osgi.org/mailman/listinfo/osgi-dev > > > > _______________________________________________ > OSGi Developer Mail List > [email protected] > https://mail.osgi.org/mailman/listinfo/osgi-dev > -- Raymond Augé <http://twitter.com/#!/rotty3000> Senior Software Architect Liferay, Inc. <http://www.liferay.com> <https://twitter.com/#!/liferay> -- Liferay West Coast Symposium September 21-22, 2011 Register today: www.liferay.com/WCS2011 New! Going to the Symposium? Add Portal Admin Training Express<http://www.regonline.com/Register/Checkin.aspx?EventID=991750> or Building Themes in Liferay<http://www.regonline.com/Register/Checkin.aspx?EventID=991751>
_______________________________________________ OSGi Developer Mail List [email protected] https://mail.osgi.org/mailman/listinfo/osgi-dev
