On 6/12/08, Simon Nash <[EMAIL PROTECTED]> wrote: > I am very pleased to see this discussion happening. My thoughts below. > > Simon > > Rajini Sivaram wrote: > >> On 6/12/08, Graham Charters <[EMAIL PROTECTED]> wrote: >> >> Hi Rajini, I think your summary on the wiki is great. I have a couple >>> of comments: >>> >>> 1. I believe SpringSource try to create sensible version ranges based >>> on the versioning governance of the project, such as that of Apache >>> [1]. I have no doubt this takes quite a bit of effort and there are a >>> number of examples which do not follow guidelines, but it seems like a >>> sensible place to start. >>> >> >> >> "Sensible" is quite difficult to define. Here is example of one of the >> SpringSource versioned jars - Axiom 1.2.5 (this is on Apache License): >> >> The versions going upto infinity are those provided by the Java SDK. For >> the >> others, the minimum versions are similar to the minimum versions that we >> would generate using maven-bundle-plugin in Tuscany (based on current >> versions), and the maximum versions (at least in this example) go upto the >> maximum within the major version range. I have only looked at a few, but >> the >> ones I looked at follow this pattern. >> >> javax.activation [1.1.0, 2.0.0) >> javax.mail [1.4.0, 2.0.0) >> javax.mail.internet [1.4.0, 2.0.0) >> javax.xml.namespace [0.0.0,infinity) >> javax.xml.stream [1.0.1, 2.0.0) >> junit.framework [3.8.2, 4.0.0) >> org.apache.commons.logging [1.1.1, 2.0.0) >> org.jaxen [1.1.1, 2.0.0) >> org.jaxen.saxpath [1.1.1, 2.0.0) >> org.jaxen.util [1.1.1, 2.0.0) >> org.w3c.dom [0.0.0,infinity) >> org.xml.sax [0.0.0,infinity) >> org.xml.sax.helpers [0.0.0,infinity) >> >> Whenever we import a package using a version other than the currently >> implemented and tested versions, we are making an assumption about being >> future-proof. IMHO, that is hardly ever the case. eg. If Tuscany 1.2.1 was >> released with Axis version 1.3, and Axis version 1.4 was not available for >> testing at the time, can we really assume that Tuscany 1.2.1 will work >> with >> Axis 1.4 when it comes out just because the major version has not changed? >> By choosing the version range [1.3.0,2.0.0) for Tuscany imports, we are >> making that assumption. Now this does not matter as long as the user wants >> to use Tuscany out of the box and does not install Axis version 1.4 into >> the >> OSGi runtime. But what happens when the user does install Axis 1.4 to use >> it >> with another application? Tuscany could fall over in spite of the fact >> that >> there is an Axis 1.3 present in the runtime, because Tuscany will be wired >> to 1.4 since it believed that it would work with any version beyond 1.3. I >> do agree that SpringSource provides a good starting point, since a lot of >> work has already gone into it. But I would expect that some level of fine >> tuning will be required for Tuscany beyond that. >> >> I think this is a serious problem and we do need to take into account > that users will install higher levels of Tuscany dependencies such as > Axis2. > > We could guard against Tuscany breaking in this case by specifying the > Axis2 dependency as a narrower range such as (1.3.0, 1.4.0) instead of > (1.3.0, 2.0.0). This would mean that if Axis2 1.4 is installed, Tuscany > will still use Axis2 1.3 even though other code is using Axis2 1.4. > This should avoid breakage.
Yes, this is true. But how narrow should the range be? SpringSource assumes compatibility at major version. Should we assume minor version? Or should we restrict to revision? Are we saying that we work with 1.3.0, so we would work with 1.3.2, but not 1.4? Or are we saying that we have only tested against 1.3.0, so we will only use 1.3.0? > The next step is for Tuscany to determine whether or not Axis2 1.4 > is compatible for Tuscany purposes. If it is compatible, Tuscany > could release an update for its bundles that changes their Axis2 > dependency to (1.3.0, 1.5.0), with no change to the Tuscany code. The dependency graphs for Tuscany (even ignoring applications) is likely to be very complex, because we have 149 3rd party libraries with many 3rd party libraries having dependencies on each other. In the first instance, yes, I agree, we can easily restrict to a single major.minor.revision for each library. We test Tuscany with one combinarion of these 149 libraries. And it works, so we force this combination. But what happens in a year or two, when each of these 149 libraries may have 5 releases (minor releases perhaps, but new versions nevertheless). Do we test all 5 ** 149 combinations of 3rd party libraries to ensure that they can all work together? Even if we had tools which could work through the dependency graphs and provide a minimal set of combinations that could coexist, this would still be an awful lot of testing. Add to this application versioning requirements, third party bundles from other sources, and we will end up with a problem of enormous complexity. > This approach keeps Tuscany working at all times, no matter what > other versions of its dependencies are installed. The downside > is the work needed for Tuscany to do compatibility testing and > release updated dependency specifications from time to time, and > for the user to install these updates when she wants to move > forward to the latest possible Tuscany dependencies without also > upgrading Tuscany. > > Are there any other problems with this approach? Broad version ranges can potentially lead to incompatibility issues, but narrow ones can potentially lead to classloading errors, and force restrictions on application's use of 3rd party libraries. 1. TCCL based classloading (XML parsing for instance) : as we allow more versions to co-exist, there is more likelihood of TCCL picking up the wrong versions of libraries 2. Class sharing between applications and Tuscany - It would be really good to have a definitive list of classes which applications share with Tuscany directly or indirectly. For classes which are shared, narrow version ranges in Tuscany automatically impose narrow version ranges for applications. For applications which also use 3rd party libraries from other sources, the result will be runtime classloading exceptions. 3. Upgrading 3rd party libraries will require new Tuscany releases. This is inconsistent with how we work in the non-OSGi space where the user has the flexibility to upgrade 3rd party libraries by just adding them to CLASSPATH. 4. This is inconsistent with other repositories like SpringSource etc (and also not standard/best practice in OSGi) I do think that there may be some libraries like Axis2 kernel where a narrow range (at minor version?) makes the most sense. But I would expect that a larger number of 3rd party libraries can cope with broader ranges, and I think we should adopt broad ranges "where possible" (not sure what that means). > At least this approach is not arbitrary and >>> therefore, IMO, stands a chance of working in most situations. >>> Arbitrary schemes will be difficult to explain and defend when issues >>> arise. I dislike the idea of narrow or broad ranges for the reasons >>> you state. >>> >> >> >> Yes, "stands a chance of working in most situations", where "most" is the >> key word. We will work in all situations where applications dont install >> different versions of Tuscany's 3rd party libs. Once we have an >> application >> and Tuscany within one OSGi runtime with multiple versions of 3rd party >> libraries installed, there will always remain some situations where we >> dont >> work as expected. The default behaviour for maven-bundle-plugin (which we >> will probably end up using to generate our bundles) is [current.version, >> infinity). If we go out of our way to modify this for individual 3rd >> party bundles, I doubt that is in anyway going to be "arbitrary". It >> really >> needs to be based on real scenarios. >> >> Any non-infinite version number requires making some assumption > about the future compatibility of the 3rd party library. Without > a crystal ball, this will be hard to get right every time. This > is why I am suggesting a more conservative approach. > > [current.version, next.major.version) gives the best starting point IMO. In >> past notes about multiple extensions in Tuscany running with different >> versions of a 3rd party lib, I am not sure if the examples involved >> different major versions of a 3rd party lib. I would really like to >> understand real scenarios: >> >> 1. Why do we want two Tuscany extensions to use two different versions >> of >> a 3rd party library? Would these ever involve two minor versions at the >> same >> major version level? If so, couldn't both extensions just work with the >> higher of those two - do we really want to force one of them to work >> with >> the lower version? >> > > > This could happen if the Tuscany extensions are using the same 3rd party > library in different ways, such that one extension can't tolerate the > upgrade to a higher level of the library but the other can tolerate it. The problem is I have no idea how this can be generalized. If we have extension X that works with foo.jar version 1.2.3, and extension Y that works with foo.jar version 1.2.6, what exactly stops extension X from working with foo.jar 1.2.6? Is this a very special case that happens very rarely? Or is this the general assumption that we should make for every library that we ever use - that libraries at different revision levels or different minor version levels may not be compatible? To me this is the equivalent of saying that Tuscany version x.y.z is built against this combination of 149 3rd party libs, and we want to prevent the accidental use of any other version of 3rd party lib. And for the next version of Tuscany, we provide a brand new list of 3rd party libs, but again restricting to a single combination of 149 libs. I find it difficult to believe that application developers using Tuscany will find that acceptable. > 2. Why would an end user want to run Tuscany and an application with two >> different versions of a 3rd party library? Would these ever involve two >> minor versions at the same major version level? If so, what stops both >> from >> using the higher of the two? >> >> If the user is developing the application herself, that should not occur. > If it's developed by someone else, this situation could apply. The user > might need Tuscany level X and Other Application level Y. These might > have different version dependencies on the same 3rd party library. Again, same as my previous comment above - either we need to accept that our versioning will be as conservative as possible as you suggest, since we dont believe that any of our 3rd party libraries will remain backward compatible, or we need to understand which libraries are likely to become incompatible at minor version levels and why. Simon > > >> >> 2. You asked if we should be able to work with bundles from other >>> projects (e.g. ServiceMix, SpringSource Application Platform). I >>> wonder if we're able to learn from these projects to seed our >>> versioning. The SpringSource repository, for example, seems to allow >>> this, although I'm no legal expert [2]. >>> >>> Regards, Graham. >>> >>> [1] http://commons.apache.org/releases/versioning.html >>> [2] >>> >>> http://www.springsource.com/repository/app/faq;jsessionid=3F9467729AC282FE4E08199FDCE40863#q6 >>> >>> >>> >>> 2008/6/11 Rajini Sivaram <[EMAIL PROTECTED]>: >>> >>>> Following on from the discussion on OSGi-enabling third party libraries >>>> ( >>>> http://markmail.org/message/snltdk2yovr6maq5), this thread addresses >>>> the >>>> options for versioning Tuscany bundles and 3rd party libraries >>>> >>> distributed >>> >>>> with Tuscany and the implications of choosing these options. I have put >>>> together some notes on the wiki ( >>>> >>>> >>> http://cwiki.apache.org/confluence/display/TUSCANYWIKI/Tuscany+Versioning >>> ) >>> >>>> There were two outstanding questions from Simon Nash in the previous >>>> discussion which I will summarize here to ensure that they are not lost >>>> >>> in >>> >>>> this discussion. >>>> >>>> 1. Why can't we generate import constraints which will suit all >>>> applications? >>>> 2. *I'm concerned by the assumption here that Tuscany's versions of 3rd >>>> party bundles will be used both by Tuscany and by applications. An >>>> application may be using other software as well as Tuscany, and this >>>> >>> other >>> >>>> software may include its own versions of bundles for javax.servlet or >>>> >>> jaxb. >>> >>>> If Tuscany requires its versions of these bundles to be used, and the >>>> >>> other >>> >>>> software requires its versions to be used, this requires the >>>> >>> application >>> >>>> developer to understand how to resolve any conflicts.* >>>> >>>> The answer to 1) relates to how broad (or narrow) version ranges in >>>> >>> imports >>> >>>> are. Broad ranges prevent isolation and reduce scope for side-by-side >>>> execution, narrow ranges prevent class sharing and upgrading to newer >>>> versions. There is more detail with examples on the wiki. >>>> >>>> Question 2) is addressed first on the wiki (Figure 1 and Figure 2 show >>>> >>> these >>> >>>> scenarios). I would personally like to follow OSGi best practice and >>>> >>> enable >>> >>>> maximum sharing. There are some cases where we have no choice but to >>>> >>> share >>> >>>> (eg. SDO). I don't believe we can eliminate conflicts altogether - but >>>> following standard practice will make it less complicated for OSGi >>>> developers to resolve conflicts. >>>> >>>> Thoughts? >>>> >>>> >>>> Thank you... >>>> >>>> Regards, >>>> >>>> Rajini >>>> >>>> >> >> >> > -- Thank you... Regards, Rajini
