Hi Raymond comments in line...
Simon On Mon, Jan 19, 2009 at 5:32 PM, Raymond Feng <[email protected]> wrote: > Hi, > > I'm catching up the discussions and I have to admit the thread has too many > levels of nesting and it becomes difficult to read. Here is my view: > > 1) The structure of the distribution: > > I see this as a repository of the Tuscany modules, 3rd party jars and other > things for the Tuscany standalone runtime and other environments where > Tuscany is embedded. IMO, we should keep the structure simple and > independent of the grouping. I'm happy with the current approach in 2.x: > > Agreed. It's just a local repo. Do you think adding a little structure add technical difficulty or is a flat structure a personal preference? I'm interested in situations like this where we have some people who want solution A and others want solution B (where both solutions are valid). How do we come to a conclusion? In the past this has tended to stall us a little so this is a good chance to see if we can do better;-) > modules > +--- tuscany-assembly-2.0.0.jar > +--- tuscany-assembly-xml-2.0.0.jar > +--- stax-api-1.0-2 > +--- META-INF/MANIFEST.MF > +--- stat-api-1.0-2.jar > > I'm against adding extra structures for the libs folder to achieve the > grouping. > > 2) Grouping of modules into functional features > > This should be a very light layer on top of 1). It should be just a set of > configurations to organize the modules by function. When another > distribution is downloaded, it should be possible to just unzip it into the > existing distribution without any conflicts. > > For the 1st milestone of 2.x, we could try to get "api", "core" and > potentially "webservice" distributions out. > These all sounds like fine collections of function to me but can we go with the "all" distro for M1 where all = api + core + webserivce. It seems we can agree on "all". It seems we can agree that we need to be able to point samples etc at more coarsly grained modules, we don't seem to be able to agree whether these coarser grained modules should be distributed. Once we know how this works and have tried to create the distribution we may take a different view. > > 3) Configuration of the features for specific hosting environments > > * Maven: A pom project that list the set of modules for the functional > feature > * JSE/WebApp: generate a launcher jar whose META-INF/MANIFEST.MF that uses > the Class-Path header to define the classpath entries > Will the launcher be required to start the OSGi runtime? > * OSGi: Generate a config.ini which lists the set of bundles to be used by > the OSGi runtime > * Eclipse: Generate a target platform definition or feature.xml > > Thanks, > Raymond > > *From:* Simon Laws <[email protected]> > *Sent:* Monday, January 19, 2009 7:25 AM > *To:* [email protected] ; [email protected] > *Subject:* Re: [2.0] Align samples with the distributions > > > > On Sat, Jan 17, 2009 at 8:53 AM, ant elder <[email protected]> wrote: > >> >> >> On Fri, Jan 16, 2009 at 2:01 PM, Simon Laws >> <[email protected]>wrote: >> >>> >>> >>> On Fri, Jan 16, 2009 at 1:04 PM, ant elder <[email protected]> wrote: >>> >>>> >>>> >>>> On Fri, Jan 16, 2009 at 12:28 PM, Simon Laws < >>>> [email protected]> wrote: >>>> >>>>> snip.... >>>>> >>>>>> >>>>>> Yes that what the current compact distribution does, its in 1.x right >>>>>> now as there's few extensions going in 2.x but there should be no >>>>>> problems >>>>>> with doing it there too. >>>>>> >>>>>> ...ant >>>>>> >>>>>> I looked at the compact distro in 1.x and the distro projects in 2.x >>>>> and this is what I found: >>>>> >>>>> Key >>>>> Plain text = common between the two >>>>> [] = different between the two >>>>> * = my opinion >>>>> >>>>> A set of maven modules which define dependencies for features >>>>> [in /modules or /distribution] >>>>> * I prefer /distribution as the feature modules are not providing >>>>> any new function >>>> >>>> >>>> There's pros and cons, where they are right now enables a facility i've >>>> not mentioned yet but the where they go is not big deal at the end of the >>>> day. >>>> >>>> >>>>> >>>>> Samples depend on only the feature modules that they require >>>>> >>>>> An "all" distribution that ships all features >>>>> >>>>> [separate distributions that ship individual features] >>>>> * I don't mind having them if people want to support them. >>>> >>>> >>>> The problem with doing that is it adds some significant complexity, from >>>> things like the download page, to things like the documentation for samples >>>> and how to use etc. >>>> >>> >>> As I say below I don't think we need to decide for M1. I would assert >>> that we need to be able to have samples depend on separate feature >>> distributions, modules call then what you will. Doesn't mean we need to ship >>> them but does mean they need to exist. As in my first point I think this is >>> a common feature of both approaches as they are currently implemented. >>> >> >> I don't think that would work very well, the "feature" approach is not a >> feature per extension type so its only really useful for making >> downloadable distributions. That means for example a sample using xqueryyou >> have to know >> xquery is in the "process" feature distribution and download that, but >> you wouldn't put a maven dependency on the process feature in the >> xquerysample as that would drag in all the other unnecessary dependencies >> like >> bpel and ode, same thing for say a jsonrpc sample which would have a >> dependency on the jsonrpc binding not the web20 feature as that would >> drag in all the scripting stuff. The feature idea was that there would be a >> small number of downloadable feature distributions as a way to manage the >> growing number of Tuscany extensions, the problem is though that there are >> no obvious groupings for most of the extensions so the features get quite >> arbitrary as more extensions are include which makes them unintuitive which >> just further complicates an already complicated project. Since they were >> proposed we've asked on the dev list, user list, and in the user survey >> and pretty much everyone is happy with the one big distribution, i think a >> 70Meg or so download is just not a big deal these days. What people do seem >> to want is less jars and an easier way to know which jars are for what, and >> the compact jars and structured lib folder do that. Even if we didn't use >> the compact jars, the structure lib folder makes a single distribution much >> more usable, its obvious what jars are for and users can easily make up >> their own custom distributions if required by just deleting unneeded lib >> folders. >> > > So we seem to be getting to the heart of the matter. We tentatively agree; > > 1 - start with one "all" distro > 2 - add some structure to the libs dir > 3 - have some grouping of jars that group modules together for easy > reference from samples etc. > > On point 3 the disagreement is the granularity of the grouping. Features vs > extensions/compact. > > Features = small number of features that group more that one extension > Extension/compact = larger number grouping each extension implementation > > I'm not against having a finer granularity of feature > > >> >> >>> >>> >>>> >>>> >>>>> >>>>> * I suggest though in this first instance (M1) we just ship the "all" >>>>> distro and we can see how we get on with that. This is still build from >>>>> the >>>>> separate feature modules. >>>>> >>>> >>>> Yes i agree with just shipping an "all" distro for now. I'd like to >>>> leave the "how its built" till this discussion has progressed a little >>>> further though. >>>> >>>> >>>>> >>>>> [separate tuscany jars vs feature jars] >>>>> * I don't particularly like the feature jars as it's another step to >>>>> go wrong. In particular it uses the shade(?) plugin and you have to >>>>> configure transformers. >>>> >>>> >>>> I'm puzzeled that you consider that as "another step to go wrong" but >>>> don't mind having multiple different distributions or building >>>> distributions >>>> out of each other, both of which seem to add much more complexity, >>>> restrictions, and points of failure to me :) >>>> >>> >>> To me it looks like a step in the process that we don't necessarily need. >>> In the context of this point my measure of complexity is how many places I >>> have to specify things. I agree that there are also complexities in >>> distributing more that one distribution package but I don't see that that's >>> particularly related to this point. >>> >> >> I'm still not understanding the problem here, adding a shade transformer >> config is a pretty easy task, we've been doing it for years in the past >> Tuscany releases and i don't remember a single occasion where its caused any >> problem. And its only an occasional task for a Tuscany "developer" which >> makes the "users" life easier every time they use Tuscany - and users have >> complained about Tuscany having so many jars on lots of occasions. Also, if >> thats the only hurdle in doing this then all we need to do is write a >> relatively trivial Tuscany specific extension of the shade plugin and we >> then wouldn't even need that transformer configuration. We're getting quite >> a lot of Tuscany specific plugins these days so one more is no big deal. >> >> > Personally I've had a lot of problems with it. 50% of the time it fails in > my environment (having sais that I haven't tried is since I moved to the Sun > JDK). Anyhow the utility of this plugin to us will be dictated by the shape > we want our distribution to take. If we have a single distribution then it > seems we need to ship separate module jars as the OSGi runtime is expecting > to load bundles. I guess we could write a manifest aggregator for the shade > plugin but why would we when we can ship the separate module jars and > provide a manifest jar to make manual classpath specification easier. Maybe > I'm missing something here. > > >> >>> >>> >>>> >>>> >>>> >>>>> >>>>> [Launcher vs manually specified classpath comes into this] >>>>> * Manual classpath is easier with feature jars. Can we use a >>>>> different approach to support the manual classpath? Manifest jars for each >>>>> feature? >>>> >>>> >>>> I think we need to make a bit more progress on the "Tuscany runtime >>>> launching" thread before that can be answered. >>>> >>> >>> Agreed >>> >>> >>>> >>>> >>>> >>>>> >>>>> >>>>> [structured libraries directory] >>>>> * I like this. It gives some sense of order to the distro lib >>>>> directory. >>>>> >>>>> >>>> Me too. Comparing the distros we've had in the past and the different >>>> varrieties being talked about now i think a structured libraries directory >>>> makes the most significant improvement for making the distro easier to use >>>> in the way users have been asking for. >>>> >>>> >>>>> >>>>> Some other distro thoughts came to mind as I went through: >>>>> >>>>> 1/ Can we do something about 3rd party licenses? E.g. automate the >>>>> command line tools we have that check distro jars are represented in the >>>>> LICENSE file so that this happens automatically when distro is built. Also >>>>> it would be nice to have a tool that checks that module NOTICE/LICENSE >>>>> files match the requirements of the source that they hold. This last is >>>>> manual at the moment. >>>> >>>> >>>> The way to do this would be to change to use the maven release plugin >>>> and IANAL plugin which would vastly simplify our distribution build and >>>> release process, i think it is where i think really we need to get to but >>>> it >>>> will put some restriction on how we shape the distributions which is one of >>>> the reasons i'm less keen on all these "feature" distros >>>> >>> >>> I looked at this a while back and I agree it adds restrictions. I don't >>> remember the details so I'll have to read up. I seem to think it was early >>> days when I looked at it so it was pretty rough at that point. >>> >> >> I think its pretty good these days, I know Geronimo uses it ok. >> >> >>> >>> >>>> >>>> >>>>> 2/ We need to follow through on the reason Raymond started this thread >>>>> originally and ensure that samples are tested in the ways that a user will >>>>> run then when the distribution is built. >>>>> 3/ As per 2/ but for webapp samples. Can we re-use some of the work >>>>> that was done for webapps in 1.x itest? >>>>> >>>> >>>> I'm starting to think there should be seperate distributions for OSGi >>>> and JSE/Webapps etc. Trying to munge all this into a single distribution >>>> that works for all environments so far looks like its going to make the >>>> distribution a lot more complex and less user friendly than it needs to be. >>>> You could consider this part of the "modularity" exercise of 2.0 and not >>>> munging unrelated things together. If we did that then we can test the >>>> samples included in the distribution properly in the best way for the >>>> target >>>> environment. >>>> >>> >>> what would the difference be between these distributions? I think the >>> jars and their organization would be the same. It comes down to how you >>> launch/host the runtime so probably some hosting/launching jars would come >>> and go. Again we need to do more on the launching thread for this point. >>> >> >> Probably need actual distributions to show all the differences but for >> example the organization isn't the same - the OSGi distro requires the 3rd >> party jars be each included in its own sub folder which is unnecessary for >> other environments and would make things like "java -cp" a bit unwieldy. >> However, if we did go for the one common distro with a structure lib folder >> then this becomes less of an issue for Ant scripts and launchers. >> > > I suggest we lay out the distribution file directory in some detail and try > review it. Is this what we're talking about > > tuscany-sca-2.0 > bin > tuscany.bat (is this a bat or a jar?) > tuscany-default.conf > tuscany-osgi.conf > ... > lib > base > acitvation-1.1 > activation-1.1.jar > META-INF > MANIFEST.MF > asm-all-3.1.jar > tuscany-assembly-2.0.jar > tuscany-core-2.0.jar > tuscany-contribution-2.0.jar > ... > tuscany-base-manifest-2.0.jar > spring > commons-logging-1.1.1.jar > spring-2.5.5.jar > tuscany-implementation-spring-2.0.jar > tuscany-spring-manifest-2.0.jar > webservices > samples > calculator > tutorial > store > > >
