On 11/30/06, Pete Robbins <[EMAIL PROTECTED]> wrote:
On 30/11/06, Simon Laws <[EMAIL PROTECTED]> wrote: > > On 11/30/06, Pete Robbins <[EMAIL PROTECTED]> wrote: > > > > On 30/11/06, Andrew Borley <[EMAIL PROTECTED]> wrote: > > > > > > On 11/30/06, Pete Robbins <[EMAIL PROTECTED]> wrote: > > > > On 30/11/06, Simon Laws <[EMAIL PROTECTED]> wrote: > > > > > > > > > > On 11/30/06, Pete Robbins <[EMAIL PROTECTED]> wrote: > > > > > > > > > > > > On 30/11/06, Pete Robbins <[EMAIL PROTECTED]> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 30/11/06, Simon Laws <[EMAIL PROTECTED]> wrote: > > > > > > > > > > > > > > > > On 11/30/06, Pete Robbins <[EMAIL PROTECTED]> wrote: > > > > > > > > > > > > > > > > > > On 30/11/06, Pete Robbins <[EMAIL PROTECTED]> > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 30/11/06, Jean-Sebastien Delfino < > > [EMAIL PROTECTED]> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > Pete Robbins wrote: > > > > > > > > > > > > Our current method of packaging and loading an > > extension > > > is > > > > > > > > fairly > > > > > > > > > > > > simple: > > > > > > > > > > > > we load all schema and libraries in the extensions > > path. > > > > > This > > > > > > > > has a > > > > > > > > > > > > number > > > > > > > > > > > > of problems. > > > > > > > > > > > > > > > > > > > > > > > > 1. An extension may consist of more than one library > > e.g > > > . > > > > > > > > > > > > libmy_extension.so > > > > > > > > > > > > and libmy_extension_utils.so. Our current loading > > scheme > > > > > will > > > > > > > > > attempt > > > > > > > > > > > to > > > > > > > > > > > > load both of these and may fail on the one that > > doesn't > > > > > > provide > > > > > > > > the > > > > > > > > > > > > extension initialize method. On MacOS the output of > > our > > > > > build > > > > > > > > > produces > > > > > > > > > > > a > > > > > > > > > > > > libx.dylib and a series of symlinks to this called > > > > > > libx.0.dylib, > > > > > > > > > > > > libx.0.0.dylib etc.. ur runtime loads ALL of these > > which > > > > > > doesn't > > > > > > > > > cause > > > > > > > > > > > > problems as they are all the sam library and just > > > repeatedly > > > > > > > > > register > > > > > > > > > > > to > > > > > > > > > > > > handle the same requests. Very inefficient though! > > > > > > > > > > > > > > > > > > > > > > > > 2. Control of whether or not to load an extension > > > library is > > > > > > > > > currently > > > > > > > > > > > by > > > > > > > > > > > > renaming the library so the runtime doesn't find it. > > An > > > > > > example > > > > > > > > is > > > > > > > > > > > > that we > > > > > > > > > > > > ship our python extension as > > > > > libtuscany_sca_python.so.diabled. > > > > > > > > This > > > > > > > > > is > > > > > > > > > > > > > > > > > > > > > > > horrible and error prone. > > > > > > > > > > > > > > > > > > > > > > > > We could improve this by having a system > configuration > > > file > > > > > > that > > > > > > > > > lists > > > > > > > > > > > > the > > > > > > > > > > > > required extensions but the I like the self > contained > > > > > package > > > > > > > > > approach > > > > > > > > > > > > that > > > > > > > > > > > > we have now. I'd like to implement an improved > scheme > > > for > > > > > > > > packaging > > > > > > > > > an > > > > > > > > > > > > extension by introducing a per extension > configuration > > > file. > > > > > > > > > Something > > > > > > > > > > > > like: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > tuscany_sca_ws_binding.extension > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > <scacpp:extension name="ws binding" enabled="true"> > > > > > > > > > > > > <library name="tuscany_sca_ws_reference"/> > > > > > > > > > > > > <library name="tuscany_sca_ws_service"/> > > > > > > > > > > > > </scacppp:extension> > > > > > > > > > > > > > > > > > > > > > > > > So the package would look like: > > > > > > > > > > > > > > > > > > > > > > > > extensions/ > > > > > > > > > > > > ws/ > > > > > > > > > > > > tusany_sca_ws_binding.extension > > > > > > > > > > > > lib/ > > > > > > > > > > > > xsd/ > > > > > > > > > > > > other_folder/ > > > > > > > > > > > > ... > > > > > > > > > > > > > > > > > > > > > > > > The .extension configuration file is saying to load > > the > > > > > > library > > > > > > > > > which > > > > > > > > > > > is > > > > > > > > > > > > located somewhere in the package... the runtime will > > > find > > > > > > it... > > > > > > > > no > > > > > > > > > > > > need to > > > > > > > > > > > > specify a path. > > > > > > > > > > > > > > > > > > > > > > > > Taking this further the configuration file could > list > > > the > > > > > > schema > > > > > > > > to > > > > > > > > > be > > > > > > > > > > > > loaded. Currently the runtime will just load any it > > > finds > > > > > but > > > > > > > > these > > > > > > > > > > > > may not > > > > > > > > > > > > be needed by the runtime e.g. the schema may be for > > some > > > > > > > > extension > > > > > > > > > > > > implementation specific purpose. > > > > > > > > > > > > > > > > > > > > > > > > I think it would also be good for the extension > > > > > > initialization() > > > > > > > > > > > > method to > > > > > > > > > > > > take as a parameter the root of the extension e.g. > > > > > > > > > > > > extension("/tuscany/extensions/ws"). This would > allow > > > the > > > > > > > > extension > > > > > > > > > > > > package > > > > > > > > > > > > to contain any configuration information that it > > needs. > > > > > > > > > > > > > > > > > > > > > > > > I'd like to start by at least introducing the > > .extension > > > > > file > > > > > > > > for > > > > > > > > > each > > > > > > > > > > > > extension and loading only the specified > library(ies) > > if > > > the > > > > > > > > > extension > > > > > > > > > > > is > > > > > > > > > > > > enabled. > > > > > > > > > > > > > > > > > > > > > > > > Any thoughts? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Two thoughts: > > > > > > > > > > > - convention over configuration > > > > > > > > > > > - the runtime should be consumable without having to > go > > > tweak > > > > > > XML > > > > > > > > > > > configuration files > > > > > > > > > > > > > > > > > > > > > > If I remember correctly, renaming the dlls to > .disabled > > > was a > > > > > > last > > > > > > > > > > > > > > > > > > > minute change to work around DLL loading errors with > our > > > M2 > > > > > > > > release on > > > > > > > > > > > Windows. > > > > > > > > > > > > > > > > > > > > > > I agree that we should do better than renaming to > > > .disabled, > > > > > but > > > > > > > > I'd > > > > > > > > > > > like to understand better the actual issues that we > > faced > > > > > before > > > > > > > > > > > inventing yet another XML based runtime configuration > > > > > language > > > > > > :) > > > > > > > > > > > > > > > > > > > > > > I'm aware of the following issues: > > > > > > > > > > > 1. We need the runtime to load extension libs only, > not > > > other > > > > > > libs > > > > > > > > > under > > > > > > > > > > > > > > > > > > > > > > the extension directory which are not actual > extensions > > > > > > > > > > > 2. Same for XSDs, we need to load XSDs that contribute > > to > > > > > SCDL, > > > > > > > > and > > > > > > > > > > > leave other XSDs under the extension directory alone > > > > > > > > > > > 3. Extensions that cannot be loaded because some of > > their > > > > > > > > dependencies > > > > > > > > > > > are not there should no break the runtime > > > > > > > > > > > 4. The system admin / installer should be able to > > disable > > > > > > > > extensions > > > > > > > > > > > that won't load because their dependencies are not > there > > > (I'm > > > > > > not > > > > > > > > yet > > > > > > > > > > > convinced that this is still an issue if we manage to > > > solve > > > > > > issue > > > > > > > > #3) > > > > > > > > > > > > > > > > > > > > > > Did we run into any other big issues? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > No, that's about it. > > > > > > > > > > > > > > > > > > > > 2. XSD loading can be done by convention (schemas for > the > > > > > runtime > > > > > > > > are > > > > > > > > > > always in a folder called 'xsd' > > > > > > > > > > 1. Could be solved in a similar way by only loading > > > libraries in > > > > > a > > > > > > > > > folder > > > > > > > > > > called ??? > > > > > > > > > > 3. Can be solved by just ignoring the load > errors/issuing > > a > > > > > > warning > > > > > > > > > rather > > > > > > > > > > than giving up > > > > > > > > > > 4. Can be solved by solution to 3. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I recall now why 3. was a big problem. Windows sometimes > > > throws up > > > > > > an > > > > > > > > > error > > > > > > > > > dialog when the load fails so it was not just a case of > the > > > > > runtime > > > > > > > > > handling > > > > > > > > > the load failure so we had to "disable" the extension. > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Pete > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Pete > > > > > > > > > > > > > > > > > > Pete, just summarizing so that I understand. It seems > there > > > are > > > > > two > > > > > > > > parts > > > > > > > > to this. > > > > > > > > > > > > > > > > A/ Building and installing an extension in order that it can > > be > > > > > > consumed > > > > > > > > by > > > > > > > > the runtime. Building and installing extensions is > potentially > > a > > > > > > > > separate > > > > > > > > activity from building and installing the runtime itself. > > > Looking at > > > > > > the > > > > > > > > M2 > > > > > > > > download they all come together at the moment. > > > > > > > > > > > > > > > > > > > > > Yes. It just happens that we are the only providers of > > extensions > > > at > > > > > the > > > > > > > moment. > > > > > > > > > > > > > > B/ Optionally consuming/enabling an extension in the runtime, > > once > > > the > > > > > > > > extension has been installed, in such a way that the runtime > > is > > > able > > > > > > to > > > > > > > > find > > > > > > > > and load it. > > > > > > > > > > > > > > > > > > > > > Yes. > > > > > > > > > > > > > > If I understand the previous posts (and the current code) A/ > is > > > > > achieved > > > > > > > > by > > > > > > > > building the extension in the deployed folder layout > > > > > > > > extension type/ > > > > > > > > bin > > > > > > > > lib > > > > > > > > xsd > > > > > > > > > > > > > > > > > > > > > The current runtime does not mandate any layout. It will seek > > and > > > > > > > destroy.. I mean seek and load any library it finds and any > .xsd > > > it > > > > > > finds. > > > > > > > That layout is just how we package extensions in ou build at > the > > > > > moment. > > > > > > > > > > > > > > > > > > > > > and Pete you are proposing that this is adjusted so that there > > are > > > > > > > > directories that hold just the stuff that the contains the > > > extension > > > > > > > > rather > > > > > > > > than the things it depends on. > > > > > > > > extension type/ > > > > > > > > bin > > > > > > > > extensionbin? > > > > > > > > lib > > > > > > > > xsd > > > > > > > > extensionxsd? > > > > > > > > > > > > > > > > You are also proposing that B/ is achieved by ensuring that > > this > > > > > > > > directory > > > > > > > > structure be placed in a location that the the runtime can > > > search > > > > > for > > > > > > > > active > > > > > > > > extensions (as it does at the moment in the deploy > directory) > > > > > > > > > > > > > > > > > > > > > it currently loads any extensions from under the > > > > > > > <sca_install_dir>/extensions folder > > > > > > > > > > > > > > and that the > > > > > > > > runtime ignores any badly configured extensions if possible. > > > > > Removing > > > > > > an > > > > > > > > extension's directory from the deploy > > > > > > > > > > > > > > > > > > > > > extensions not deploy > > > > > > > > > > > > > > directory structure has the effect of > > > > > > > > disabling it so I guess this is the fallback if the runtime > > > can't > > > > > > > > continue. > > > > > > > > > > > > > > > > > > > > > That is true in today's implementation. > > > > > > > > > > > > > > Anything more complex and, as Jean-Sebastien suggests, you > start > > > > > getting > > > > > > > > into the full blown package and dependency management > problem > > > that > > > > > > many > > > > > > > > other systems try to solve in different ways, e.g. rpm or > > pear. > > > > > > > > > > > > > > > > > > > > > Agreed... I'll give up on that plan ;-) > > > > > > > > > > > > > > > > > > > > > What we need is to enable the runtime to identify which > > extension > > > > > > > libraries to load and which schema to load. My suggestion is: > > > > > > > > > > > > > > my_extension/ > > > > > > > bin/ > > > > > > > extension/ > > > > > > > library_that_will_be_loaded_by_the_runtime.so > > > > > > > lib/ > > > > > > > lib_my_extension.so > > > > > > > include/ > > > > > > > ... some headers maybe > > > > > > > xsd/ > > > > > > > my_binding.xsd > > > > > > > any_other_folder/ > > > > > > > any_stuff_I_like.xsd > > > > > > > > > > > > > > So the runtime will load any xsd that is in a folder named xsd > > and > > > > > > attempt > > > > > > > to load any library in a folder called extension. It would not > > > attempt > > > > > > to > > > > > > > load the any_stuff_I_like.xsd or the lib_my_extension.so > > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > > > > -- > > > > > > > Pete > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Pete > > > > > > > > > > > > Sounds like the right idea to me. What goes in the bin dir?. > > > > > > > > > > > > > > > > Anything you like! > > > > > > > > > On windows, .exe's and .dll's are often found in bin, whereas the > > > .lib's (used for linking against at compile time) are generally found > > > in lib. Some projects (e.g. Axis2C) put dll's and lib's in lib (which > > > is more unix-like, and, IMO, nicer) > > > > > > > > > > > Should the > > > > > lib be... > > > > > > > > > > lib/ > > > > > lib_my_extension.lib > > > > > > > > > > > > lif you like... again... anything you like. > > > > > > > > Maybe you could stick with bin,include, lib,xsd and then have your > > > "other" > > > > > directory have everything else in it that the extension might rely > > on. > > > > > Depends what you now anticipate being in bin and lib over and > above > > > the > > > > > actual library exposing the extension. > > > > > > > > > > Will you continue to mandate a specific place where extensions are > > > > > installed, i.e. from you post <sca_install_dir>/extensions folder? > > > > > > > > > > > > Yes > > > > > > > > So that > > > > > you can move an extension out of there if you don't want it to > load. > > > > > > > > > > > > Well the idea here is you don't need to remove the whole > package..just > > > the > > > > library from the extension dir > > > > > > > > Simon > > > > > > > > > > > > > > The layout of the extension package is whatever the package creater > > > wants to > > > > make it. We don't care. All we will care about is loading any > library > > in > > > a > > > > folder anywhere in the extension package that is in a folder called > > > > 'extension' and expecting it to implement the extension initialize > > > method. > > > > We will also load any schema files in any folder called 'xsd'. An > > > extension > > > > only needs the xsd folder if it has schema required for the runtime > > > model. > > > > > > > > As an example here is the ws binding extension implemented for > axis2c: > > > > > > > > <sca_install>/extensions/ > > > > ... somewhere in this tree... could be anywhere... > > > > ws/ > > > > xsd/ > > > > ws-binding.xsd > > > > reference/ > > > > lib/ > > > > libtuscany_sca_reference.so > > > > service/ > > > > lib/ > > > > libtuscany_sca_service.so > > > > > > > > > > > > OK... today the runtime would load the ws-binding.xsd as it loads > any > > > xsd it > > > > finds. It would also load any .so it finds so both reference aand > > > service > > > > extensions would be loaded/initialized. > > > > > > > > With my proposed change the xsd would be loaded (as it's in a folder > > > named > > > > 'xsd' ) but the libraries would not. To get the extensions loaded > you > > > would > > > > have to package with an 'extension' folder ... again anywhere in the > > > tree... > > > > which "contains" the libraries to load eg. > > > > > > > > ws/ > > > > extension/ > > > > libload_this.so -> reference/lib/libtuscany_sca_reference.so > > > > xsd/ > > > > ws-binding.xsd > > > > reference/ > > > > lib/ > > > > libtuscany_sca_reference.so > > > > service/ > > > > lib/ > > > > libtuscany_sca_service.so > > > > > > > > Here the runtime loads anything in any folder named 'extension' so > the > > > > reference extension would be loaded/initialized. > > > > > > Windows doesn't really do symlinks - I guess on windows we'd have the > > > tuscany_sca_reference.dll in extension/ and nothing in reference/lib/ > > > or reference/bin/? For extensions that have dependencies on other > > > libraries, those dll's could go in lib/ > > > > > > I would actually have a copy of the dll in extension on windows and > delete > > it to disable. For a large extension it would be very wasy to make a > tiny > > dll which just implements the initialise interface expected by the > runtime > > and leave everything else in a separate dll in bin/lib. > > > > > I would package the cpp language extension as > > > > > > > > cpp/ > > > > bin/ > > > > lib/ > > > > include/ > > > > xsd/ > > > > extension/ > > > > > > > > The bin, lib, include are exactly what you would expect from a > package > > > that > > > > you might want to build against. The extension/ folder would hold > the > > > > library that implements the extension interface and xsd holds... > well > > > xsds! > > > > > > > > > > All sounds good to me. +1 > > > > > > Andy > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > > > > -- > > Pete > > > > Ok, so I think I get it. My concern is that giving the developer the > flexibility to put "extension" and "xsd" anywhere they like means that > they > will accidentally duplicate these directories as these are pretty common > names for directories. So we could ask them always to have them at the top > level. Doesn't remove the possibility for duplication but reduces the risk > a > little. Or we could make the names a little more exotic with some > qualification "sca_extension", "sca_xsd" for example, and maintain the > flexibility you want. > > Simon > >Yes... the names are always the hardest thing to come up with! They should not be sca_ as this is Tuscany specific. so... we could get away with just the one folder tuscany_extension that contains the schema and libraries: cpp/ bin/ lib/ include/ xsd/ tuscany_extension/ <- schema and libraries implementing extension i/f live under this tree xsd -> ../xsd <- could even be a symlink to where they really are I like that better. Runtime will load library and schema from any subtree under <sca_install>/extensions that is named 'tuscany_extension' and ignore anything else. Nice and flexible.
Works for me. +1 --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
