hi Alex, that sounds fair enough. We'll want to add good documentation around what it means if *everything* is now a bundle (and how we wrap plain catalogs into bundles on users' behalf). We can use the text of your email as a starting point for the docs; it would be good to explain some of the _motivation_ around the choice to make everything a bundle, rather than just say what we do.
I do like the idea of version ranges in brooklyn.libraries, but agree it's best to proceed in small steps so that would be a later development. Some more questions on implementation details - is it part of this plan to move bundle storage into the persistence location so that HA Brooklyn nodes can find all bundles? - if a catalog.bom (symbolic name xyz) includes brooklyn.libraries and specifies a remote URL (which gets downloaded), what happens if we delete the bundle xyz - do the downloaded bundles get "cascade deleted"? Geoff On Tue, 20 Jun 2017 at 17:22 Alex Heneveld <[email protected]> wrote: > > Hi Geoff > > Thanks. Replies to specific points: > > > whether we are "leaking" an implementation detail (the fact that we > use an OSGI runtime) > > My motivation was to try to minimize this leak. The fact that we use > OSGi internally for > dependency isolation forces us to use OSGi versions with bundles. But > most people > most places use the "-qualifier" suffix not ".qualifier", so to > accommodate that the > proposal suggests a recommended syntax in line with what most people do > ("-qualifer") > which we can internally safely convert to an OSGi version when needed. > > There is no need for authors ever to see OSGi and the docs PR doesn't > mention OSGi > until an "advanced" section. > > > proposal would rely on Brooklyn moving exclusively to Karaf > > No. The bundle version format is already a problem with the use of > embedded Felix > (it solves the problem noted in #672 about needing to use ".qualifer" in > your catalog.bom) > > > libraries / required-bundle / version ranges > > Where users want to indicate dependent bundles they currently have the > choice of > "brooklyn.libraries" (non-osgi) or "require-bundle" (osgi). Both work. > We don't need to disable the > former. I'm in two minds about whether we'd want to, at some point, but > not right now of course. > > An argument for disabling is that it sets up a single canonical way to > do things. Yes it > means people have to learn OSGi but this is a fairly advanced use case > and OSGi is the right tool for the job. > > If we didn't, it almost certainly means adding version range support in > `brooklyn.libraries` which > is yet more duplication of what OSGi lets us do. However it would be > easier for non-OSGi users > (and a lot of OSGi users I bet!) if we did invest in a nice range syntax > in `brooklyn.libraries`. > > Note `brooklyn.libraries` is the only place where you can specify a URL > for a dependent bundle; > the OSGi manifest doesn't let you do this (though if using Karaf you can > configure it to look in a set of repos, > which is possibly). > > It boils down to how we expect people to manage the installation of > multiple catalog bundles > (URLs v configured repos v something else ... maybe KAR archives which > could easily be supported > but they're wasteful if the same dependency is included in many KARs). > > Not sure where we'll want to go with this but it's not something we need > to solve for this proposal. > > > HTH > > Best > Alex > > > On 20/06/2017 17:00, Geoff Macartney wrote: > > hi Alex, > > > > In general I think this is a good direction to go in, some questions and > > thoughts below. > > > > One concern is whether we are "leaking" an implementation detail (the > fact > > that we > > use an OSGI runtime) into the specification of what Brooklyn is. I think > > it is good to > > aim for the improved modularity of having a bundle of catalog definitions > > and related > > resources, all at a given version and added or removed as a unit. I'm > less > > sure we > > want to advertise that all Brooklyn definitions must be contained in OSGI > > bundles, > > even if we will wrap them for you. This seems to impose an additional > > requirement > > on blueprint authors (learn OSGI) that we don't really want to impose? > > > > As an aside, I'm assuming that your proposal would rely on Brooklyn > moving > > exclusively to the Karaf launcher, is that the case? > > > > Specific notes inline below. > > > > Geoff > > > > On Mon, 19 Jun 2017 at 17:39 Alex Heneveld < > [email protected]> > > wrote: > > > >> Hi All- > >> > >> With bundles now being neatly supported for add and remove, I think it > >> makes sense to use make the "bundle" a first-class concept in managing a > >> brooklyn catalog (type registry). > >> > >> The basic idea is that users will work with ZIPs (or directories if > >> using the br CLI), where a "catalog.bom" is in the root; this is a > >> bundle, and operations like add/remove/update apply to the bundle. > >> Increasingly people are doing this for additions, so it seems pretty > >> obvious that delete should follow suit, and something like a "/bundles" > >> REST endpoint is appropriate. > >> > > I do like this; as part of this we should re-instate the automatic > scanning > > of bundles for catalog.bom (feature FEATURE_LOAD_BUNDLE_CATALOG_BOM). > > Now that catalog items can tell you their containing bundle > > (getContainingBundle()) > > it should be possible to make this work with rebind. > > > > > > > >> You can still upload just a catalog.bom file; in this proposal it will > >> be wrapped in a bundle (internally, hidden from user). A bundle name > >> and version will have to be inferred/guessed for backwards > >> compatibility, and probably we should deprecate supplying a BOM which > >> doesn't define a bundle name and version. > >> > > +1 > > > > > >> If a ZIP/folder is supplied, it doesn't have to contain any OSGi bundle > >> definition; that call all be inferred. If a MANIFEST.MF is supplied, > >> any bundle name/version given there must of course match that in the > >> catalog.bom file. > > > > > > > >> If dependent bundles are declared, they are obeyed at > >> runtime and can be used to find things (pretty much identically to how > >> we treat "libraries" in the catalog.bom.) > >> > > Not so sure about that last paragraph - again this is related to leaking > > the use > > of OSGI. This would allow someone to write a catalog.bom that would ONLY > > work when supplied along with a related manifest (to specify certain > > dependencies > > and so set up the search paths). Do we want to allow that, or should all > > definitions > > required for entities in a catalog be supplied in the catalog.bom (or its > > included > > catalog definitions). > > > > > > > >> * Search Paths - Bundles fix an issue where types have implicit version > >> dependencies. Consider an "acme-cluster" of "acme-node" entities, > >> defined in a bundle. If I release a 1.0 version of this, and then a 2.0 > >> version, I'd probably expect the 1.0 acme-cluster to load 1.0 acme-node > >> entities, and same for 2.0. But currently any reference to "acme-node" > >> takes the latest version, even if there is a different version in the > >> same bundle. Shifting to bundles, I think we should change this, so > >> that versions of items in the same bundle (or on the OSGi or library > >> search path for that bundle) are preferred over more recent versions in > >> the repository. > > > > +1 this sounds like a better match to what blueprint authors intend > > > > > >> (And since we will follow OSGi loading, we can also, if > >> we need to, use OSGi config to say that "acme" bundle requires a given > >> version _range_ for some other bundle, to have more control over > >> dependency versions when required.) > >> > > again, how "use OSGI config" - there may be value in specifying ranges > but > > we wouldn't want users to have to use OSGI declarations to achieve it. > > > > > > > >> For most users and blueprints this will cause no changes. However there > >> are some changes to note: > >> > >> * If people have been sloppy with versions or metadata, in most cases > >> they will get a warning > >> * In some cases if uploading a ZIP, the catalog.bom may need to be > >> edited (the upload will give a clear error, and it should not expect > >> rebind) > >> * When a type is referenced with no version, the loading semantics may > >> change if that type is in the same bundle (this is probably what the > >> user expects) > >> > >> Does anyone oppose any of the above, or have further questions? > >> > >> Best > >> Alex > >> > >> > >> [1] https://github.com/apache/brooklyn-server/pull/672 > >> > >> > >> > >
