+1
Some thoughts:
* (A) add a utility class BundleMaker
Sounds very similar to
https://ops4j1.jira.com/wiki/display/ops4j/Tinybundles <
https://ops4j1.jira.com/wiki/display/ops4j/Tinybundles>
Looking at the code it's much more focused on zip files so I guess
there's no much overlap, but worth keeping in mind
* (C) accept bundle symbolic name and version
Why require them at all? Could infer them from the catalog.bom in some
way - maybe require those properties to be in there. If not present are
they really needed?
* (G) Bundles installed via this mechanism are not persisted currently &
(I) We persist the individual catalog items as YAML, so we end up with two
records
Suggest marking the catalog items coming from bundles as
non-persistable. Then try to share the bundles between HA nodes. (Karaf
Cellar?)
* (J) Introduce a catalogGroupId field on catalog items;
Agree this could be useful and I like the idea of deleting the bundle
altogether with the catalog items. From user's perspective I don't see the
need for an extra field (i.e. it's an implementation detail).
Svet.
On 16.12.2016 г., at 12:50, Alex Heneveld <
alex.henev...@cloudsoftcorp.com> wrote:
Hi Brooklyners-
In the code we currently have two routes for users to install new
blueprints:
(1) upload a catalog YAML file to /v1/catalog
(2) install a bundle with catalog.bom in the root
The feature (2) is disabled by default, but I'd like to move towards
enabling it. This will make it easier to create nicely structured BOM
files because scripts etc can be taken out of the BOM, stored as files in
the same bundle. (Because URLs of the form
`classpath://scripts/install.sh` use the bundle's classpath to resolve.)
As a first step in #485 [1] I do a few things:
(A) add a utility class BundleMaker that lets us create and modify
bundles/zips, to make it easier to do things we might want to with
bundles,
especially for testing
(B) add an endpoint to the REST API which allows uploading a bundle ZIP
(C) accept bundle symbolic name and version in that REST API to
facilitate
uploading non-bundle ZIPs where the OSGi MANIFEST.MF is automatically
generated
With this PR, if you have a directory on your local file system with
scripts and config files, and a BOM which refers to them, you can just
ZIP
that up an upload it, specifying the bundle name so that a YAML blueprint
author never needs to touch any java-isms.
Where I see this going is a development workflow where a user can edit
files locally and upload the ZIP to have that installed, and if they make
changes locally they can POST it again to have catalog items updated
(because default version is a SNAPSHOT). We could also:
(D) have `br catalog add ~/my/project/ --name my.project` create a ZIP
and
POST it, with bundle name metadata, so essentially the user's process is
just to run that whenever they make a change
(E) have a mechanism whereby deployed entities based on an affected
blueprint are optionally migrated to the new code, so if you've changed
an
enricher the changes are picked up, or if say a launch.sh script has
changed, a restart will run the new code
The above are fairly straightforward programmatically (although good user
interaction with (E) needs some thought). So I think we can pretty
quickly
get to a much smoother dev workflow.
That's the highlight of this message. You can jump to the end, unless
you're interested in some important but low level details...
I'm also tempted by:
(F) Integration with web-based IDE and/or Brooklyn reading and writing
straight from GitHub -- but this seems like a lot of work and I'm not
convinced it's much better than (D) workflow-wise
Before we can change (2) to be the default, or start widely using the
POST
a ZIP feature, we need to sort out some issues to do with persistence and
reloading:
(G) Bundles installed via this mechanism are not persisted currently, so
if
you move to a different Brooklyn using the same backing store, you'll
lose
those bundles
(H) On rebind, bundles aren't always activated when needed, meaning items
can't be loaded
(I) We persist the individual catalog items as YAML, so we end up with
two
records — the YAML from the catalog.bom in the bundle, and the YAML
persisted for the item. This isn't a problem per se, but something to
think about, and some sometimes surprising behaviour. In particular if
you
delete the persisted YAML, the bundle is still there so the item is no
longer deleted after a full rebind.
One idea which might be useful is:
(J) Introduce a catalogGroupId field on catalog items; this will do two
things: if you try to delete an item with such a record, you'll be
encouraged to delete all such items (maybe disallowed to delete an
individual one), with the effect of deleting the bundle if it comes from
a
bundle; and when resolving types we search first for items with the same
catalogGroupId (so that e.g. if I install MyCluster:1.0 and MyNode:1.0 in
the same group, the former can refer simply to "MyNode" but if I install
a
2.0 version of that group, the 1.0 cluster still loads the 1.0 node --
this
has bitten people i the past)
There is a related Brooklyn upgrade problem worth mentioning, which the
above might help with, where:
(K) If I migrate from Brooklyn 10 to 11 when it comes out, I'll no longer
have certain entities that were at v10, since we don't include those; an
upgrade could include rules that certain groupIds need to be updated, or
it
can search and attempt to automatically apply the updates
Quite a lot here and we don't need to solve it but I wanted to:
* Share the current thinking
* Get opinions on the general dev workflow suggested by (D)
Thanks for feedback -- and if we like it help with (D) would be
appreciated!
Best
Alex
[1] . https://github.com/apache/brooklyn-server/pull/485
