GitHub user ahgittin opened a pull request:
https://github.com/apache/brooklyn-server/pull/746
[WIP] Auto wrap YAML BOMs as catalog bundles
Wraps pure-YAML BOMs uploaded to catalog in a bundle, creating a name and
version if needed.
This allows us to see all items added as a unit. This will allow us to
prefer loading co-defined items (though that's not fully done, still relies on
preferring search path when doing item lookup), and to edit or remove that item
as a set (crucially we now keep the BOM, previously we discarded it!).
Docs changed to reflect this and explain differences in another PR.
The basic wrapping was pretty simple but a surprising number of subtleties
emerged which this fixes:
* java scanning doesn't work in OSGi; docs and errors make this clear
* impacts replacement semantics (now it does a hash on the bundle to make
most things the same, though a re-post of a non-SNAPSHOT BOM that doesn't
declare a bundle will now fail because the random bundle name is stored on the
item as containingBundle and is different on a re-post; and needed extra work
to remove leaked anonymous bundles)
* had to change the way we set up the libraries / search path, to exclude
this "bundle" (mainly so tests are unchanged, but also improves efficiency for
simple BOMs; also the code is cleaner now)
And there is one thing that emerged that I *haven't* fixed: on rebind it
became vital to handle the situation where dependent bundles might not be
loaded before the depender bundle; see failing test`testLongReferenceSequence`.
Not entirely sure how to do this /cc @neykov @geomacy @aledsage.
Depends on #743.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ahgittin/brooklyn-server osgi-auto-wrap-yaml
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/brooklyn-server/pull/746.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #746
----
commit 36ffdeeca0ac5290727b3bf5d2a7267664e9638a
Author: Alex Heneveld <[email protected]>
Date: 2017-06-16T08:49:44Z
remove REST reference to removed method
commit 3860c3498d63a9a9d880480363c7947599d40a7c
Author: Alex Heneveld <[email protected]>
Date: 2017-06-16T10:53:33Z
new utilities for registered type (catalog item id) naming syntax
commit 607fc853aa395915e15d527405b7a7a8a6e97b48
Author: Alex Heneveld <[email protected]>
Date: 2017-06-16T12:24:28Z
move many things to use RegisteredTypeNaming rather than hokey
`looksLikeVersionedId`
commit eb08f8ec0d65b59bb2c4201a98632aa070ddac97
Author: Alex Heneveld <[email protected]>
Date: 2017-06-16T12:38:45Z
notes on related methods for looking up catalog items flexibly
commit c9d03c93e35eb5417ca6168207f4b0ddc3de9e7c
Author: Alex Heneveld <[email protected]>
Date: 2017-06-16T12:46:20Z
remove deprecated CAMP ServiceTypeResolver
changed a while ago to EntitySpecResolver; removes references to catalog
items and hokey lookup
commit db77f4e14d07fe9e23fb464fb23302afc1fba2f0
Author: Alex Heneveld <[email protected]>
Date: 2017-06-20T08:37:29Z
fix tests that relied on removed deprecated behaviour
commit dc4942a345dc2fbd5125a34131addb632e634fe8
Author: Alex Heneveld <[email protected]>
Date: 2017-06-20T08:51:42Z
Merge branch 'master' into version-id
commit 5b2cc48685ee606207257e74fd8a61f13f876a00
Author: Alex Heneveld <[email protected]>
Date: 2017-06-20T15:48:25Z
prefer versions which comply with recommendations, heuristics elsewhere
expand tests. some obscure ordering items are different now.
commit cc703928e1a2bd9c97795acab9a747a3733eb4a2
Author: Alex Heneveld <[email protected]>
Date: 2017-06-20T15:49:57Z
tidy version comparator, removing old code
commit 45cb10a57af9e4aba0e7d624f1af6bfb25e5cee2
Author: Alex Heneveld <[email protected]>
Date: 2017-06-21T16:04:04Z
add lots of text explaining BrooklynVersionSyntax, update comparator,
tests, and usages
commit ed98e27e934ddf54d6c1c80b4a61afaab1225e47
Author: Alex Heneveld <[email protected]>
Date: 2017-06-21T17:26:43Z
add routines to convert brooklyn/osgi versions
and bundle finder supports either, and does the conversion for you
commit b46188e60e62247f8ffbe09c11acf37aaee69fad
Author: Alex Heneveld <[email protected]>
Date: 2017-06-21T17:33:06Z
This closes #737
commit 5a7b64a751abdfbd3d75f20232a2c2a4dbb28feb
Author: Alex Heneveld <[email protected]>
Date: 2017-06-21T17:33:14Z
Merge branch 'version-new-rules' into version-new-rules-prep
commit c5c5afc385341620f199737ddae3d2efaeb3a0a4
Author: Alex Heneveld <[email protected]>
Date: 2017-06-22T13:11:22Z
This closes #737
commit d8c03c82a14f2d82c4280067322604bc0982a976
Author: Alex Heneveld <[email protected]>
Date: 2017-06-22T13:11:38Z
This closes #740
commit 89fb13955eef518f53e3c3330a20191c89a3cf2e
Author: Alex Heneveld <[email protected]>
Date: 2017-06-22T12:17:32Z
record underlying supplied version in VersionedName
support for persistence, and test
commit 701a1d7fb70203fd6a0503c369b16674972344ac
Author: Alex Heneveld <[email protected]>
Date: 2017-06-22T12:18:28Z
update OsgiBundleWithUrl to act like versioned name, and uses of
VersionedName
OSGi versions are taken at the very last moment, allowing us to preserve
brooklyn-recommended syntax throughout nearly all our code; and lookups/matches
in OSGi space compare the OSGi-target
commit 97b50e4c304f0886f96a7cd503f1d634d1fbd9c4
Author: Alex Heneveld <[email protected]>
Date: 2017-06-26T11:49:38Z
add routine to create new temp bundle
commit d7975d1414269f0f14617d3987170eba729c9c7d
Author: Alex Heneveld <[email protected]>
Date: 2017-06-23T09:53:51Z
implicitly wrap yaml in bundles when uploading to catalog
creates a new bundle when given yaml in catalog; bundle name and version
now recommended in the BOM. scan-java option tweaked but
in a way that is consistent with the past and sensible in the new world.
auto-wrapped bundles are identified with a header so we can simplify their
handling in some cases (scanning, search paths).
some things clumsy and need fixed:
* uploading a different BOM (or bundle) at the same version says "ignoring
because identical"; osgi identicality check should look
at contents
* auto-wrapped bundles are added to the library search path (no need for
this as the bundles are empty)
* failed installations keep the bundle installed, blocking subsequent
installs; probably don't install unless forced?
* warn if different version declared in BOM
some things clumsy but we could live with:
* containing bundles are added as libraries by munging the yaml; now we
have a record to that so can do a bit better than munge yaml
* if bundle has no name, a random one is chosen (probably deprecated this
behaviour); if you re-submit we don't know it's the same bundle so we don't
detect it's a bundle update; this means:
* uploading the exact same non-snapshot BOM twice will fail the second
time saying the items are different (because containing bundle is different)
* uploading an updated item in a snapshot BOM will correctly replace, but
the old bundle will still be around (just masked; though we can fix that)
commit 2165778a354fa4221eec0e8ba03a96ef5bef92a3
Author: Alex Heneveld <[email protected]>
Date: 2017-06-27T13:23:34Z
fix bundles set on scanning, and aborted work to scan osgi bundles
scanning for OSGi can be made to work in some cases if we have the jar,
but it won't work after rebind, or even in some cases if persisting (race
deleting the JAR),
plus there are likely classloading issues;
note OSGi deliberately doesn't give us the JAR.
java scanning is discouraged by OSGi; it wants us to explicitly list things
e.g. in the catalog.bom,
which is fine.
commit e7dc875af19138261f95994c5da08a2187ee64c7
Author: Alex Heneveld <[email protected]>
Date: 2017-06-27T14:38:15Z
Add checksum to managed bundle so we can tell if it changes and fail
accordingly
This permits us to allow the same BOM YAML to be added multiple times
(whether or not bundles are declared), except where it's a non-snapshot BOM
that doesn't declare a bundle, because the containing bundle will be different
there.
This also fixes a bug where containingBundle wasn't persisted; not a big
problem as we re-read bundle items, but cleaner to have it working.
commit f16bda9310469ec35f4de8f3a084746abfe08a44
Author: Alex Heneveld <[email protected]>
Date: 2017-06-27T15:28:33Z
support force when wrapping a BOM
new batch of tests around catalog versioning with OSGi
commit 8385f133273116ec5df628325a7d71ad29d4dcb8
Author: Alex Heneveld <[email protected]>
Date: 2017-06-28T08:12:19Z
repair tests where semantics changed, new tests
including explicit test for new load order problems:
* during rebind osgi bundles can have BOM loaded in any order, but they
expect their deps to be available
two remaining problems:
* wrapper library is added (changing assertions about numbers)
* added containing library changes YAML (changing assertions about comments)
commit e32d74d15359152c9b1900db2ee0a7958b15c29b
Author: Alex Heneveld <[email protected]>
Date: 2017-06-28T08:28:13Z
comments on BOM scanning
commit dc43f03fe2fa269e49a7865eafb491430f4e7e32
Author: Alex Heneveld <[email protected]>
Date: 2017-06-28T08:46:13Z
add library bundles directly, without munging yaml, and only if not wrapper
commit 480a0c76ae434a7d05455ec0de63e33b5c05aebc
Author: Alex Heneveld <[email protected]>
Date: 2017-06-28T09:43:17Z
Delete wrapper bundles who have no types installed.
There is no point to them (they aren't OSGi).
Fixes bug where uploading the same BOM multiple times, could cause bundle
leakage.
And tidy logging, more tests.
commit 21e93d27bed7efd3d8f115c8fc2234b39360ee3a
Author: Alex Heneveld <[email protected]>
Date: 2017-06-28T10:28:24Z
restore test bundle counts, and resolve library URLs on install
commit 0236dbcd5809827c0f2548561c1ddfa5cc26d7a3
Author: Alex Heneveld <[email protected]>
Date: 2017-06-28T10:45:28Z
address comments on PR in #737
commit 14aa7486ffbc07f57acf67836e5466205cb3b524
Author: Alex Heneveld <[email protected]>
Date: 2017-06-28T10:52:35Z
Merge branch 'version-id' into version-new-rules-prep
commit ee853f5806f3d2d89f2c76521ec236695f61f80d
Author: Alex Heneveld <[email protected]>
Date: 2017-06-28T11:24:36Z
tidy esp javadoc and regexes, as per PR comments
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
