Closes #746 Auto wrap YAML BOMs as catalog bundles
Wraps pure-YAML BOMs uploaded to catalog in a bundle, creating a name and version if needed. This allows us to see all items added as a unit. This will allow us to prefer loading co-defined items (though that's not fully done, still relies on preferring search path when doing item lookup), and to edit or remove that item as a set (crucially we now keep the BOM, previously we discarded it!). Docs changed to reflect this and explain differences in another PR. The basic wrapping was pretty simple but a surprising number of subtleties emerged which this fixes: * java scanning doesn't work in OSGi; docs and errors make this clear * impacts replacement semantics (now it does a hash on the bundle to make most things the same, though a re-post of a non-SNAPSHOT BOM that doesn't declare a bundle will now fail because the random bundle name is stored on the item as containingBundle and is different on a re-post; and needed extra work to remove leaked anonymous bundles) * had to change the way we set up the libraries / search path, to exclude this "bundle" (mainly so tests are unchanged, but also improves efficiency for simple BOMs; also the code is cleaner now) And there is one thing that emerged that I *haven't* fixed: on rebind it became vital to handle the situation where dependent bundles might not be loaded before the depender bundle; see failing test`testLongReferenceSequence`. Not entirely sure how to do this /cc @neykov @geomacy @aledsage. Depends on #743. Project: http://git-wip-us.apache.org/repos/asf/brooklyn-server/repo Commit: http://git-wip-us.apache.org/repos/asf/brooklyn-server/commit/c02b3e35 Tree: http://git-wip-us.apache.org/repos/asf/brooklyn-server/tree/c02b3e35 Diff: http://git-wip-us.apache.org/repos/asf/brooklyn-server/diff/c02b3e35 Branch: refs/heads/master Commit: c02b3e355987fa109d77d1e61cd08299d28e5be9 Parents: fe31292 97f2524 Author: Geoff Macartney <[email protected]> Authored: Wed Jul 19 17:10:32 2017 +0100 Committer: Geoff Macartney <[email protected]> Committed: Wed Jul 19 17:10:32 2017 +0100 ---------------------------------------------------------------------- .../brooklyn/api/catalog/BrooklynCatalog.java | 34 + .../brooklyn/api/location/LocationRegistry.java | 20 +- .../mementos/BrooklynMementoPersister.java | 1 + .../rebind/mementos/CatalogItemMemento.java | 2 + .../brooklyn/api/objs/BrooklynObjectType.java | 34 +- .../api/typereg/BrooklynTypeRegistry.java | 9 +- .../brooklyn/api/typereg/ManagedBundle.java | 6 +- .../brooklyn/api/typereg/OsgiBundleWithUrl.java | 5 + .../brooklyn/api/typereg/RegisteredType.java | 9 + .../BrooklynComponentTemplateResolver.java | 6 +- .../brooklyn/spi/creation/CampResolver.java | 4 + .../spi/creation/CampTypePlanTransformer.java | 7 +- .../camp/brooklyn/AbstractYamlRebindTest.java | 2 + .../camp/brooklyn/AbstractYamlTest.java | 49 +- .../camp/brooklyn/ApplicationsYamlOsgiTest.java | 31 + .../camp/brooklyn/ApplicationsYamlTest.java | 29 +- .../camp/brooklyn/ConfigParametersYamlTest.java | 13 +- .../camp/brooklyn/ReferencedOsgiYamlTest.java | 9 +- .../camp/brooklyn/ReferencedYamlOsgiTest.java | 54 ++ .../camp/brooklyn/ReferencedYamlTest.java | 10 +- .../catalog/CatalogMakeOsgiBundleTest.java | 20 - .../catalog/CatalogOsgiLibraryTest.java | 52 +- .../CatalogOsgiVersionMoreEntityRebindTest.java | 53 +- .../CatalogOsgiVersionMoreEntityTest.java | 15 +- .../catalog/CatalogOsgiYamlEntityTest.java | 25 +- .../catalog/CatalogOsgiYamlLocationTest.java | 10 +- .../catalog/CatalogOsgiYamlVersioningTest.java | 107 +++ .../brooklyn/catalog/CatalogScanOsgiTest.java | 154 ++++ .../camp/brooklyn/catalog/CatalogScanTest.java | 11 +- .../catalog/CatalogYamlAppOsgiTest.java | 31 + .../brooklyn/catalog/CatalogYamlAppTest.java | 15 +- .../CatalogYamlEntityOsgiTypeRegistryTest.java | 66 ++ .../brooklyn/catalog/CatalogYamlEntityTest.java | 32 +- .../catalog/CatalogYamlLocationTest.java | 21 +- .../brooklyn/catalog/CatalogYamlRebindTest.java | 173 ++-- .../catalog/CatalogYamlVersioningTest.java | 59 +- .../catalog/SpecParameterParsingOsgiTest.java | 13 +- .../catalog/SpecParameterUnwrappingTest.java | 42 +- .../brooklyn/test/lite/CampYamlLiteTest.java | 41 +- .../core/BrooklynFeatureEnablement.java | 11 - .../apache/brooklyn/core/BrooklynVersion.java | 3 +- .../catalog/internal/BasicBrooklynCatalog.java | 866 +++++++++++++++---- .../catalog/internal/CatalogBomScanner.java | 130 --- .../core/catalog/internal/CatalogBundleDto.java | 22 +- .../catalog/internal/CatalogBundleLoader.java | 120 +-- .../catalog/internal/CatalogBundleTracker.java | 78 -- .../catalog/internal/CatalogClasspathDo.java | 4 +- .../core/catalog/internal/CatalogDo.java | 2 +- .../catalog/internal/CatalogInitialization.java | 16 +- .../internal/CatalogItemDtoAbstract.java | 9 +- .../core/catalog/internal/CatalogUtils.java | 73 +- .../internal/JavaCatalogToSpecTransformer.java | 9 +- .../core/location/BasicLocationRegistry.java | 172 ++-- .../apache/brooklyn/core/mgmt/BrooklynTags.java | 3 + .../core/mgmt/ha/OsgiArchiveInstaller.java | 99 ++- .../mgmt/ha/OsgiBundleInstallationResult.java | 22 +- .../brooklyn/core/mgmt/ha/OsgiManager.java | 82 +- .../persist/FileBasedStoreObjectAccessor.java | 8 +- .../rebind/BasicCatalogItemRebindSupport.java | 1 + .../core/mgmt/rebind/RebindContextImpl.java | 7 +- .../mgmt/rebind/RebindContextLookupContext.java | 7 +- .../core/mgmt/rebind/RebindIteration.java | 72 +- .../rebind/dto/BasicCatalogItemMemento.java | 14 + .../mgmt/rebind/dto/MementosGenerators.java | 2 + .../core/objs/proxy/InternalFactory.java | 2 +- .../typereg/AbstractTypePlanTransformer.java | 4 +- .../core/typereg/BasicBrooklynTypeRegistry.java | 163 +++- .../core/typereg/BasicManagedBundle.java | 23 +- .../core/typereg/BasicOsgiBundleWithUrl.java | 7 + .../core/typereg/BasicRegisteredType.java | 53 ++ .../typereg/BasicTypeImplementationPlan.java | 23 + .../ReferencedUnresolvedTypeException.java | 44 + .../core/typereg/RegisteredTypeKindVisitor.java | 3 + .../typereg/RegisteredTypeLoadingContexts.java | 3 +- .../core/typereg/RegisteredTypePredicates.java | 78 +- .../brooklyn/core/typereg/RegisteredTypes.java | 147 +++- .../core/typereg/TypePlanTransformers.java | 1 + .../typereg/UnsupportedTypePlanException.java | 2 + .../brooklyn/util/core/ClassLoaderUtils.java | 40 +- .../brooklyn/util/core/LoaderDispatcher.java | 2 +- .../brooklyn/util/core/ResourceUtils.java | 2 +- .../util/core/javalang/ReflectionScanner.java | 3 + .../brooklyn/util/core/osgi/BundleMaker.java | 41 +- .../apache/brooklyn/util/core/osgi/Osgis.java | 1 + .../apache/brooklyn/util/core/task/Tasks.java | 8 +- .../main/resources/brooklyn/empty.catalog.bom | 4 +- .../deserializingClassRenames.properties | 2 + .../core/location/LocationManagementTest.java | 10 +- .../core/location/LocationRegistryTest.java | 13 +- .../core/mgmt/rebind/RebindCatalogItemTest.java | 47 +- .../core/mgmt/rebind/RebindTestFixture.java | 40 +- .../core/mgmt/rebind/RebindTestUtils.java | 14 +- .../typereg/BasicBrooklynTypeRegistryTest.java | 11 +- .../util/core/osgi/BundleMakerTest.java | 78 +- .../resources/OSGI-INF/blueprint/blueprint.xml | 11 - .../command/support/CloudExplorerSupport.java | 2 +- .../location/jclouds/JcloudsLocation.java | 4 +- .../apache/brooklyn/rest/api/CatalogApi.java | 3 +- .../rest/resources/CatalogResource.java | 121 +-- .../rest/resources/LocationResource.java | 2 +- .../rest/transform/CatalogTransformer.java | 200 ++++- .../rest/transform/LocationTransformer.java | 9 +- .../rest/util/BrooklynRestResourceUtils.java | 5 + .../rest/resources/ApplicationResourceTest.java | 28 +- .../rest/resources/CatalogResourceTest.java | 57 +- .../rest/resources/LocationResourceTest.java | 10 +- .../main/java/org/apache/brooklyn/cli/Main.java | 2 +- .../main/resources/brooklyn/default.catalog.bom | 6 +- .../src/main/resources/catalog.bom | 1 - .../entities/src/main/resources/catalog.bom | 1 - .../src/main/resources/catalog.bom | 1 - .../src/main/resources/catalog.bom | 1 - .../src/main/resources/catalog.bom | 1 - .../brooklyn/util/exceptions/Exceptions.java | 36 +- .../brooklyn/util/osgi/VersionedName.java | 14 +- .../apache/brooklyn/util/stream/Streams.java | 21 + .../util/text/BrooklynVersionSyntax.java | 7 + .../brooklyn/util/text/VersionComparator.java | 6 +- .../util/exceptions/ExceptionsTest.java | 8 +- .../brooklyn/util/stream/StreamsTest.java | 36 + .../brooklyn-test-osgi-com-example-entities.jar | Bin 22139 -> 22099 bytes .../osgi/brooklyn-test-osgi-entities.jar | Bin 22902 -> 22864 bytes .../brooklyn-test-osgi-more-entities_0.1.0.jar | Bin 16003 -> 15967 bytes .../brooklyn-test-osgi-more-entities_0.2.0.jar | Bin 16922 -> 16884 bytes ...-test-osgi-more-entities_evil-twin_0.2.0.jar | Bin 14098 -> 14061 bytes .../TestResourceUnavailableException.java | 6 +- 126 files changed, 3336 insertions(+), 1238 deletions(-) ----------------------------------------------------------------------
