[
https://issues.apache.org/jira/browse/NIFIREG-211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693418#comment-16693418
]
ASF GitHub Bot commented on NIFIREG-211:
----------------------------------------
GitHub user bbende opened a pull request:
https://github.com/apache/nifi-registry/pull/148
NIFIREG-211 Initial work for adding extenion bundles to NiFi Registry
This PR includes the foundational work for adding extension bundles as a
new type of versioned item. With this PR it is possible to upload NARs via the
REST API with them being stored in the local filesystem persistence provider,
and then interact with them through various REST APIs.
In the current state of this PR, registry does not have a way to know
anything about the actual extensions contained in a bundle. This will need to
come from a component descriptor provided within the NAR which will then be
parsed and extracted on registry side.
We can leave NIFIREG-211 open after merging this so that we can continue to
work against it and build out the additional functionality.
A NAR can be uploaded by making a multi-part form POST to the end-point:
_buckets/{bucketId}/extensions/bundles/{bundleType}_
An example using curl would be:
`curl -v -F file=@./nifi-example-processors-nar-1.0.0.nar
http://localhost:18080/nifi-registry-api/buckets/65d6d70d-ecb7-478f-bbc4-1e378e52827c/extensions/bundles/nifi-nar`
From there you can interact with the REST API from the swagger docs.
I've also started a branch of NiFi with CLI commands related to extension
bundles:
https://github.com/bbende/nifi/tree/extensions
If you run a full build of this PR first with mvn clean install, and then a
build of the NiFi branch, you can launch the CLI in
nifi-toolkit/nifi-toolkit-assembly/target/nifi-toolkit-1.9.0-SNAPSHOT-bin/nifi-toolkit-1.9.0-SNAPSHOT/bin/cli.sh
and run the "upload-nars" command:
`registry upload-nars -b 024dba45-95c2-4a54-b79d-0abe1bb1122a -ebd
/path/to/nifi-releases/nifi-1.8.0/lib -u http://localhost:18080 -verbose`
This should upload all of the NARs from the 1.8.0 release to your local
registry.
Here is the commit history for all the changes made in this PR:
- Setting up DB tables and entities for extensions
- Updated MetadataService and DatabaseMetadataService with new methods for
extension entities
- Added ON DELETE CASCADE to existing tables and simplified delete logic
for buckets and flows
- Created data model for extension bundles and mapping to/from DB entities
- Created ExtensionBundleExtractor with an implemenetation for NARs
- Setup LinkService and LinkBuilder for extension bundles
- Setup pluggable persistence provider for extension bundles and
implemented a local file-system provider.
- Refactored LinkService and add links for all extension related items
- Changed extension service to write out bundles to a temp directory before
extracting and passing to persistence provider
- Implemented multi-part form upload for extensions bundles
- Upgraded to spring-boot 2.1.0
- Added SHA-256 checksums for bundle versions
- Initial client methods for uploading and retrieving bundles
- Configuring NiFi Registry Jersey client to use chunked entity processing
so we don't load the entire bundle content into memory during an upload
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bbende/nifi-registry extensions
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi-registry/pull/148.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #148
----
commit 7744072b5f059fe5240746e968493eddefa719a6
Author: Bryan Bende <bbende@...>
Date: 2018-11-20T15:57:08Z
NIFIREG-211 Initial work for adding extenion bundles to NiFi Registry
- Setting up DB tables and entities for extensions
- Updated MetadataService and DatabaseMetadataService with new methods for
extension entities
- Added ON DELETE CASCADE to existing tables and simplified delete logic
for buckets and flows
- Created data model for extension bundles and mapping to/from DB entities
- Created ExtensionBundleExtractor with an implemenetation for NARs
- Setup LinkService and LinkBuilder for extension bundles
- Setup pluggable persistence provider for extension bundles and
implemented a local file-system provider.
- Refactored LinkService and add links for all extension related items
- Changed extension service to write out bundles to a temp directory before
extracting and passing to persistence provider
- Implemented multi-part form upload for extensions bundles
- Upgraded to spring-boot 2.1.0
- Added SHA-256 checksums for bundle versions
- Initial client methods for uploading and retrieving bundles
- Configuring NiFi Registry Jersey client to use chunked entity processing
so we don't load the entire bundle content into memory during an upload
----
> Add extension bundles as a type of versioned item
> -------------------------------------------------
>
> Key: NIFIREG-211
> URL: https://issues.apache.org/jira/browse/NIFIREG-211
> Project: NiFi Registry
> Issue Type: Improvement
> Reporter: Bryan Bende
> Assignee: Bryan Bende
> Priority: Major
> Fix For: 0.4.0
>
>
> This ticket is to capture the work for adding extension bundles to NiFi
> Registry.
> This work may require several follow on tickets, but at a high-level will
> include some of the following:
> - Add a new type of item called an extension bundle, where each bundle
> can contain one ore extensions or APIs
>
> - Support bundles for traditional NiFi (aka NARs) and also bundles for
> MiNiFi CPP
>
> - Ability to upload the binary artifact for a bundle and extract the
> metadata about the bundle, and metadata about the extensions contained
> in the bundle (more on this later)
>
> - Provide a pluggable storage provider for saving the content of each
> extension bundle so that we can have different implementations like
> local fileysystem, S3, and other object stores
>
> - Provide a REST API for listing and retrieving available bundles,
> integrate this into the registry Java client and NiFi CLI
> - Security considerations such as checksums and cryptographic signatures for
> bundles
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
