Re: [VOTE] Release Apache NiFi Registry 0.4.0
+1 (binding) Followed the steps in the release helper's guide. One minor thing I noticed was that a few dates in the nifi-registry-assembly NOTICE file need to be updated (the source NOTICE looks good though). There are a ton of great improvements in this release. Nice work everyone, and thanks for RM'ing, Bryan! Kevin On Thu, May 16, 2019 at 5:36 PM Bryan Bende wrote: > > Hello, > > I am pleased to be calling this vote for the source release of Apache > NiFi Registry nifi-registry-0.4.0. > > The source zip, including signatures, digests, etc. can be found at: > https://repository.apache.org/content/repositories/orgapachenifi-1145 > > The Git tag is nifi-registry-0.4.0-RC1 > The Git commit ID is 4d7add09cd915b20dcc9959b49ecdf202d5eac2a > https://gitbox.apache.org/repos/asf?p=nifi-registry.git;a=commit;h=4d7add09cd915b20dcc9959b49ecdf202d5eac2a > > Checksums of nifi-registry-0.4.0-source-release.zip: > SHA256: a8753372fbc24f7d293df6205c465edee7a4167d835811e84e27df3e609d2be1 > SHA512: > 70ade1ba2dcf5363b6568669080a3607e5dfd3dffd28651ba5f6961db835e0f61ff7f9758757358800774e4e1063cdcfdd5d44a712a845e34052544cfce9b047 > > Release artifacts are signed with the following key: > https://people.apache.org/keys/committer/bbende.asc > > KEYS file available here: > https://dist.apache.org/repos/dist/release/nifi/KEYS > > 37 issues were closed/resolved for this release: > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12320920=12344183 > > Release note highlights can be found here: > https://cwiki.apache.org/confluence/display/NIFIREG/Release+Notes#ReleaseNotes-NiFiRegistry0.4.0 > > The vote will be open for 96 hours. > Please download the release candidate and evaluate the necessary items > including checking hashes, signatures, build from source, and test. > Then please vote: > > [ ] +1 Release this package as nifi-registry-0.4.0 > [ ] +0 no opinion > [ ] -1 Do not release this package because...
Apache NiFi Registry 0.4.0 RC1 Release Helper Guide
Hello Apache NiFi community, Please find the associated guidance to help those interested in validating/verifying the release so they can vote. # Download latest KEYS file: https://dist.apache.org/repos/dist/dev/nifi/KEYS # Import keys file: gpg --import KEYS # [optional] Clear out local maven artifact repository # Pull down nifi-registry-0.4.0 source release artifacts for review: wget https://dist.apache.org/repos/dist/dev/nifi/nifi-registry/nifi-registry-0.4.0/nifi-registry-0.4.0-source-release.zip wget https://dist.apache.org/repos/dist/dev/nifi/nifi-registry/nifi-registry-0.4.0/nifi-registry-0.4.0-source-release.zip.asc wget https://dist.apache.org/repos/dist/dev/nifi/nifi-registry/nifi-registry-0.4.0/nifi-registry-0.4.0-source-release.zip.sha256 wget https://dist.apache.org/repos/dist/dev/nifi/nifi-registry/nifi-registry-0.4.0/nifi-registry-0.4.0-source-release.zip.sha512 # Verify the signature gpg --verify -v nifi-registry-0.4.0-source-release.zip.asc # Verify the hashes (sha256, sha512) match the source and what was provided in the vote email thread shasum -a 256 nifi-registry-0.4.0-source-release.zip shasum -a 512 nifi-registry-0.4.0-source-release.zip # Unzip nifi-registry-0.4.0-source-release.zip # Verify the build works including release audit tool (RAT) checks cd nifi-registry-0.4.0 mvn clean install -Pcontrib-check # Optionally run integration tests mvn clean install -Pcontrib-check,integration-tests # Verify the contents contain a good README, NOTICE, and LICENSE. # Verify the git commit ID is correct # Verify the RC was branched off the correct git commit ID # Look at the resulting convenience binary as found in nifi-registry-assembly/target # Make sure the README, NOTICE, and LICENSE are present and correct # Run the resulting convenience binary and make sure it works as expected # Send a response to the vote thread indicating a +1, 0, -1 based on your findings. Thank you for your time and effort to validate the release!
[VOTE] Release Apache NiFi Registry 0.4.0
Hello, I am pleased to be calling this vote for the source release of Apache NiFi Registry nifi-registry-0.4.0. The source zip, including signatures, digests, etc. can be found at: https://repository.apache.org/content/repositories/orgapachenifi-1145 The Git tag is nifi-registry-0.4.0-RC1 The Git commit ID is 4d7add09cd915b20dcc9959b49ecdf202d5eac2a https://gitbox.apache.org/repos/asf?p=nifi-registry.git;a=commit;h=4d7add09cd915b20dcc9959b49ecdf202d5eac2a Checksums of nifi-registry-0.4.0-source-release.zip: SHA256: a8753372fbc24f7d293df6205c465edee7a4167d835811e84e27df3e609d2be1 SHA512: 70ade1ba2dcf5363b6568669080a3607e5dfd3dffd28651ba5f6961db835e0f61ff7f9758757358800774e4e1063cdcfdd5d44a712a845e34052544cfce9b047 Release artifacts are signed with the following key: https://people.apache.org/keys/committer/bbende.asc KEYS file available here: https://dist.apache.org/repos/dist/release/nifi/KEYS 37 issues were closed/resolved for this release: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12320920=12344183 Release note highlights can be found here: https://cwiki.apache.org/confluence/display/NIFIREG/Release+Notes#ReleaseNotes-NiFiRegistry0.4.0 The vote will be open for 96 hours. Please download the release candidate and evaluate the necessary items including checking hashes, signatures, build from source, and test. Then please vote: [ ] +1 Release this package as nifi-registry-0.4.0 [ ] +0 no opinion [ ] -1 Do not release this package because...
Re: [DISCUSS] Closing in on a NiFi Registry 0.4.0 release?
Excellent Bryan. Looking forward to this release. Thanks for taking the initiative here. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On May 16, 2019, at 12:34 PM, Bryan Bende wrote: > > The last few items tagged for 0.4.0 have been merged so we should be > good to put out an RC for 0.4.0. > > I'll start working on getting everything together and try to get > something out soon. > > On Fri, May 10, 2019 at 9:00 AM Bryan Bende wrote: >> >> I think we should be able to kick 0.4.0 very soon, just a couple of >> items to get in. Assuming those get reviewed and merged soon we could >> possibly kick out an RC early next week. I can plan to be RM unless >> anyone else is interested. >> >> On Thu, Mar 28, 2019 at 3:39 PM Bryan Bende wrote: >>> >>> I think Kevin summarized it nicely. Auto-pulling the extensions with a >>> versioned flow is definitely the ultimate vision, but we need to take >>> steps to get there. >>> >>> The process of automatically bringing in the extensions is something >>> that would be implemented on the NiFi side and will require a bit of >>> thought and design around the user experience. >>> >>> Before we can do any of that, we first need somewhere to store and >>> retrieve versioned extensions, which is the work that is underway on >>> the NiFi Registry side. >>> >>> On Thu, Mar 28, 2019 at 3:29 PM Kevin Doran wrote: Hi Ryan, Bryan can speak more to this than I can, but, yes, providing the experience you describe is the eventual goal! Aside from the NiFi Extension Registry support though (the initial bits of which is what Bryan is referring to in 0.4.0), getting the full, streamlined experience in NiFi will require changes to NiFi as well, for example in the logic to import a flow as it now also has to reach back to Registry to pull extensions referenced by the flow during import. So the changes to NiFi Registry to support hosting NiFi Extensions is only half of the puzzle. This is why Bryan also mentioned it might be worth holding back 0.4.0 (or releasing it, but marking the new extension stuff as experimental), as it is likely the current implementation and interfaces will need to change once the NiFi work begins. Cheers, Kevin On Thu, Mar 28, 2019 at 2:56 PM Ryan Hendrickson wrote: > > We'd like to connect a NiFi to a Registry, Pull down a Process Group, and > get any NARs required for that without having to manually install them on > the NiFi. > > That's what I'm understanding the Extension part of the 0.4.0 release to > be, am I on point, or way off? > > Thanks, > Ryan > > On Wed, Mar 27, 2019 at 4:17 PM Bryan Rosander > wrote: > >> Hey Pierre, >> >> You can version the metadata db in the root dir of the git repo with >> event >> hooks. >> >> With an h2 jdbc url of >> >> 'jdbc:h2:$NIFI_REGISTRY_HOME/database/nifi-registry-primary;AUTOCOMMIT=OFF;LOCK_MODE=3;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=TRUE' >> >> You can connect to the db in a separate process, we're using that with a >> shell event hook to dump the db to sql script when mutations happen (now >> tenant events too :) ), storing that in the git repo using >> org.h2.tools.Script, and committing it. Then we can restore from that >> sql >> script on startup before calling the stock startup script. >> >> Side benefit is you can see metadata state changes in the git history and >> manually inspect the sql. >> >> >> On Wed, Mar 27, 2019 at 1:57 PM Pierre Villard < >> pierre.villard...@gmail.com> >> wrote: >> >>> Bryan, >>> >>> Thanks for the details. On my side, the main feature I (and others I >>> suppose) am looking for is the ability to rebuild the metadata DB from >> the >>> Git repo. But to be honest, this is not a blocker at all: I can live >> with a >>> custom Docker image containing this feature. >>> >>> So, from my point of view, I don't have any issue with #2 but maybe #1 >>> would also give access to experimental features and have interesting >>> feedback from the community. But we would, indeed, need to make it >> crystal >>> clear that the APIs could change in ulterior release(s). >>> >>> Pierre >>> >>> >>> >>> Le mer. 27 mars 2019 à 18:11, Bryan Bende a écrit : >>> Ryan, The short answer to your question is no, there isn't actually anything new in the UI. The registry UI is setup in a generic way to list "versioned items", so versioned extension bundles will just automatically show up in the list with versioned flows. Let me put together a wiki page that outlines
Re: [DISCUSS] Closing in on a NiFi Registry 0.4.0 release?
The last few items tagged for 0.4.0 have been merged so we should be good to put out an RC for 0.4.0. I'll start working on getting everything together and try to get something out soon. On Fri, May 10, 2019 at 9:00 AM Bryan Bende wrote: > > I think we should be able to kick 0.4.0 very soon, just a couple of > items to get in. Assuming those get reviewed and merged soon we could > possibly kick out an RC early next week. I can plan to be RM unless > anyone else is interested. > > On Thu, Mar 28, 2019 at 3:39 PM Bryan Bende wrote: > > > > I think Kevin summarized it nicely. Auto-pulling the extensions with a > > versioned flow is definitely the ultimate vision, but we need to take > > steps to get there. > > > > The process of automatically bringing in the extensions is something > > that would be implemented on the NiFi side and will require a bit of > > thought and design around the user experience. > > > > Before we can do any of that, we first need somewhere to store and > > retrieve versioned extensions, which is the work that is underway on > > the NiFi Registry side. > > > > On Thu, Mar 28, 2019 at 3:29 PM Kevin Doran wrote: > > > > > > Hi Ryan, > > > > > > Bryan can speak more to this than I can, but, yes, providing the > > > experience you describe is the eventual goal! Aside from the NiFi > > > Extension Registry support though (the initial bits of which is what > > > Bryan is referring to in 0.4.0), getting the full, streamlined > > > experience in NiFi will require changes to NiFi as well, for example > > > in the logic to import a flow as it now also has to reach back to > > > Registry to pull extensions referenced by the flow during import. So > > > the changes to NiFi Registry to support hosting NiFi Extensions is > > > only half of the puzzle. This is why Bryan also mentioned it might be > > > worth holding back 0.4.0 (or releasing it, but marking the new > > > extension stuff as experimental), as it is likely the current > > > implementation and interfaces will need to change once the NiFi work > > > begins. > > > > > > Cheers, > > > Kevin > > > > > > On Thu, Mar 28, 2019 at 2:56 PM Ryan Hendrickson > > > wrote: > > > > > > > > We'd like to connect a NiFi to a Registry, Pull down a Process Group, > > > > and > > > > get any NARs required for that without having to manually install them > > > > on > > > > the NiFi. > > > > > > > > That's what I'm understanding the Extension part of the 0.4.0 release to > > > > be, am I on point, or way off? > > > > > > > > Thanks, > > > > Ryan > > > > > > > > On Wed, Mar 27, 2019 at 4:17 PM Bryan Rosander > > > > wrote: > > > > > > > > > Hey Pierre, > > > > > > > > > > You can version the metadata db in the root dir of the git repo with > > > > > event > > > > > hooks. > > > > > > > > > > With an h2 jdbc url of > > > > > > > > > > 'jdbc:h2:$NIFI_REGISTRY_HOME/database/nifi-registry-primary;AUTOCOMMIT=OFF;LOCK_MODE=3;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=TRUE' > > > > > > > > > > You can connect to the db in a separate process, we're using that > > > > > with a > > > > > shell event hook to dump the db to sql script when mutations happen > > > > > (now > > > > > tenant events too :) ), storing that in the git repo using > > > > > org.h2.tools.Script, and committing it. Then we can restore from > > > > > that sql > > > > > script on startup before calling the stock startup script. > > > > > > > > > > Side benefit is you can see metadata state changes in the git history > > > > > and > > > > > manually inspect the sql. > > > > > > > > > > > > > > > On Wed, Mar 27, 2019 at 1:57 PM Pierre Villard < > > > > > pierre.villard...@gmail.com> > > > > > wrote: > > > > > > > > > > > Bryan, > > > > > > > > > > > > Thanks for the details. On my side, the main feature I (and others I > > > > > > suppose) am looking for is the ability to rebuild the metadata DB > > > > > > from > > > > > the > > > > > > Git repo. But to be honest, this is not a blocker at all: I can live > > > > > with a > > > > > > custom Docker image containing this feature. > > > > > > > > > > > > So, from my point of view, I don't have any issue with #2 but maybe > > > > > > #1 > > > > > > would also give access to experimental features and have interesting > > > > > > feedback from the community. But we would, indeed, need to make it > > > > > crystal > > > > > > clear that the APIs could change in ulterior release(s). > > > > > > > > > > > > Pierre > > > > > > > > > > > > > > > > > > > > > > > > Le mer. 27 mars 2019 à 18:11, Bryan Bende a > > > > > > écrit : > > > > > > > > > > > > > Ryan, > > > > > > > > > > > > > > The short answer to your question is no, there isn't actually > > > > > > > anything > > > > > > > new in the UI. The registry UI is setup in a generic way to list > > > > > > > "versioned items", so versioned extension bundles will just > > > > > > > automatically show up in the list with versioned flows. > > > > > > > > > > > > > > Let me put together a wiki
Re: NIFI conectivity with HIVE to load data into HANA tables
It can be done. I was briefly working on something similar. In my case, it was HANA which was the source and Hive was the destination. You can query Hive using SelectHiveQL and then use PutSQL/PutDatabaseRecord to write to HANA. On Thu, May 16, 2019 at 5:13 PM Patan Umejaiba wrote: > Hi Team, > > My requirement is to load the HIVE files data into HANA tables using NIFI. > Let me know if that is possible and if that is a yes please provide the > process to be followed. > > Thanks, > Jaiba >
NIFI conectivity with HIVE to load data into HANA tables
Hi Team, My requirement is to load the HIVE files data into HANA tables using NIFI. Let me know if that is possible and if that is a yes please provide the process to be followed. Thanks, Jaiba
