Re: [VOTE] Release Apache NiFi 1.8.0

2018-10-18 Thread James Wing 
+1 Release this package as nifi-1.8.0.  Ran through the release helper,
tested the resulting binary, diffed some of the L and code files.

Thanks for putting this release together, Jeff.

On Wed, Oct 17, 2018 at 8:59 PM Jeff  wrote:

> Hello,
>
> I am pleased to be calling this vote for the source release of Apache NiFi
> nifi-1.8.0.
>
> The source zip, including signatures, digests, etc. can be found at:
> https://repository.apache.org/content/repositories/orgapachenifi-1133
>
> The Git tag is nifi-1.8.0-RC1
> The Git commit ID is 9b02d58626ca874ed2ed3e0bbe530512cfa0dbf8
>
> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=9b02d58626ca874ed2ed3e0bbe530512cfa0dbf8
>
> Checksums of nifi-1.8.0-source-release.zip:
> SHA256: 3ec90a7f153e507d7bba2400d6dafac02641d6f7afc7a954fed959191073ce21
> SHA512:
>
> 8b9d944da1833bfb645f502107cab98a555e3b2a7602c5ff438407272c86defdeebe18625c5ad9dfb3f344397314569e97220a35f2438182a79a700caa90721e
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/jstorck.asc
>
> KEYS file available here:
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
> 204 issues were closed/resolved for this release:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12343482
>
> Release note highlights can be found here:
>
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.8.0
>
> The vote will be open for 72 hours.
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build
> from source, and test. Then please vote:
>
> [ ] +1 Release this package as nifi-1.8.0
> [ ] +0 no opinion
> [ ] -1 Do not release this package because...
>

Re: [VOTE] Release Apache NiFi 1.8.0

2018-10-18 Thread Koji Kawamura 
+1 binding

Validated signatures and hashes
Confirmed existing flows work, load-balance and node-offload with a
secured cluster from UI and CLI.

Thank you for the RM duties, Jeff!

Koji
On Fri, Oct 19, 2018 at 6:32 AM Jeremy Dyer  wrote:
>
> +1, binding
>
> Validated signatures, hashes, and commit
> Validated existing workflows against a NiFi Registry instance
>
> Lots of good stuff!
> Jeff thanks for handling the release!
>
> - Jeremy Dyer
>
> On Thu, Oct 18, 2018 at 5:11 PM Pierre Villard 
> wrote:
>
> > +1, binding
> >
> > Checked signature, hashes, L and commit.
> > Ran multiple workflows in both secured and unsecured clusters.
> > Played with the new load-balancing and offloading features (AWESOME
> > WORK!!!)
> > Confirmed some minor fixes and improvements.
> >
> > A great release!
> > Thanks Jeff for taking care of the RM duties and thanks to all the people
> > contributing in this release!
> >
> > Pierre
> >
> > Le jeu. 18 oct. 2018 à 20:33, Otto Fowler  a
> > écrit :
> >
> > > +1
> > >
> > > Confirmed signatures
> > > Confirmed hashes
> > > Confirmed source matches commit src
> > > ran build withe check
> > > ran nifi with a template from a bug to verify fix
> > >
> > >
> > >
> > >
> > > On October 18, 2018 at 09:45:12, Joe Witt (joe.w...@gmail.com) wrote:
> > >
> > > +1 (binding)
> > >
> > > Confirmed sigs, hashes, source L, and specified commit present
> > > (including Drew's doc update which is the last code change)
> > > Confirmed full clean build with contrib check and grpc on clean
> > repo/etc..
> > > Tested local nifi instance and sample flows. Did not test full secure
> > > setup with clustering.
> > >
> > > This release has quietly turned out to be another beast and the new
> > > load balancing capabilities are awesome!
> > >
> > > Thanks
> > > On Thu, Oct 18, 2018 at 7:57 AM Mike Thomsen 
> > > wrote:
> > > >
> > > > (NOTICE, LICENSE and README all looked good when I looked at them; no
> > > > problems were apparent)
> > > >
> > > > On Thu, Oct 18, 2018 at 7:56 AM Mike Thomsen 
> > > wrote:
> > > >
> > > > > +1 binding.
> > > > >
> > > > > - Checksums and branch id/diff matched up.
> > > > > - Ran binary against a Mongo regression test that I made for the new
> > > > > client service functionality and everything worked cleanly against
> > > > > dockerized Mongo.
> > > > >
> > > > > On Wed, Oct 17, 2018 at 11:59 PM Jeff  wrote:
> > > > >
> > > > >> Hello,
> > > > >>
> > > > >> I am pleased to be calling this vote for the source release of
> > Apache
> > > NiFi
> > > > >> nifi-1.8.0.
> > > > >>
> > > > >> The source zip, including signatures, digests, etc. can be found at:
> > > > >>
> > https://repository.apache.org/content/repositories/orgapachenifi-1133
> > > > >>
> > > > >> The Git tag is nifi-1.8.0-RC1
> > > > >> The Git commit ID is 9b02d58626ca874ed2ed3e0bbe530512cfa0dbf8
> > > > >>
> > > > >>
> > >
> > >
> > https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=9b02d58626ca874ed2ed3e0bbe530512cfa0dbf8
> > > > >>
> > > > >> Checksums of nifi-1.8.0-source-release.zip:
> > > > >> SHA256:
> > > 3ec90a7f153e507d7bba2400d6dafac02641d6f7afc7a954fed959191073ce21
> > > > >> SHA512:
> > > > >>
> > > > >>
> > >
> > >
> > 8b9d944da1833bfb645f502107cab98a555e3b2a7602c5ff438407272c86defdeebe18625c5ad9dfb3f344397314569e97220a35f2438182a79a700caa90721e
> > >
> > > > >>
> > > > >> Release artifacts are signed with the following key:
> > > > >> https://people.apache.org/keys/committer/jstorck.asc
> > > > >>
> > > > >> KEYS file available here:
> > > > >> https://dist.apache.org/repos/dist/release/nifi/KEYS
> > > > >>
> > > > >> 204 issues were closed/resolved for this release:
> > > > >>
> > > > >>
> > >
> > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12343482
> > > > >>
> > > > >> Release note highlights can be found here:
> > > > >>
> > > > >>
> > >
> > >
> > https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.8.0
> > > > >>
> > > > >> The vote will be open for 72 hours.
> > > > >> Please download the release candidate and evaluate the necessary
> > items
> > > > >> including checking hashes, signatures, build
> > > > >> from source, and test. Then please vote:
> > > > >>
> > > > >> [ ] +1 Release this package as nifi-1.8.0
> > > > >> [ ] +0 no opinion
> > > > >> [ ] -1 Do not release this package because...
> > > > >>
> > > > >
> > >
> >

Re: [VOTE] Release Apache NiFi 1.8.0

2018-10-18 Thread Jeremy Dyer 
+1, binding

Validated signatures, hashes, and commit
Validated existing workflows against a NiFi Registry instance

Lots of good stuff!
Jeff thanks for handling the release!

- Jeremy Dyer

On Thu, Oct 18, 2018 at 5:11 PM Pierre Villard 
wrote:

> +1, binding
>
> Checked signature, hashes, L and commit.
> Ran multiple workflows in both secured and unsecured clusters.
> Played with the new load-balancing and offloading features (AWESOME
> WORK!!!)
> Confirmed some minor fixes and improvements.
>
> A great release!
> Thanks Jeff for taking care of the RM duties and thanks to all the people
> contributing in this release!
>
> Pierre
>
> Le jeu. 18 oct. 2018 à 20:33, Otto Fowler  a
> écrit :
>
> > +1
> >
> > Confirmed signatures
> > Confirmed hashes
> > Confirmed source matches commit src
> > ran build withe check
> > ran nifi with a template from a bug to verify fix
> >
> >
> >
> >
> > On October 18, 2018 at 09:45:12, Joe Witt (joe.w...@gmail.com) wrote:
> >
> > +1 (binding)
> >
> > Confirmed sigs, hashes, source L, and specified commit present
> > (including Drew's doc update which is the last code change)
> > Confirmed full clean build with contrib check and grpc on clean
> repo/etc..
> > Tested local nifi instance and sample flows. Did not test full secure
> > setup with clustering.
> >
> > This release has quietly turned out to be another beast and the new
> > load balancing capabilities are awesome!
> >
> > Thanks
> > On Thu, Oct 18, 2018 at 7:57 AM Mike Thomsen 
> > wrote:
> > >
> > > (NOTICE, LICENSE and README all looked good when I looked at them; no
> > > problems were apparent)
> > >
> > > On Thu, Oct 18, 2018 at 7:56 AM Mike Thomsen 
> > wrote:
> > >
> > > > +1 binding.
> > > >
> > > > - Checksums and branch id/diff matched up.
> > > > - Ran binary against a Mongo regression test that I made for the new
> > > > client service functionality and everything worked cleanly against
> > > > dockerized Mongo.
> > > >
> > > > On Wed, Oct 17, 2018 at 11:59 PM Jeff  wrote:
> > > >
> > > >> Hello,
> > > >>
> > > >> I am pleased to be calling this vote for the source release of
> Apache
> > NiFi
> > > >> nifi-1.8.0.
> > > >>
> > > >> The source zip, including signatures, digests, etc. can be found at:
> > > >>
> https://repository.apache.org/content/repositories/orgapachenifi-1133
> > > >>
> > > >> The Git tag is nifi-1.8.0-RC1
> > > >> The Git commit ID is 9b02d58626ca874ed2ed3e0bbe530512cfa0dbf8
> > > >>
> > > >>
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=9b02d58626ca874ed2ed3e0bbe530512cfa0dbf8
> > > >>
> > > >> Checksums of nifi-1.8.0-source-release.zip:
> > > >> SHA256:
> > 3ec90a7f153e507d7bba2400d6dafac02641d6f7afc7a954fed959191073ce21
> > > >> SHA512:
> > > >>
> > > >>
> >
> >
> 8b9d944da1833bfb645f502107cab98a555e3b2a7602c5ff438407272c86defdeebe18625c5ad9dfb3f344397314569e97220a35f2438182a79a700caa90721e
> >
> > > >>
> > > >> Release artifacts are signed with the following key:
> > > >> https://people.apache.org/keys/committer/jstorck.asc
> > > >>
> > > >> KEYS file available here:
> > > >> https://dist.apache.org/repos/dist/release/nifi/KEYS
> > > >>
> > > >> 204 issues were closed/resolved for this release:
> > > >>
> > > >>
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12343482
> > > >>
> > > >> Release note highlights can be found here:
> > > >>
> > > >>
> >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.8.0
> > > >>
> > > >> The vote will be open for 72 hours.
> > > >> Please download the release candidate and evaluate the necessary
> items
> > > >> including checking hashes, signatures, build
> > > >> from source, and test. Then please vote:
> > > >>
> > > >> [ ] +1 Release this package as nifi-1.8.0
> > > >> [ ] +0 no opinion
> > > >> [ ] -1 Do not release this package because...
> > > >>
> > > >
> >
>

Re: [VOTE] Release Apache NiFi 1.8.0

2018-10-18 Thread Otto Fowler 
+1

Confirmed signatures
Confirmed hashes
Confirmed source matches commit src
ran build withe check
ran nifi with a template from a bug to verify fix




On October 18, 2018 at 09:45:12, Joe Witt (joe.w...@gmail.com) wrote:

+1 (binding)

Confirmed sigs, hashes, source L, and specified commit present
(including Drew's doc update which is the last code change)
Confirmed full clean build with contrib check and grpc on clean repo/etc..
Tested local nifi instance and sample flows. Did not test full secure
setup with clustering.

This release has quietly turned out to be another beast and the new
load balancing capabilities are awesome!

Thanks
On Thu, Oct 18, 2018 at 7:57 AM Mike Thomsen 
wrote:
>
> (NOTICE, LICENSE and README all looked good when I looked at them; no
> problems were apparent)
>
> On Thu, Oct 18, 2018 at 7:56 AM Mike Thomsen 
wrote:
>
> > +1 binding.
> >
> > - Checksums and branch id/diff matched up.
> > - Ran binary against a Mongo regression test that I made for the new
> > client service functionality and everything worked cleanly against
> > dockerized Mongo.
> >
> > On Wed, Oct 17, 2018 at 11:59 PM Jeff  wrote:
> >
> >> Hello,
> >>
> >> I am pleased to be calling this vote for the source release of Apache
NiFi
> >> nifi-1.8.0.
> >>
> >> The source zip, including signatures, digests, etc. can be found at:
> >> https://repository.apache.org/content/repositories/orgapachenifi-1133
> >>
> >> The Git tag is nifi-1.8.0-RC1
> >> The Git commit ID is 9b02d58626ca874ed2ed3e0bbe530512cfa0dbf8
> >>
> >>
https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=9b02d58626ca874ed2ed3e0bbe530512cfa0dbf8
> >>
> >> Checksums of nifi-1.8.0-source-release.zip:
> >> SHA256:
3ec90a7f153e507d7bba2400d6dafac02641d6f7afc7a954fed959191073ce21
> >> SHA512:
> >>
> >>
8b9d944da1833bfb645f502107cab98a555e3b2a7602c5ff438407272c86defdeebe18625c5ad9dfb3f344397314569e97220a35f2438182a79a700caa90721e

> >>
> >> Release artifacts are signed with the following key:
> >> https://people.apache.org/keys/committer/jstorck.asc
> >>
> >> KEYS file available here:
> >> https://dist.apache.org/repos/dist/release/nifi/KEYS
> >>
> >> 204 issues were closed/resolved for this release:
> >>
> >>
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12343482
> >>
> >> Release note highlights can be found here:
> >>
> >>
https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.8.0
> >>
> >> The vote will be open for 72 hours.
> >> Please download the release candidate and evaluate the necessary items
> >> including checking hashes, signatures, build
> >> from source, and test. Then please vote:
> >>
> >> [ ] +1 Release this package as nifi-1.8.0
> >> [ ] +0 no opinion
> >> [ ] -1 Do not release this package because...
> >>
> >

Re: Apache NiFi 1.8.0 RC1 Release Helper Guide

2018-10-18 Thread Jeff 
Sorry Otto!  The tag [1] should be there now.

[1] 
*https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=tag;h=2fdc819e65481e609532312d18a41d30d58f16f3
*

On Thu, Oct 18, 2018 at 8:34 AM Otto Fowler  wrote:

> I don’t see a tag for nifi-1.8.0-RC1
>
>
> On October 18, 2018 at 00:01:06, Jeff (jsto...@apache.org) wrote:
>
> Hello Apache NiFi community,
>
> Please find the associated guidance to help those interested in
> validating/verifying the release so they can vote.
>
> # Download latest KEYS file:
> https://dist.apache.org/repos/dist/dev/nifi/KEYS
>
> # Import keys file:
> gpg --import KEYS
>
> # [optional] Clear out local maven artifact repository
>
> # Pull down nifi-1.8.0 source release artifacts for review:
>
> wget
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.8.0/nifi-1.8.0-source-release.zip
> wget
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.8.0/nifi-1.8.0-source-release.zip.asc
> wget
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.8.0/nifi-1.8.0-source-release.zip.sha256
> wget
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.8.0/nifi-1.8.0-source-release.zip.sha512
>
> # Verify the signature
> gpg --verify nifi-1.8.0-source-release.zip.asc
>
> # Verify the hashes (sha256, sha512) match the source and what was provided
> in the vote email thread
> shasum -a 256 nifi-1.8.0-source-release.zip
> shasum -a 512 nifi-1.8.0-source-release.zip
>
> # Unzip nifi-1.8.0-source-release.zip
>
> # Verify the build works including release audit tool (RAT) checks
> cd nifi-1.8.0
> mvn clean install -Pcontrib-check,include-grpc
>
> # Verify the contents contain a good README, NOTICE, and LICENSE.
>
> # Verify the git commit ID is correct
>
> # Verify the RC was branched off the correct git commit ID
>
> # Look at the resulting convenience binary as found in nifi-assembly/target
>
> # Make sure the README, NOTICE, and LICENSE are present and correct
>
> # Run the resulting convenience binary and make sure it works as expected
>
> # Send a response to the vote thread indicating a +1, 0, -1 based on your
> findings.
>
> Thank you for your time and effort to validate the release!
>

Re: [VOTE] Release Apache NiFi 1.8.0

2018-10-18 Thread Joe Witt 
+1 (binding)

Confirmed sigs, hashes, source L, and specified commit present
(including Drew's doc update which is the last code change)
Confirmed full clean build with contrib check and grpc on clean repo/etc..
Tested local nifi instance and sample flows.  Did not test full secure
setup with clustering.

This release has quietly turned out to be another beast and the new
load balancing capabilities are awesome!

Thanks
On Thu, Oct 18, 2018 at 7:57 AM Mike Thomsen  wrote:
>
> (NOTICE, LICENSE and README all looked good when I looked at them; no
> problems were apparent)
>
> On Thu, Oct 18, 2018 at 7:56 AM Mike Thomsen  wrote:
>
> > +1 binding.
> >
> > - Checksums and branch id/diff matched up.
> > - Ran binary against a Mongo regression test that I made for the new
> > client service functionality and everything worked cleanly against
> > dockerized Mongo.
> >
> > On Wed, Oct 17, 2018 at 11:59 PM Jeff  wrote:
> >
> >> Hello,
> >>
> >> I am pleased to be calling this vote for the source release of Apache NiFi
> >> nifi-1.8.0.
> >>
> >> The source zip, including signatures, digests, etc. can be found at:
> >> https://repository.apache.org/content/repositories/orgapachenifi-1133
> >>
> >> The Git tag is nifi-1.8.0-RC1
> >> The Git commit ID is 9b02d58626ca874ed2ed3e0bbe530512cfa0dbf8
> >>
> >> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=9b02d58626ca874ed2ed3e0bbe530512cfa0dbf8
> >>
> >> Checksums of nifi-1.8.0-source-release.zip:
> >> SHA256: 3ec90a7f153e507d7bba2400d6dafac02641d6f7afc7a954fed959191073ce21
> >> SHA512:
> >>
> >> 8b9d944da1833bfb645f502107cab98a555e3b2a7602c5ff438407272c86defdeebe18625c5ad9dfb3f344397314569e97220a35f2438182a79a700caa90721e
> >>
> >> Release artifacts are signed with the following key:
> >> https://people.apache.org/keys/committer/jstorck.asc
> >>
> >> KEYS file available here:
> >> https://dist.apache.org/repos/dist/release/nifi/KEYS
> >>
> >> 204 issues were closed/resolved for this release:
> >>
> >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12343482
> >>
> >> Release note highlights can be found here:
> >>
> >> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.8.0
> >>
> >> The vote will be open for 72 hours.
> >> Please download the release candidate and evaluate the necessary items
> >> including checking hashes, signatures, build
> >> from source, and test. Then please vote:
> >>
> >> [ ] +1 Release this package as nifi-1.8.0
> >> [ ] +0 no opinion
> >> [ ] -1 Do not release this package because...
> >>
> >

Re: Apache NiFi 1.8.0 RC1 Release Helper Guide

2018-10-18 Thread Otto Fowler 
I don’t see a tag for nifi-1.8.0-RC1


On October 18, 2018 at 00:01:06, Jeff (jsto...@apache.org) wrote:

Hello Apache NiFi community,

Please find the associated guidance to help those interested in
validating/verifying the release so they can vote.

# Download latest KEYS file:
https://dist.apache.org/repos/dist/dev/nifi/KEYS

# Import keys file:
gpg --import KEYS

# [optional] Clear out local maven artifact repository

# Pull down nifi-1.8.0 source release artifacts for review:

wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.8.0/nifi-1.8.0-source-release.zip
wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.8.0/nifi-1.8.0-source-release.zip.asc
wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.8.0/nifi-1.8.0-source-release.zip.sha256
wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.8.0/nifi-1.8.0-source-release.zip.sha512

# Verify the signature
gpg --verify nifi-1.8.0-source-release.zip.asc

# Verify the hashes (sha256, sha512) match the source and what was provided
in the vote email thread
shasum -a 256 nifi-1.8.0-source-release.zip
shasum -a 512 nifi-1.8.0-source-release.zip

# Unzip nifi-1.8.0-source-release.zip

# Verify the build works including release audit tool (RAT) checks
cd nifi-1.8.0
mvn clean install -Pcontrib-check,include-grpc

# Verify the contents contain a good README, NOTICE, and LICENSE.

# Verify the git commit ID is correct

# Verify the RC was branched off the correct git commit ID

# Look at the resulting convenience binary as found in nifi-assembly/target

# Make sure the README, NOTICE, and LICENSE are present and correct

# Run the resulting convenience binary and make sure it works as expected

# Send a response to the vote thread indicating a +1, 0, -1 based on your
findings.

Thank you for your time and effort to validate the release!

Re: [VOTE] Release Apache NiFi 1.8.0

2018-10-18 Thread Mike Thomsen 
(NOTICE, LICENSE and README all looked good when I looked at them; no
problems were apparent)

On Thu, Oct 18, 2018 at 7:56 AM Mike Thomsen  wrote:

> +1 binding.
>
> - Checksums and branch id/diff matched up.
> - Ran binary against a Mongo regression test that I made for the new
> client service functionality and everything worked cleanly against
> dockerized Mongo.
>
> On Wed, Oct 17, 2018 at 11:59 PM Jeff  wrote:
>
>> Hello,
>>
>> I am pleased to be calling this vote for the source release of Apache NiFi
>> nifi-1.8.0.
>>
>> The source zip, including signatures, digests, etc. can be found at:
>> https://repository.apache.org/content/repositories/orgapachenifi-1133
>>
>> The Git tag is nifi-1.8.0-RC1
>> The Git commit ID is 9b02d58626ca874ed2ed3e0bbe530512cfa0dbf8
>>
>> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=9b02d58626ca874ed2ed3e0bbe530512cfa0dbf8
>>
>> Checksums of nifi-1.8.0-source-release.zip:
>> SHA256: 3ec90a7f153e507d7bba2400d6dafac02641d6f7afc7a954fed959191073ce21
>> SHA512:
>>
>> 8b9d944da1833bfb645f502107cab98a555e3b2a7602c5ff438407272c86defdeebe18625c5ad9dfb3f344397314569e97220a35f2438182a79a700caa90721e
>>
>> Release artifacts are signed with the following key:
>> https://people.apache.org/keys/committer/jstorck.asc
>>
>> KEYS file available here:
>> https://dist.apache.org/repos/dist/release/nifi/KEYS
>>
>> 204 issues were closed/resolved for this release:
>>
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12343482
>>
>> Release note highlights can be found here:
>>
>> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.8.0
>>
>> The vote will be open for 72 hours.
>> Please download the release candidate and evaluate the necessary items
>> including checking hashes, signatures, build
>> from source, and test. Then please vote:
>>
>> [ ] +1 Release this package as nifi-1.8.0
>> [ ] +0 no opinion
>> [ ] -1 Do not release this package because...
>>
>

Re: [VOTE] Release Apache NiFi 1.8.0

2018-10-18 Thread Mike Thomsen 
+1 binding.

- Checksums and branch id/diff matched up.
- Ran binary against a Mongo regression test that I made for the new client
service functionality and everything worked cleanly against dockerized
Mongo.

On Wed, Oct 17, 2018 at 11:59 PM Jeff  wrote:

> Hello,
>
> I am pleased to be calling this vote for the source release of Apache NiFi
> nifi-1.8.0.
>
> The source zip, including signatures, digests, etc. can be found at:
> https://repository.apache.org/content/repositories/orgapachenifi-1133
>
> The Git tag is nifi-1.8.0-RC1
> The Git commit ID is 9b02d58626ca874ed2ed3e0bbe530512cfa0dbf8
>
> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=9b02d58626ca874ed2ed3e0bbe530512cfa0dbf8
>
> Checksums of nifi-1.8.0-source-release.zip:
> SHA256: 3ec90a7f153e507d7bba2400d6dafac02641d6f7afc7a954fed959191073ce21
> SHA512:
>
> 8b9d944da1833bfb645f502107cab98a555e3b2a7602c5ff438407272c86defdeebe18625c5ad9dfb3f344397314569e97220a35f2438182a79a700caa90721e
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/jstorck.asc
>
> KEYS file available here:
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
> 204 issues were closed/resolved for this release:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12343482
>
> Release note highlights can be found here:
>
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.8.0
>
> The vote will be open for 72 hours.
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build
> from source, and test. Then please vote:
>
> [ ] +1 Release this package as nifi-1.8.0
> [ ] +0 no opinion
> [ ] -1 Do not release this package because...
>

Re: Scaling source processors in nifi horizontally.

2018-10-18 Thread Mike Thomsen 
I initially thought you were saying that you had 250 Avro schemas that you
had to use, as in 250 different distinct data models.

Maybe someone else has a suggestion on how to do it, but I think this may
just be a fundamental problem of having that many different databases in
MySQL and trying to do CDC with them.

Is there a hard business requirement to segregate data like that or some
factor like pulling from many remote databases that is at play here?

On Thu, Oct 18, 2018 at 6:19 AM ashwin konale 
wrote:

> Hi,
>
> The flow is like this,
>
> MysqlCDC -> UpdateAttributes -> MergeContent -> (PutHDFS, PutGCS)
>
> But we have around 250 schemas to pull data from, So with clustering setup,
>
> MysqlCDC_schema1 -> RPG
> MysqlCDC_schema2 -> RPG
> MysqlCDC_schema3 -> RPG and so on
>
> InputPort -> UpdateAttributes -> MergeContent -> (PutHDFS, PutGCS)
>
> But MysqlCDC can run only in primary node in the cluster, I will end up
> running all of input processors in single node. This can easily become
> bottleneck with increasing number of schemas we have. Could you suggest me
> any alternative approach to this problem.
>
> On 2018/10/17 21:14:09, Mike Thomsen  wrote:
> > > may have to build some kind of tooling on top of it to
> monitor/provision>
> > new processor for newly added schemas etc.>
> >
> > Could you elaborate on this part of your use case?>
> >
> > On Wed, Oct 17, 2018 at 2:31 PM ashwin konale >
> > wrote:>
> >
> > > Hi,>
> > >>
> > > I am experimenting with nifi for one of our usecases with plans of>
> > > extending it to various other data routing, ingestion usecases. Right
> now I>
> > > need to ingest data from mysql binlogs to hdfs/GCS. We have around 250>
> > > different schemas and about 3000 tables to read data from. Volume of
> the>
> > > data flow ranges from 500 - 2000 messages per second in different
> schemas.>
> > >>
> > > Right now the problem is mysqlCDC processor can run in only one thread.
> To>
> > > overcome this issue I have two options.>
> > >>
> > > 1. Use primary node execution, so different processors for each of the>
> > > schemas. So eventually all processors which reads from mysql will run
> in>
> > > single node, which will be a bottleneck no matter how big my nifi
> cluster>
> > > is.>
> > >>
> > > 2. Another approach is to use multiple nifi instances to pull data and
> have>
> > > master nifi cluster for ingestion to various sinks. In this approach I
> will>
> > > have to manage all these small nifi instances, and may have to build
> some>
> > > kind of tooling on top of it to monitor/provision new processor for
> newly>
> > > added schemas etc.>
> > >>
> > > Is there any better way to achieve my usecase with nifi ? Please advice
> me>
> > > on the architechture.>
> > >>
> > > Looking forward for suggestion.>
> > >>
> > > - Ashwin>
> > >>
> >
>

Re: Scaling source processors in nifi horizontally.

2018-10-18 Thread ashwin konale 
Hi,

The flow is like this,

MysqlCDC -> UpdateAttributes -> MergeContent -> (PutHDFS, PutGCS)

But we have around 250 schemas to pull data from, So with clustering setup,

MysqlCDC_schema1 -> RPG
MysqlCDC_schema2 -> RPG
MysqlCDC_schema3 -> RPG and so on

InputPort -> UpdateAttributes -> MergeContent -> (PutHDFS, PutGCS)

But MysqlCDC can run only in primary node in the cluster, I will end up
running all of input processors in single node. This can easily become
bottleneck with increasing number of schemas we have. Could you suggest me
any alternative approach to this problem.

On 2018/10/17 21:14:09, Mike Thomsen  wrote:
> > may have to build some kind of tooling on top of it to
monitor/provision>
> new processor for newly added schemas etc.>
>
> Could you elaborate on this part of your use case?>
>
> On Wed, Oct 17, 2018 at 2:31 PM ashwin konale >
> wrote:>
>
> > Hi,>
> >>
> > I am experimenting with nifi for one of our usecases with plans of>
> > extending it to various other data routing, ingestion usecases. Right
now I>
> > need to ingest data from mysql binlogs to hdfs/GCS. We have around 250>
> > different schemas and about 3000 tables to read data from. Volume of
the>
> > data flow ranges from 500 - 2000 messages per second in different
schemas.>
> >>
> > Right now the problem is mysqlCDC processor can run in only one thread.
To>
> > overcome this issue I have two options.>
> >>
> > 1. Use primary node execution, so different processors for each of the>
> > schemas. So eventually all processors which reads from mysql will run
in>
> > single node, which will be a bottleneck no matter how big my nifi
cluster>
> > is.>
> >>
> > 2. Another approach is to use multiple nifi instances to pull data and
have>
> > master nifi cluster for ingestion to various sinks. In this approach I
will>
> > have to manage all these small nifi instances, and may have to build
some>
> > kind of tooling on top of it to monitor/provision new processor for
newly>
> > added schemas etc.>
> >>
> > Is there any better way to achieve my usecase with nifi ? Please advice
me>
> > on the architechture.>
> >>
> > Looking forward for suggestion.>
> >>
> > - Ashwin>
> >>
>

Re: Scaling source processors in nifi horizontally.

2018-10-18 Thread Ashwin Konale 
Hi,

The flow is like this,

MysqlCDC -> UpdateAttributes -> MergeContent -> (PutHDFS, PutGCS)

But we have around 250 schemas to pull data from, So with clustering setup,

MysqlCDC_schema1 -> RPG
MysqlCDC_schema2 -> RPG
MysqlCDC_schema3 -> RPG and so on

InputPort -> UpdateAttributes -> MergeContent -> (PutHDFS, PutGCS)

But MysqlCDC can run only in primary node in the cluster, I will end up running 
all of input processors in single node. This can easily become bottleneck with 
increasing number of schemas we have. Could you suggest me any alternative 
approach to this problem.


On 2018/10/17 21:14:09, Mike Thomsen  wrote: 
> > may have to build some kind of tooling on top of it to monitor/provision> 
> new processor for newly added schemas etc.> 
> 
> Could you elaborate on this part of your use case?> 
> 
> On Wed, Oct 17, 2018 at 2:31 PM ashwin konale > 
> wrote:> 
> 
> > Hi,> 
> >> 
> > I am experimenting with nifi for one of our usecases with plans of> 
> > extending it to various other data routing, ingestion usecases. Right now 
> > I> 
> > need to ingest data from mysql binlogs to hdfs/GCS. We have around 250> 
> > different schemas and about 3000 tables to read data from. Volume of the> 
> > data flow ranges from 500 - 2000 messages per second in different schemas.> 
> >> 
> > Right now the problem is mysqlCDC processor can run in only one thread. To> 
> > overcome this issue I have two options.> 
> >> 
> > 1. Use primary node execution, so different processors for each of the> 
> > schemas. So eventually all processors which reads from mysql will run in> 
> > single node, which will be a bottleneck no matter how big my nifi cluster> 
> > is.> 
> >> 
> > 2. Another approach is to use multiple nifi instances to pull data and 
> > have> 
> > master nifi cluster for ingestion to various sinks. In this approach I 
> > will> 
> > have to manage all these small nifi instances, and may have to build some> 
> > kind of tooling on top of it to monitor/provision new processor for newly> 
> > added schemas etc.> 
> >> 
> > Is there any better way to achieve my usecase with nifi ? Please advice me> 
> > on the architechture.> 
> >> 
> > Looking forward for suggestion.> 
> >> 
> > - Ashwin> 
> >> 
>