Re: Apache NiFi 1.5.0 (RC1) Release Helper Guide

[Joe Witt]

Thanks for fixing that directory/link Aldrin!

JeffZ: With regard to your ask on hipchat "Hi guys. Building 1.5.0 RC
on OSX and I get a few failing tests in TestListenSMTP. The tests pass
on Ubuntu. Just wondering if those are known failures? I didn't see
any open JIRAs with a quick search."

I responded with

"[10:50 PM] Joe Witt: not known errors.  Please share the build
command you're using
[10:51 PM] Joe Witt: Those tests have been problematic in the past
because they started on a specific port and in parallel build cases
the port might be in use.  However, some changes were made to improve
that.  We probably need to just make them be integration tests only.
Please do file a JIRA with your observations"

Thanks

On Tue, Jan 9, 2018 at 6:09 AM, Aldrin Piri  wrote:
> Looks like there was an extra nifi directory created.  I moved them in the
> repo to reflect the paths in the helper email.
>
>
> On Tue, Jan 9, 2018 at 7:52 AM, Jeff Zemerick  wrote:
>
>> Hi Joe,
>>
>> I'm getting a 404 on those download links.
>>
>> Thanks
>>
>>
>> On Tue, Jan 9, 2018 at 5:19 AM, Joe Witt  wrote:
>>
>> > Hello Apache NiFi community,
>> >
>> > Please find the associated guidance to help those interested in
>> > validating/verifying the release so they can vote.
>> >
>> > # Download latest KEYS file:
>> > https://dist.apache.org/repos/dist/dev/nifi/KEYS
>> >
>> > # Import keys file:
>> > gpg --import KEYS
>> >
>> > # [optional] Clear out local maven artifact repository
>> >
>> > # Pull down nifi-1.5.0 source release artifacts for review:
>> >
>> > wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-
>> > 1.5.0-source-release.zip
>> > wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-
>> > 1.5.0-source-release.zip.asc
>> > wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-
>> > 1.5.0-source-release.zip.md5
>> > wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-
>> > 1.5.0-source-release.zip.sha1
>> > wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-
>> > 1.5.0-source-release.zip.sha256
>> >
>> > # Verify the signature
>> > gpg --verify nifi-1.5.0-source-release.zip.asc
>> >
>> > # Verify the hashes (md5, sha1, sha256) match the source and what was
>> > provided in the vote email thread
>> > md5sum nifi-1.5.0-source-release.zip
>> > sha1sum nifi-1.5.0-source-release.zip
>> > sha256sum nifi-1.5.0-source-release.zip
>> >
>> > # Unzip nifi-1.5.0-source-release.zip
>> >
>> > # Verify the build works including release audit tool (RAT) checks
>> > cd nifi-1.5.0
>> > mvn clean install -Pcontrib-check,include-grpc
>> >
>> > # Verify the contents contain a good README, NOTICE, and LICENSE.
>> >
>> > # Verify the git commit ID is correct
>> >
>> > # Verify the RC was branched off the correct git commit ID
>> >
>> > # Look at the resulting convenience binary as found in
>> nifi-assembly/target
>> >
>> > # Make sure the README, NOTICE, and LICENSE are present and correct
>> >
>> > # Run the resulting convenience binary and make sure it works as expected
>> >
>> > # Send a response to the vote thread indicating a +1, 0, -1 based on
>> > your findings.
>> >
>> > Please note that the convenience binaries and artifacts are provided
>> > except for the tar.gz/zip of the actual nifi build.
>> > It is simply too large to upload at this time.  But once
>> > https://issues.apache.org/jira/browse/INFRA-15816 is
>> > addressed I will upload it.
>> >
>> > Thank you for your time and effort to validate the release!
>> >
>>

Re: [VOTE] Release Apache NiFi 1.5.0 (RC1)

[Jeremy Dyer]

+1 non binding. Ran through release guide with no issues and ran several test 
workflows from personal projects

- Jeremy Dyer

> On Jan 9, 2018, at 8:11 PM, Andy LoPresto  wrote:
> 
> +1, binding
> 
> I performed the following verifications:
> 
> * verified the GPG key signature
> * verified the MD5/SHA1/SHA256 checksums
> * verified the git commit ID
> * verified the presence of LICENSE, NOTICE, and README
> * verified the Maven build was successful, including all unit tests, RAT 
> check, and code style guidelines
> * verified the application started successfully
> * verified the new behavior for host header handling was present
> * verified the application integrates successfully with the Apache NiFi 
> Registry
> * verified the CountText processor works as anticipated
> * verified the flow versioning functionality by making local changes, 
> committing, and restoring prior versions of the flow
> 
> Andy LoPresto
> alopre...@apache.org
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> 
>> On Jan 9, 2018, at 9:13 AM, Mark Payne  wrote:
>> 
>> +1 (binding) - Release this package as nifi-1.5.0
>> 
>> Was able to successfully build with contrib-check and grpc profile. Verified 
>> MD5 sum.
>> Started up, added a Flow Registry Client pointing to an existing Flow 
>> Registry and was
>> able to immediately import a flow and start it running. This flow also 
>> verified the fix for
>> NIFI-4749 made it into the build yesterday.
>> 
>> Thanks for taking on the RM duties this time, Joe! And a big thank you and 
>> congrats
>> to all of the members of the community that contributed to this release - 
>> code, JIRA's,
>> docs, and support!
>> 
>> Thanks
>> -Mark
>> 
>> 
>>> On Jan 9, 2018, at 5:19 AM, Joe Witt  wrote:
>>> 
>>> Hello,
>>> 
>>> I am pleased to be calling this vote for the source release of Apache
>>> NiFi nifi-1.5.0.
>>> 
>>> The source zip, including signatures, digests, etc. can be found at:
>>> https://repository.apache.org/content/repositories/orgapachenifi-1116
>>> 
>>> The Git tag is nifi-1.5.0-RC1
>>> The Git commit ID is 46d30c7e92f0ad034d9b35bf1d05c350ab5547ed
>>> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=46d30c7e92f0ad034d9b35bf1d05c350ab5547ed
>>> 
>>> Checksums of nifi-1.5.0-source-release.zip:
>>> MD5: 046f2dde4af592dd8c05e55c2bbb3c4f
>>> SHA1: 63b9a68b9f89200fd31f5561956a15b45b1b9c8c
>>> SHA256: 40b155c4911414907835f2eb0d5a4da798935f27f1e5134218d904fe6c942d13
>>> 
>>> Release artifacts are signed with the following key:
>>> https://people.apache.org/keys/committer/joewitt.asc
>>> 
>>> KEYS file available here:
>>> https://dist.apache.org/repos/dist/release/nifi/KEYS
>>> 
>>> 195 issues were closed/resolved for this release:
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12341668
>>> 
>>> Release note highlights can be found here:
>>> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.5.0
>>> 
>>> The vote will be open for 72 hours.
>>> Please download the release candidate and evaluate the necessary items
>>> including checking hashes, signatures, build
>>> from source, and test.  The please vote:
>>> 
>>> [ ] +1 Release this package as nifi-1.5.0
>>> [ ] +0 no opinion
>>> [ ] -1 Do not release this package because...
>> 
> 

Re: [VOTE] Release Apache NiFi 1.5.0 (RC1)

[Andy LoPresto]

+1, binding

I performed the following verifications:

* verified the GPG key signature
* verified the MD5/SHA1/SHA256 checksums
* verified the git commit ID
* verified the presence of LICENSE, NOTICE, and README
* verified the Maven build was successful, including all unit tests, RAT check, 
and code style guidelines
* verified the application started successfully
* verified the new behavior for host header handling was present
* verified the application integrates successfully with the Apache NiFi Registry
* verified the CountText processor works as anticipated
* verified the flow versioning functionality by making local changes, 
committing, and restoring prior versions of the flow

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Jan 9, 2018, at 9:13 AM, Mark Payne  wrote:
> 
> +1 (binding) - Release this package as nifi-1.5.0
> 
> Was able to successfully build with contrib-check and grpc profile. Verified 
> MD5 sum.
> Started up, added a Flow Registry Client pointing to an existing Flow 
> Registry and was
> able to immediately import a flow and start it running. This flow also 
> verified the fix for
> NIFI-4749 made it into the build yesterday.
> 
> Thanks for taking on the RM duties this time, Joe! And a big thank you and 
> congrats
> to all of the members of the community that contributed to this release - 
> code, JIRA's,
> docs, and support!
> 
> Thanks
> -Mark
> 
> 
>> On Jan 9, 2018, at 5:19 AM, Joe Witt  wrote:
>> 
>> Hello,
>> 
>> I am pleased to be calling this vote for the source release of Apache
>> NiFi nifi-1.5.0.
>> 
>> The source zip, including signatures, digests, etc. can be found at:
>> https://repository.apache.org/content/repositories/orgapachenifi-1116
>> 
>> The Git tag is nifi-1.5.0-RC1
>> The Git commit ID is 46d30c7e92f0ad034d9b35bf1d05c350ab5547ed
>> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=46d30c7e92f0ad034d9b35bf1d05c350ab5547ed
>> 
>> Checksums of nifi-1.5.0-source-release.zip:
>> MD5: 046f2dde4af592dd8c05e55c2bbb3c4f
>> SHA1: 63b9a68b9f89200fd31f5561956a15b45b1b9c8c
>> SHA256: 40b155c4911414907835f2eb0d5a4da798935f27f1e5134218d904fe6c942d13
>> 
>> Release artifacts are signed with the following key:
>> https://people.apache.org/keys/committer/joewitt.asc
>> 
>> KEYS file available here:
>> https://dist.apache.org/repos/dist/release/nifi/KEYS
>> 
>> 195 issues were closed/resolved for this release:
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12341668
>> 
>> Release note highlights can be found here:
>> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.5.0
>> 
>> The vote will be open for 72 hours.
>> Please download the release candidate and evaluate the necessary items
>> including checking hashes, signatures, build
>> from source, and test.  The please vote:
>> 
>> [ ] +1 Release this package as nifi-1.5.0
>> [ ] +0 no opinion
>> [ ] -1 Do not release this package because...
> 



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: [DRAFT][REPORT] Apache NiFi Board Report Jan 2018

[Tony Kurc]

Looks great! Thanks for pulling this together, Joe!

On Jan 9, 2018 12:40 PM, "Joe Witt"  wrote:

> Thanks for the prompt replies all.  Board Report for Jan submitted.
>
> ## Description:
>  - Apache NiFi is an easy to use, powerful, and reliable system to
>process and distribute data.
>  - Apache NiFi MiNiFi is an edge data collection agent built to seamlessly
>integrate with and leverage the command and control of NiFi. There are
>both Java and C++ implementations.
>  - Apache NiFi Registry is a centralized registry for key configuration
> items
>including flow versions, assets, and extensions for Apache NiFi
>and Apache MiNiFi.
>  - Apache NiFi Nar Maven Plugin is a release artifact used for supporting
>the NiFi classloader isolation model.
>
> ## Issues:
>  - There are no issues requiring board attention at this time.
>
> ## Activity:
>  - Conducted several releases including Apache NiFi Registry,
>Apache NiFi MiNiFi CPP and Apache NiFi MiNiFi Java. The Apache NiFi
> Registry
>release is a very important step forward for the community as it
> provides a
>powerful software development lifecycle mechanism for Apache NiFi users
> and
>answers a long standing request and often discussed capability.
>  - Apache NiFi 1.5.0 Release Candidate under vote at the time of this board
>report writing.
>  - Voted a significant number of new committers and PMC members and are
>extremely fortunate to see strong committer pipeline continue through
> newer
>contributors with significant contributions.
>  - The PMC has identified and worked with Trademarks/Legal at Apache for a
>couple of potential trademark or licensing abuse issues.  Consensus was
> that
>neither issue rises to the level of requiring any specific official
> response
>though in one case we intend to provide a courtesy notification to a
>commercial entity that is heavily leverage Apache NiFi source code
> without
>any attribution.  It is more of a poor form issue than a real problem.
>Thanks to several highly experienced Apache Members who helped discuss
> the
>threads in detail.
>
> ## Health report:
>  - Health of the community remains strong. Mailing list and JIRA activity
> is
>consistent. ASF Hipchat is serving as an on-ramp for new users to our
>mailing list and JIRA systems. We continue to see new users and
>contributors.
>  - The PMC continues to demonstrate ability to detect and handle potential
>trademark issues though it needs to focus better on staying tight and
> prompt
>on security matters all the way through resolution.
>  - As we called out in our previous report we anticipated significant
>recognition of earned meritocracy within the committer and PMC ranks.
> That
>held true and the pipeline remains encouraging.
>
> ## PMC changes:
>
>  - Currently 26 PMC members.
>  - New PMC members:
> - Marc Parisi was added to the PMC on Wed Dec 13 2017
> - Jeff Storck was added to the PMC on Mon Dec 04 2017
> - Scott Aslan was added to the PMC on Fri Dec 01 2017
> - Michael W Moser was added to the PMC on Sun Nov 19 2017
>  - Last PMC member added Wed Dec 13 2017.
>
> ## Committer base changes:
>
>  - Currently 38 committers.
>  - New commmitters:
> - Kevin Doran was added as a committer on Wed Jan 03 2018
> - Andrew Ian Christianson was added as a committer on Mon Nov 13 2017
> - Mike Hogue was added as a committer on Thu Nov 09 2017
> - Peter Wicks was added as a committer on Thu Nov 09 2017
>  - Last committer added Wed Jan 03 2018.
>
> ## Releases:
>
>  - Apache NiFi Registry 0.1.0 was released Jan 1 2018.
>  - Apache NiFi MiNiFi Java 0.3.0 was released Dec 22 2017.
>  - Apache NiFi MiNiFi CPP 0.3.0 was released Nov 30 2017.
>
> ## Mailing list activity:
>
>  - Activity on the mailing lists remains high with a mixture
>of new users, contributors, and deeper more experienced users and
>contributors sparking discussion and questions and filing bugs or
>new features.
>
>  - us...@nifi.apache.org:
> - 581 subscribers (up 25 in the last 3 months):
> - 783 emails sent to list (721 in previous quarter)
>
>  - dev@nifi.apache.org:
> - 388 subscribers (up 5 in the last 3 months):
> - 679 emails sent to list (590 in previous quarter)
>
>  - iss...@nifi.apache.org:
> - 46 subscribers (up 0 in the last 3 months):
> - 6950 emails sent to list (5388 in previous quarter)
>
>
> ## JIRA activity:
>
>  - 356 JIRA tickets created in the last 3 months
>  - 276 JIRA tickets closed/resolved in the last 3 months
>
> On Tue, Jan 9, 2018 at 9:57 AM, Andrew Lim 
> wrote:
> > +1 LGTM
> >
> > -Drew
> >
> >> On Jan 9, 2018, at 2:15 AM, Joe Witt  wrote:
> >>
> >> Team,
> >>
> >> It is that time again to send in our board report.  I'll submit it
> >> tomorrow but would appreciate if you can take a quick look and point
> >> out things 

Re: FlowFile Routing to Failure

[Brett Ryan]

> On 10 Jan 2018, at 04:29, Jean De Leon  wrote:
> 
> What are reasons why a FlowFile will route to failure without prompting an 
> error message?

If there was no exception reported or any error logs emitted by the processor 
then I think nothing will appear in the bulletin board. As an example, I was 
using InvokeHTTP that would not emit an error on a 500 response, in that case 
you can read invokehttp.status.code and invokehttp.status.message.

Re: [DRAFT][REPORT] Apache NiFi Board Report Jan 2018

[Joe Witt]

Thanks for the prompt replies all.  Board Report for Jan submitted.

## Description:
 - Apache NiFi is an easy to use, powerful, and reliable system to
   process and distribute data.
 - Apache NiFi MiNiFi is an edge data collection agent built to seamlessly
   integrate with and leverage the command and control of NiFi. There are
   both Java and C++ implementations.
 - Apache NiFi Registry is a centralized registry for key configuration items
   including flow versions, assets, and extensions for Apache NiFi
   and Apache MiNiFi.
 - Apache NiFi Nar Maven Plugin is a release artifact used for supporting
   the NiFi classloader isolation model.

## Issues:
 - There are no issues requiring board attention at this time.

## Activity:
 - Conducted several releases including Apache NiFi Registry,
   Apache NiFi MiNiFi CPP and Apache NiFi MiNiFi Java. The Apache NiFi Registry
   release is a very important step forward for the community as it provides a
   powerful software development lifecycle mechanism for Apache NiFi users and
   answers a long standing request and often discussed capability.
 - Apache NiFi 1.5.0 Release Candidate under vote at the time of this board
   report writing.
 - Voted a significant number of new committers and PMC members and are
   extremely fortunate to see strong committer pipeline continue through newer
   contributors with significant contributions.
 - The PMC has identified and worked with Trademarks/Legal at Apache for a
   couple of potential trademark or licensing abuse issues.  Consensus was that
   neither issue rises to the level of requiring any specific official response
   though in one case we intend to provide a courtesy notification to a
   commercial entity that is heavily leverage Apache NiFi source code without
   any attribution.  It is more of a poor form issue than a real problem.
   Thanks to several highly experienced Apache Members who helped discuss the
   threads in detail.

## Health report:
 - Health of the community remains strong. Mailing list and JIRA activity is
   consistent. ASF Hipchat is serving as an on-ramp for new users to our
   mailing list and JIRA systems. We continue to see new users and
   contributors.
 - The PMC continues to demonstrate ability to detect and handle potential
   trademark issues though it needs to focus better on staying tight and prompt
   on security matters all the way through resolution.
 - As we called out in our previous report we anticipated significant
   recognition of earned meritocracy within the committer and PMC ranks.  That
   held true and the pipeline remains encouraging.

## PMC changes:

 - Currently 26 PMC members.
 - New PMC members:
- Marc Parisi was added to the PMC on Wed Dec 13 2017
- Jeff Storck was added to the PMC on Mon Dec 04 2017
- Scott Aslan was added to the PMC on Fri Dec 01 2017
- Michael W Moser was added to the PMC on Sun Nov 19 2017
 - Last PMC member added Wed Dec 13 2017.

## Committer base changes:

 - Currently 38 committers.
 - New commmitters:
- Kevin Doran was added as a committer on Wed Jan 03 2018
- Andrew Ian Christianson was added as a committer on Mon Nov 13 2017
- Mike Hogue was added as a committer on Thu Nov 09 2017
- Peter Wicks was added as a committer on Thu Nov 09 2017
 - Last committer added Wed Jan 03 2018.

## Releases:

 - Apache NiFi Registry 0.1.0 was released Jan 1 2018.
 - Apache NiFi MiNiFi Java 0.3.0 was released Dec 22 2017.
 - Apache NiFi MiNiFi CPP 0.3.0 was released Nov 30 2017.

## Mailing list activity:

 - Activity on the mailing lists remains high with a mixture
   of new users, contributors, and deeper more experienced users and
   contributors sparking discussion and questions and filing bugs or
   new features.

 - us...@nifi.apache.org:
- 581 subscribers (up 25 in the last 3 months):
- 783 emails sent to list (721 in previous quarter)

 - dev@nifi.apache.org:
- 388 subscribers (up 5 in the last 3 months):
- 679 emails sent to list (590 in previous quarter)

 - iss...@nifi.apache.org:
- 46 subscribers (up 0 in the last 3 months):
- 6950 emails sent to list (5388 in previous quarter)


## JIRA activity:

 - 356 JIRA tickets created in the last 3 months
 - 276 JIRA tickets closed/resolved in the last 3 months

On Tue, Jan 9, 2018 at 9:57 AM, Andrew Lim  wrote:
> +1 LGTM
>
> -Drew
>
>> On Jan 9, 2018, at 2:15 AM, Joe Witt  wrote:
>>
>> Team,
>>
>> It is that time again to send in our board report.  I'll submit it
>> tomorrow but would appreciate if you can take a quick look and point
>> out things to tweak or add or remove.  Really proud of what we're
>> accomplishing!
>>
>> Thanks
>> Joe
>>
>> -=-=-=-=-=-=-
>>
>> ## Description:
>> - Apache NiFi is an easy to use, powerful, and reliable system to
>>   process and distribute data.
>> - Apache NiFi MiNiFi is an edge data collection agent built to seamlessly
>>   integrate with and leverage 

Re: FlowFile Routing to Failure

[Aldrin Piri]

Could you please provide the particular processors that are exhibiting this
behavior?

On Tue, Jan 9, 2018 at 12:29 PM, Jean De Leon  wrote:

> What are reasons why a FlowFile will route to failure without prompting an
> error message?
>
>
> v/r,
>
> Jean De Leon
>

FlowFile Routing to Failure

[Jean De Leon]

What are reasons why a FlowFile will route to failure without prompting an 
error message?


v/r,

Jean De Leon

Re: [VOTE] Release Apache NiFi 1.5.0 (RC1)

[Mark Payne]

+1 (binding) - Release this package as nifi-1.5.0

Was able to successfully build with contrib-check and grpc profile. Verified 
MD5 sum.
Started up, added a Flow Registry Client pointing to an existing Flow Registry 
and was
able to immediately import a flow and start it running. This flow also verified 
the fix for
NIFI-4749 made it into the build yesterday.

Thanks for taking on the RM duties this time, Joe! And a big thank you and 
congrats
to all of the members of the community that contributed to this release - code, 
JIRA's,
docs, and support!

Thanks
-Mark


> On Jan 9, 2018, at 5:19 AM, Joe Witt  wrote:
> 
> Hello,
> 
> I am pleased to be calling this vote for the source release of Apache
> NiFi nifi-1.5.0.
> 
> The source zip, including signatures, digests, etc. can be found at:
> https://repository.apache.org/content/repositories/orgapachenifi-1116
> 
> The Git tag is nifi-1.5.0-RC1
> The Git commit ID is 46d30c7e92f0ad034d9b35bf1d05c350ab5547ed
> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=46d30c7e92f0ad034d9b35bf1d05c350ab5547ed
> 
> Checksums of nifi-1.5.0-source-release.zip:
> MD5: 046f2dde4af592dd8c05e55c2bbb3c4f
> SHA1: 63b9a68b9f89200fd31f5561956a15b45b1b9c8c
> SHA256: 40b155c4911414907835f2eb0d5a4da798935f27f1e5134218d904fe6c942d13
> 
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/joewitt.asc
> 
> KEYS file available here:
> https://dist.apache.org/repos/dist/release/nifi/KEYS
> 
> 195 issues were closed/resolved for this release:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12341668
> 
> Release note highlights can be found here:
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.5.0
> 
> The vote will be open for 72 hours.
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build
> from source, and test.  The please vote:
> 
> [ ] +1 Release this package as nifi-1.5.0
> [ ] +0 no opinion
> [ ] -1 Do not release this package because...

Re: PutMongo 1.3.0 - Upsert issue

[Pierre Villard]

Yes, if you have a running 1.5.0-SNAPSHOT NiFi running, then you could get
the NAR from the lib directory and just drop it into the directory in your
1.3 instance. Note that it's not recommended to put custom NARs into the
lib directory. Best is to define an external directory for custom NARs in
nifi.properties and drop your files in there. Note that you'll need to
restart NiFi.

If you don't have a NiFi 1.5.0-SNAPSHOT running, then you could use the
current release candidate, and follow the below process:

# Pull down nifi-1.5.0 source release:
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-
1.5.0-source-release.zip

# Unzip nifi-1.5.0-source-release.zip

# Build the sources
cd nifi-1.5.0
mvn clean install -DskipTests

Pierre


2018-01-09 17:24 GMT+01:00 fabe_bdx :

> Thanks for the repply, but i don't have any knowledge to complile NAR
> object
> :/
>
> is it possible to extract them of 1.5.0 and push it into nifi 1.3.0
> directly
> ?
>
> Thx,
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>

Re: Cross NAR Controller Service access with gradle

[Matt Burgess]

Nicholas,

You shouldn't need to reference StandardSSLContextService explicitly,
as your processor should be using the interface SSLContextService
only.  In the Maven world, this is done by bringing in
nifi-ssl-context-service-api as a "provided" dependency (not compile),
and adding a dependency on your NAR module to the
nifi-standard-services-api-nar NAR. This is handled by a custom
NarMojo for doing NiFi ARchive (NAR) packaging [1].

In the Gradle world we'll need a task to do the same thing, I believe
it is mostly a matter of getting the separate subprojects set up to
mimic the Maven modules, and a task (whether published as a plugin
like the NarMojo plugin is for Maven or not) to update the NAR
manifest and such.  I have begun work on this [2], please feel free to
take a look and let me know what you think. At present I believe I am
missing a directive to copy all the processors' subproject
dependencies into the NAR output artifact, I think I know how to fix
it (i.e. from project(':processors').tasks['jar'].archivePath) but
haven't had time to try it out.

Please let me know if you'd like to work on the Gradle stuff together,
I would be happy to collaborate :)

Regards,
Matt

[1] https://github.com/apache/nifi-maven
[2] https://github.com/mattyb149/nifi-gradle-nar-example

On Tue, Jan 9, 2018 at 11:47 AM, Nicholas Deane  wrote:
> Hi,
>
> I’m working on a custom processor and want to utilize NiFi’s 
> SSLContextService.
>
> In my build.gradle dependencies section, I’ve included:
>   compile "org.apache.nifi:nifi-ssl-context-service-api:$nifiVersion"
>
> and in my code I’ve created the property descriptor with 
> .identifiesControllerService(SSLContextService.class)
>
> When I’m configuring the processor, I noticed that the property on the 
> processor says that it “Requires Controller Service: SSLContextService from 
> my-custom-nar”. When I create a StandardSSLContextService from the default 
> org.apache.nifi bundle, it doesn’t show up in the processor’s list (as 
> expected based on that info message).
>
> I tried bringing in “nifi-ssl-context-service” and now I have two versions of 
> StandardSSLContextService in the controller service list (one from the 
> org.apache.nifi bundle and the other from my-custom-nar bundle). Is this how 
> I should be doing something like this or is there a better way for me to use 
> Controller Services across bundles and avoid having duplicates in that list?
>
>
> Notice - Confidential Information The information in this communication and 
> any attachments is strictly confidential and intended only for the use of the 
> individual(s) or entity(ies) named above. If you are not the intended 
> recipient, any dissemination, distribution, copying or other use of the 
> information contained in this communication and/or any attachment is strictly 
> prohibited. If you have received this communication in error, please first 
> notify the sender immediately and then delete this communication from all 
> data storage devices and destroy all hard copies.

Re: [EXT] Re: processorLoadAverage In SystemDiagnostics

[Mark Payne]

Karthik,

No, the JVM does not supply this information. You'd need to use some other 
host-level monitoring service, such as Ganglia, to gather that kind of info.

Thanks
-Mark

> On Jan 9, 2018, at 11:45 AM, Karthik Kothareddy (karthikk) [CONT - Type 2] 
>  wrote:
> 
> Mark,
> 
> Thanks for quick response, Is there a way that NiFi provides these metrics 
> individually for a processor that the average load ? Or can we use any REST 
> endpoint to grab the processor-wise metrics?
> 
> -Karthik
> 
> -Original Message-
> From: Mark Payne [mailto:marka...@hotmail.com] 
> Sent: Tuesday, January 09, 2018 9:22 AM
> To: dev@nifi.apache.org
> Subject: [EXT] Re: processorLoadAverage In SystemDiagnostics 
> 
> Hi Karthik,
> 
> This is the 1-minute CPU load average, as reported by the operating system.
> 
> Thanks
> -Mark
> 
> 
>> On Jan 9, 2018, at 11:18 AM, Karthik Kothareddy (karthikk) [CONT - Type 2] 
>>  wrote:
>> 
>> Hello All,
>> 
>> I was going through SystemDiagnostics json for populating custom metrics for 
>> our NiFi instances and came across the fields  availableProcessors and 
>> processorLoadAverage in the Json. From the documentation I understood that 
>> it is the underlying hardware's processor (number of cores) count. But for 
>> the processorLoadAverage, I'm not yet clear on how this metric is calculated 
>> by NiFi and what factors it considers in doing so. Can anyone please explain 
>> me how this is measured and is it the average for just one core (one 
>> processor) or is it aggregation on all available cores?
>> 
>> 
>> -Karthik
> 

Re: [DRAFT][REPORT] Apache NiFi Board Report Jan 2018

[Andrew Lim]

+1 LGTM

-Drew

> On Jan 9, 2018, at 2:15 AM, Joe Witt  wrote:
> 
> Team,
> 
> It is that time again to send in our board report.  I'll submit it
> tomorrow but would appreciate if you can take a quick look and point
> out things to tweak or add or remove.  Really proud of what we're
> accomplishing!
> 
> Thanks
> Joe
> 
> -=-=-=-=-=-=-
> 
> ## Description:
> - Apache NiFi is an easy to use, powerful, and reliable system to
>   process and distribute data.
> - Apache NiFi MiNiFi is an edge data collection agent built to seamlessly
>   integrate with and leverage the command and control of NiFi. There are
>   both Java and C++ implementations.
> - Apache NiFi Registry is a centralized registry for key configuration items
>   including flow versions, assets, and extensions for Apache NiFi
>   and Apache MiNiFi.
> - Apache NiFi Nar Maven Plugin is a release artifact used for supporting
>   the NiFi classloader isolation model.
> 
> ## Issues:
> - There are no issues requiring board attention at this time.
> 
> ## Activity:
> - Conducted several releases including Apache NiFi Registry,
>   Apache NiFi MiNiFi CPP and Apache NiFi MiNiFi Java. The Apache NiFi Registry
>   release is a very important step forward for the community as it provides a
>   powerful software development lifecycle mechanism for Apache NiFi users and
>   answers a long standing request and often discussed capability.
> - Apache NiFi 1.5.0 Release Candidate under vote at the time of this board
>   report writing.
> - Voted a significant number of new committers and PMC members and are
>   extremely fortunate to see strong committer pipeline continue through newer
>   contributors with significant contributions.
> 
> ## Health report:
> - Health of the community remains strong. Mailing list and JIRA activity is
>   consistent. ASF Hipchat is serving as an on-ramp for new users to our
>   mailing list and JIRA systems. We continue to see new users and
>   contributors.
> - As we called out in our previous report we anticipated significant
>   recognition of earned meritocracy within the committer and PMC ranks.  That
>   held true and the pipeline remains encouraging.
> 
> ## PMC changes:
> 
> - Currently 26 PMC members.
> - New PMC members:
>- Marc Parisi was added to the PMC on Wed Dec 13 2017
>- Jeff Storck was added to the PMC on Mon Dec 04 2017
>- Scott Aslan was added to the PMC on Fri Dec 01 2017
>- Michael W Moser was added to the PMC on Sun Nov 19 2017
> - Last PMC member added Wed Dec 13 2017.
> 
> ## Committer base changes:
> 
> - Currently 38 committers.
> - New commmitters:
>- Kevin Doran was added as a committer on Wed Jan 03 2018
>- Andrew Ian Christianson was added as a committer on Mon Nov 13 2017
>- Mike Hogue was added as a committer on Thu Nov 09 2017
>- Peter Wicks was added as a committer on Thu Nov 09 2017
> - Last committer added Wed Jan 03 2018.
> 
> ## Releases:
> 
> - Apache NiFi Registry 0.1.0 was released Jan 1 2018.
> - Apache NiFi MiNiFi Java 0.3.0 was released Dec 22 2017.
> - Apache NiFi MiNiFi CPP 0.3.0 was released Nov 30 2017.
> 
> ## Mailing list activity:
> 
> - Activity on the mailing lists remains high with a mixture
>   of new users, contributors, and deeper more experienced users and
>   contributors sparking discussion and questions and filing bugs or
>   new features.
> 
> - us...@nifi.apache.org:
>- 581 subscribers (up 25 in the last 3 months):
>- 783 emails sent to list (721 in previous quarter)
> 
> - dev@nifi.apache.org:
>- 388 subscribers (up 5 in the last 3 months):
>- 679 emails sent to list (590 in previous quarter)
> 
> - iss...@nifi.apache.org:
>- 46 subscribers (up 0 in the last 3 months):
>- 6950 emails sent to list (5388 in previous quarter)
> 
> 
> ## JIRA activity:
> 
> - 356 JIRA tickets created in the last 3 months
> - 276 JIRA tickets closed/resolved in the last 3 months

Cross NAR Controller Service access with gradle

[Nicholas Deane]

Hi,

I’m working on a custom processor and want to utilize NiFi’s SSLContextService.

In my build.gradle dependencies section, I’ve included:
  compile "org.apache.nifi:nifi-ssl-context-service-api:$nifiVersion"

and in my code I’ve created the property descriptor with 
.identifiesControllerService(SSLContextService.class)

When I’m configuring the processor, I noticed that the property on the 
processor says that it “Requires Controller Service: SSLContextService from 
my-custom-nar”. When I create a StandardSSLContextService from the default 
org.apache.nifi bundle, it doesn’t show up in the processor’s list (as expected 
based on that info message).

I tried bringing in “nifi-ssl-context-service” and now I have two versions of 
StandardSSLContextService in the controller service list (one from the 
org.apache.nifi bundle and the other from my-custom-nar bundle). Is this how I 
should be doing something like this or is there a better way for me to use 
Controller Services across bundles and avoid having duplicates in that list?


Notice - Confidential Information The information in this communication and any 
attachments is strictly confidential and intended only for the use of the 
individual(s) or entity(ies) named above. If you are not the intended 
recipient, any dissemination, distribution, copying or other use of the 
information contained in this communication and/or any attachment is strictly 
prohibited. If you have received this communication in error, please first 
notify the sender immediately and then delete this communication from all data 
storage devices and destroy all hard copies.

RE: [EXT] Re: processorLoadAverage In SystemDiagnostics

[Karthik Kothareddy (karthikk) [CONT - Type 2]]

Mark,

Thanks for quick response, Is there a way that NiFi provides these metrics 
individually for a processor that the average load ? Or can we use any REST 
endpoint to grab the processor-wise metrics?

-Karthik

-Original Message-
From: Mark Payne [mailto:marka...@hotmail.com] 
Sent: Tuesday, January 09, 2018 9:22 AM
To: dev@nifi.apache.org
Subject: [EXT] Re: processorLoadAverage In SystemDiagnostics 

Hi Karthik,

This is the 1-minute CPU load average, as reported by the operating system.

Thanks
-Mark


> On Jan 9, 2018, at 11:18 AM, Karthik Kothareddy (karthikk) [CONT - Type 2] 
>  wrote:
> 
> Hello All,
> 
> I was going through SystemDiagnostics json for populating custom metrics for 
> our NiFi instances and came across the fields  availableProcessors and 
> processorLoadAverage in the Json. From the documentation I understood that it 
> is the underlying hardware's processor (number of cores) count. But for the 
> processorLoadAverage, I'm not yet clear on how this metric is calculated by 
> NiFi and what factors it considers in doing so. Can anyone please explain me 
> how this is measured and is it the average for just one core (one processor) 
> or is it aggregation on all available cores?
> 
> 
> -Karthik

Re: PutMongo 1.3.0 - Upsert issue

[fabe_bdx]

Thanks for the repply, but i don't have any knowledge to complile NAR object
:/

is it possible to extract them of 1.5.0 and push it into nifi 1.3.0 directly
?

Thx,



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: processorLoadAverage In SystemDiagnostics

[Mark Payne]

Hi Karthik,

This is the 1-minute CPU load average, as reported by the operating system.

Thanks
-Mark


> On Jan 9, 2018, at 11:18 AM, Karthik Kothareddy (karthikk) [CONT - Type 2] 
>  wrote:
> 
> Hello All,
> 
> I was going through SystemDiagnostics json for populating custom metrics for 
> our NiFi instances and came across the fields  availableProcessors and 
> processorLoadAverage in the Json. From the documentation I understood that it 
> is the underlying hardware's processor (number of cores) count. But for the 
> processorLoadAverage, I'm not yet clear on how this metric is calculated by 
> NiFi and what factors it considers in doing so. Can anyone please explain me 
> how this is measured and is it the average for just one core (one processor) 
> or is it aggregation on all available cores?
> 
> 
> -Karthik

processorLoadAverage In SystemDiagnostics

[Karthik Kothareddy (karthikk) [CONT - Type 2]]

Hello All,

I was going through SystemDiagnostics json for populating custom metrics for 
our NiFi instances and came across the fields  availableProcessors and 
processorLoadAverage in the Json. From the documentation I understood that it 
is the underlying hardware's processor (number of cores) count. But for the 
processorLoadAverage, I'm not yet clear on how this metric is calculated by 
NiFi and what factors it considers in doing so. Can anyone please explain me 
how this is measured and is it the average for just one core (one processor) or 
is it aggregation on all available cores?


-Karthik

Re: PutMongo 1.3.0 - Upsert issue

[Pierre Villard]

Hi,

Easiest solution is probably to build the Mongo NAR with 1.5.0 code, and
drop it into your NiFi 1.3. This way you'll have the two versions of the
processor available in NiFi and you can easily switch from one version of
the processor to another.

https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#component-versioning

Thanks
Pierre

2018-01-09 16:00 GMT+01:00 fabe_bdx :

> Hi All,
>
> We actually used nifi 1.3.0, with a PutMongo 1.3.0 processor, we have an
> issue with the upsert capabilities of this processor.
>
> During the Upsert if an attribute of the document is not present in the
> request from putmongo, the processor suppress it the mongo document
> collection.
>
> We don"t find any solution to this problem for 1.3.0 version, but that seem
> to be resolved in the 1.5.0.
>
> Is it possible to apply a patch to 1.3.0 version to solve this problem ?
>
> Many thanks for your help.
>
> Fabien.
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>

Re: [DRAFT][REPORT] Apache NiFi Board Report Jan 2018

[Pierre Villard]

LGTM as well, thanks Joe!

2018-01-09 15:39 GMT+01:00 Matt Burgess :

> LGTM, thanks!
>
> On Tue, Jan 9, 2018 at 2:15 AM, Joe Witt  wrote:
> > Team,
> >
> > It is that time again to send in our board report.  I'll submit it
> > tomorrow but would appreciate if you can take a quick look and point
> > out things to tweak or add or remove.  Really proud of what we're
> > accomplishing!
> >
> > Thanks
> > Joe
> >
> > -=-=-=-=-=-=-
> >
> > ## Description:
> >  - Apache NiFi is an easy to use, powerful, and reliable system to
> >process and distribute data.
> >  - Apache NiFi MiNiFi is an edge data collection agent built to
> seamlessly
> >integrate with and leverage the command and control of NiFi. There are
> >both Java and C++ implementations.
> >  - Apache NiFi Registry is a centralized registry for key configuration
> items
> >including flow versions, assets, and extensions for Apache NiFi
> >and Apache MiNiFi.
> >  - Apache NiFi Nar Maven Plugin is a release artifact used for supporting
> >the NiFi classloader isolation model.
> >
> > ## Issues:
> >  - There are no issues requiring board attention at this time.
> >
> > ## Activity:
> >  - Conducted several releases including Apache NiFi Registry,
> >Apache NiFi MiNiFi CPP and Apache NiFi MiNiFi Java. The Apache NiFi
> Registry
> >release is a very important step forward for the community as it
> provides a
> >powerful software development lifecycle mechanism for Apache NiFi
> users and
> >answers a long standing request and often discussed capability.
> >  - Apache NiFi 1.5.0 Release Candidate under vote at the time of this
> board
> >report writing.
> >  - Voted a significant number of new committers and PMC members and are
> >extremely fortunate to see strong committer pipeline continue through
> newer
> >contributors with significant contributions.
> >
> > ## Health report:
> >  - Health of the community remains strong. Mailing list and JIRA
> activity is
> >consistent. ASF Hipchat is serving as an on-ramp for new users to our
> >mailing list and JIRA systems. We continue to see new users and
> >contributors.
> >  - As we called out in our previous report we anticipated significant
> >recognition of earned meritocracy within the committer and PMC
> ranks.  That
> >held true and the pipeline remains encouraging.
> >
> > ## PMC changes:
> >
> >  - Currently 26 PMC members.
> >  - New PMC members:
> > - Marc Parisi was added to the PMC on Wed Dec 13 2017
> > - Jeff Storck was added to the PMC on Mon Dec 04 2017
> > - Scott Aslan was added to the PMC on Fri Dec 01 2017
> > - Michael W Moser was added to the PMC on Sun Nov 19 2017
> >  - Last PMC member added Wed Dec 13 2017.
> >
> > ## Committer base changes:
> >
> >  - Currently 38 committers.
> >  - New commmitters:
> > - Kevin Doran was added as a committer on Wed Jan 03 2018
> > - Andrew Ian Christianson was added as a committer on Mon Nov 13 2017
> > - Mike Hogue was added as a committer on Thu Nov 09 2017
> > - Peter Wicks was added as a committer on Thu Nov 09 2017
> >  - Last committer added Wed Jan 03 2018.
> >
> > ## Releases:
> >
> >  - Apache NiFi Registry 0.1.0 was released Jan 1 2018.
> >  - Apache NiFi MiNiFi Java 0.3.0 was released Dec 22 2017.
> >  - Apache NiFi MiNiFi CPP 0.3.0 was released Nov 30 2017.
> >
> > ## Mailing list activity:
> >
> >  - Activity on the mailing lists remains high with a mixture
> >of new users, contributors, and deeper more experienced users and
> >contributors sparking discussion and questions and filing bugs or
> >new features.
> >
> >  - us...@nifi.apache.org:
> > - 581 subscribers (up 25 in the last 3 months):
> > - 783 emails sent to list (721 in previous quarter)
> >
> >  - dev@nifi.apache.org:
> > - 388 subscribers (up 5 in the last 3 months):
> > - 679 emails sent to list (590 in previous quarter)
> >
> >  - iss...@nifi.apache.org:
> > - 46 subscribers (up 0 in the last 3 months):
> > - 6950 emails sent to list (5388 in previous quarter)
> >
> >
> > ## JIRA activity:
> >
> >  - 356 JIRA tickets created in the last 3 months
> >  - 276 JIRA tickets closed/resolved in the last 3 months
>

PutMongo 1.3.0 - Upsert issue

[fabe_bdx]

Hi All,

We actually used nifi 1.3.0, with a PutMongo 1.3.0 processor, we have an
issue with the upsert capabilities of this processor. 

During the Upsert if an attribute of the document is not present in the
request from putmongo, the processor suppress it the mongo document
collection.

We don"t find any solution to this problem for 1.3.0 version, but that seem
to be resolved in the 1.5.0.

Is it possible to apply a patch to 1.3.0 version to solve this problem ?

Many thanks for your help.

Fabien.



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: [DRAFT][REPORT] Apache NiFi Board Report Jan 2018

[Matt Burgess]

LGTM, thanks!

On Tue, Jan 9, 2018 at 2:15 AM, Joe Witt  wrote:
> Team,
>
> It is that time again to send in our board report.  I'll submit it
> tomorrow but would appreciate if you can take a quick look and point
> out things to tweak or add or remove.  Really proud of what we're
> accomplishing!
>
> Thanks
> Joe
>
> -=-=-=-=-=-=-
>
> ## Description:
>  - Apache NiFi is an easy to use, powerful, and reliable system to
>process and distribute data.
>  - Apache NiFi MiNiFi is an edge data collection agent built to seamlessly
>integrate with and leverage the command and control of NiFi. There are
>both Java and C++ implementations.
>  - Apache NiFi Registry is a centralized registry for key configuration items
>including flow versions, assets, and extensions for Apache NiFi
>and Apache MiNiFi.
>  - Apache NiFi Nar Maven Plugin is a release artifact used for supporting
>the NiFi classloader isolation model.
>
> ## Issues:
>  - There are no issues requiring board attention at this time.
>
> ## Activity:
>  - Conducted several releases including Apache NiFi Registry,
>Apache NiFi MiNiFi CPP and Apache NiFi MiNiFi Java. The Apache NiFi 
> Registry
>release is a very important step forward for the community as it provides a
>powerful software development lifecycle mechanism for Apache NiFi users and
>answers a long standing request and often discussed capability.
>  - Apache NiFi 1.5.0 Release Candidate under vote at the time of this board
>report writing.
>  - Voted a significant number of new committers and PMC members and are
>extremely fortunate to see strong committer pipeline continue through newer
>contributors with significant contributions.
>
> ## Health report:
>  - Health of the community remains strong. Mailing list and JIRA activity is
>consistent. ASF Hipchat is serving as an on-ramp for new users to our
>mailing list and JIRA systems. We continue to see new users and
>contributors.
>  - As we called out in our previous report we anticipated significant
>recognition of earned meritocracy within the committer and PMC ranks.  That
>held true and the pipeline remains encouraging.
>
> ## PMC changes:
>
>  - Currently 26 PMC members.
>  - New PMC members:
> - Marc Parisi was added to the PMC on Wed Dec 13 2017
> - Jeff Storck was added to the PMC on Mon Dec 04 2017
> - Scott Aslan was added to the PMC on Fri Dec 01 2017
> - Michael W Moser was added to the PMC on Sun Nov 19 2017
>  - Last PMC member added Wed Dec 13 2017.
>
> ## Committer base changes:
>
>  - Currently 38 committers.
>  - New commmitters:
> - Kevin Doran was added as a committer on Wed Jan 03 2018
> - Andrew Ian Christianson was added as a committer on Mon Nov 13 2017
> - Mike Hogue was added as a committer on Thu Nov 09 2017
> - Peter Wicks was added as a committer on Thu Nov 09 2017
>  - Last committer added Wed Jan 03 2018.
>
> ## Releases:
>
>  - Apache NiFi Registry 0.1.0 was released Jan 1 2018.
>  - Apache NiFi MiNiFi Java 0.3.0 was released Dec 22 2017.
>  - Apache NiFi MiNiFi CPP 0.3.0 was released Nov 30 2017.
>
> ## Mailing list activity:
>
>  - Activity on the mailing lists remains high with a mixture
>of new users, contributors, and deeper more experienced users and
>contributors sparking discussion and questions and filing bugs or
>new features.
>
>  - us...@nifi.apache.org:
> - 581 subscribers (up 25 in the last 3 months):
> - 783 emails sent to list (721 in previous quarter)
>
>  - dev@nifi.apache.org:
> - 388 subscribers (up 5 in the last 3 months):
> - 679 emails sent to list (590 in previous quarter)
>
>  - iss...@nifi.apache.org:
> - 46 subscribers (up 0 in the last 3 months):
> - 6950 emails sent to list (5388 in previous quarter)
>
>
> ## JIRA activity:
>
>  - 356 JIRA tickets created in the last 3 months
>  - 276 JIRA tickets closed/resolved in the last 3 months

Re: [DRAFT][REPORT] Apache NiFi Board Report Jan 2018

[Aldrin Piri]

Looks good.  Thanks for compiling, Joe.

On Tue, Jan 9, 2018 at 2:15 AM, Joe Witt  wrote:

> Team,
>
> It is that time again to send in our board report.  I'll submit it
> tomorrow but would appreciate if you can take a quick look and point
> out things to tweak or add or remove.  Really proud of what we're
> accomplishing!
>
> Thanks
> Joe
>
> -=-=-=-=-=-=-
>
> ## Description:
>  - Apache NiFi is an easy to use, powerful, and reliable system to
>process and distribute data.
>  - Apache NiFi MiNiFi is an edge data collection agent built to seamlessly
>integrate with and leverage the command and control of NiFi. There are
>both Java and C++ implementations.
>  - Apache NiFi Registry is a centralized registry for key configuration
> items
>including flow versions, assets, and extensions for Apache NiFi
>and Apache MiNiFi.
>  - Apache NiFi Nar Maven Plugin is a release artifact used for supporting
>the NiFi classloader isolation model.
>
> ## Issues:
>  - There are no issues requiring board attention at this time.
>
> ## Activity:
>  - Conducted several releases including Apache NiFi Registry,
>Apache NiFi MiNiFi CPP and Apache NiFi MiNiFi Java. The Apache NiFi
> Registry
>release is a very important step forward for the community as it
> provides a
>powerful software development lifecycle mechanism for Apache NiFi users
> and
>answers a long standing request and often discussed capability.
>  - Apache NiFi 1.5.0 Release Candidate under vote at the time of this board
>report writing.
>  - Voted a significant number of new committers and PMC members and are
>extremely fortunate to see strong committer pipeline continue through
> newer
>contributors with significant contributions.
>
> ## Health report:
>  - Health of the community remains strong. Mailing list and JIRA activity
> is
>consistent. ASF Hipchat is serving as an on-ramp for new users to our
>mailing list and JIRA systems. We continue to see new users and
>contributors.
>  - As we called out in our previous report we anticipated significant
>recognition of earned meritocracy within the committer and PMC ranks.
> That
>held true and the pipeline remains encouraging.
>
> ## PMC changes:
>
>  - Currently 26 PMC members.
>  - New PMC members:
> - Marc Parisi was added to the PMC on Wed Dec 13 2017
> - Jeff Storck was added to the PMC on Mon Dec 04 2017
> - Scott Aslan was added to the PMC on Fri Dec 01 2017
> - Michael W Moser was added to the PMC on Sun Nov 19 2017
>  - Last PMC member added Wed Dec 13 2017.
>
> ## Committer base changes:
>
>  - Currently 38 committers.
>  - New commmitters:
> - Kevin Doran was added as a committer on Wed Jan 03 2018
> - Andrew Ian Christianson was added as a committer on Mon Nov 13 2017
> - Mike Hogue was added as a committer on Thu Nov 09 2017
> - Peter Wicks was added as a committer on Thu Nov 09 2017
>  - Last committer added Wed Jan 03 2018.
>
> ## Releases:
>
>  - Apache NiFi Registry 0.1.0 was released Jan 1 2018.
>  - Apache NiFi MiNiFi Java 0.3.0 was released Dec 22 2017.
>  - Apache NiFi MiNiFi CPP 0.3.0 was released Nov 30 2017.
>
> ## Mailing list activity:
>
>  - Activity on the mailing lists remains high with a mixture
>of new users, contributors, and deeper more experienced users and
>contributors sparking discussion and questions and filing bugs or
>new features.
>
>  - us...@nifi.apache.org:
> - 581 subscribers (up 25 in the last 3 months):
> - 783 emails sent to list (721 in previous quarter)
>
>  - dev@nifi.apache.org:
> - 388 subscribers (up 5 in the last 3 months):
> - 679 emails sent to list (590 in previous quarter)
>
>  - iss...@nifi.apache.org:
> - 46 subscribers (up 0 in the last 3 months):
> - 6950 emails sent to list (5388 in previous quarter)
>
>
> ## JIRA activity:
>
>  - 356 JIRA tickets created in the last 3 months
>  - 276 JIRA tickets closed/resolved in the last 3 months
>

Re: Enrichment plugin for adding attributes from SQL

[Brett Ryan]

Have now implemented SQLLookupService which is a LookupService 
implementation.

This works as advertised to provide a service based method of assigning 
attributes to a flow file based on a Lookup Service.
https://github.com/brettryan/nifi-drunken-bundle/blob/dev/nifi-drunken-services/src/main/java/com/drunkendev/nifi/services/SQLLookupService.java
 


The matching LookupAttributeFromRecord
https://github.com/brettryan/nifi-drunken-bundle/blob/dev/nifi-drunken-processors/src/main/java/com/drunkendev/nifi/processors/LookupAttributeFromRecord.java
 


If y'all think this could be worth contributing I'll invest further time in 
neatening these up and provide full test cases, atm these are just a PoC that's 
been proven.



> On 9 Jan 2018, at 18:50, Brett Ryan  wrote:
> 
> 
>> On 7 Jan 2018, at 22:23, Brett Ryan > > wrote:
>> 
>> (1) Write a processor that updates an attribute on a FF from a Record based 
>> LookupService. Say I call it UpdateAttributeFromRecordLookup
> 
> Ok, I've implemented this now as a PoC and have tested it with the 
> MongoDBLookupService which is working great.
> 
> The processor takes a record based lookup service and any number of 
> attributes. The processor has a required attribute that specifies a key 
> prefix (default = "key."), this prefix identifies further attributes as a key 
> lookup, any other attributes are considered names to include as 
> attribute/field pairs for the resulting FF, if none of these non-key custom 
> attributes exist then all fields from the Record are added.
> 
> if we take the (LookupRecord with MongoDB) as an example
> 
> Key Prefix = ".key"
> key.id_store = ${id_store}
> 
> We would end up with the following attributes added to the resulting FF
> 
> address = 177 Boulevard Haussmann, 75008 Paris
> address_city = Paris
> capacity = 464600
> id_store = 1
> manager = Jean Ricca
> Given all this, do you think I'm now on the right track for a more general 
> purpose processor that could be used for multiple lookups?
> 
> I could then implement the SQL Lookup Service to feed this.
> 
>   [1]: 
> https://community.hortonworks.com/articles/146198/data-flow-enrichment-with-nifi-part-3-lookuprecord.html
>  
> 
>> (2) Write a service SQLLookupService which given a query as a lookup 
>> coordinate would return a Record for the first found entry.
>> (3) Write a processor that updates an attribute from a record lookup, call 
>> it UpdateAttributeFromRecordLookup.
>> 
>> It's a bit more long winded to achieve what I originally had with a single 
>> processor, but I guess UpdateAttributeFromRecordLookup would have the 
>> benefit of updating attributes from LookupService implementations that do 
>> not meet the LookupAttribute criteria.
>> 
>> If you think this is viable, please let me know and I'll have a look at 
>> starting it tomorrow afternoon.
> 



signature.asc
Description: Message signed with OpenPGP

Re: Apache NiFi 1.5.0 (RC1) Release Helper Guide

[Aldrin Piri]

Looks like there was an extra nifi directory created.  I moved them in the
repo to reflect the paths in the helper email.


On Tue, Jan 9, 2018 at 7:52 AM, Jeff Zemerick  wrote:

> Hi Joe,
>
> I'm getting a 404 on those download links.
>
> Thanks
>
>
> On Tue, Jan 9, 2018 at 5:19 AM, Joe Witt  wrote:
>
> > Hello Apache NiFi community,
> >
> > Please find the associated guidance to help those interested in
> > validating/verifying the release so they can vote.
> >
> > # Download latest KEYS file:
> > https://dist.apache.org/repos/dist/dev/nifi/KEYS
> >
> > # Import keys file:
> > gpg --import KEYS
> >
> > # [optional] Clear out local maven artifact repository
> >
> > # Pull down nifi-1.5.0 source release artifacts for review:
> >
> > wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-
> > 1.5.0-source-release.zip
> > wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-
> > 1.5.0-source-release.zip.asc
> > wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-
> > 1.5.0-source-release.zip.md5
> > wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-
> > 1.5.0-source-release.zip.sha1
> > wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-
> > 1.5.0-source-release.zip.sha256
> >
> > # Verify the signature
> > gpg --verify nifi-1.5.0-source-release.zip.asc
> >
> > # Verify the hashes (md5, sha1, sha256) match the source and what was
> > provided in the vote email thread
> > md5sum nifi-1.5.0-source-release.zip
> > sha1sum nifi-1.5.0-source-release.zip
> > sha256sum nifi-1.5.0-source-release.zip
> >
> > # Unzip nifi-1.5.0-source-release.zip
> >
> > # Verify the build works including release audit tool (RAT) checks
> > cd nifi-1.5.0
> > mvn clean install -Pcontrib-check,include-grpc
> >
> > # Verify the contents contain a good README, NOTICE, and LICENSE.
> >
> > # Verify the git commit ID is correct
> >
> > # Verify the RC was branched off the correct git commit ID
> >
> > # Look at the resulting convenience binary as found in
> nifi-assembly/target
> >
> > # Make sure the README, NOTICE, and LICENSE are present and correct
> >
> > # Run the resulting convenience binary and make sure it works as expected
> >
> > # Send a response to the vote thread indicating a +1, 0, -1 based on
> > your findings.
> >
> > Please note that the convenience binaries and artifacts are provided
> > except for the tar.gz/zip of the actual nifi build.
> > It is simply too large to upload at this time.  But once
> > https://issues.apache.org/jira/browse/INFRA-15816 is
> > addressed I will upload it.
> >
> > Thank you for your time and effort to validate the release!
> >
>

Re: Apache NiFi 1.5.0 (RC1) Release Helper Guide

[Jeff Zemerick]

Hi Joe,

I'm getting a 404 on those download links.

Thanks


On Tue, Jan 9, 2018 at 5:19 AM, Joe Witt  wrote:

> Hello Apache NiFi community,
>
> Please find the associated guidance to help those interested in
> validating/verifying the release so they can vote.
>
> # Download latest KEYS file:
> https://dist.apache.org/repos/dist/dev/nifi/KEYS
>
> # Import keys file:
> gpg --import KEYS
>
> # [optional] Clear out local maven artifact repository
>
> # Pull down nifi-1.5.0 source release artifacts for review:
>
> wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-
> 1.5.0-source-release.zip
> wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-
> 1.5.0-source-release.zip.asc
> wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-
> 1.5.0-source-release.zip.md5
> wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-
> 1.5.0-source-release.zip.sha1
> wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-
> 1.5.0-source-release.zip.sha256
>
> # Verify the signature
> gpg --verify nifi-1.5.0-source-release.zip.asc
>
> # Verify the hashes (md5, sha1, sha256) match the source and what was
> provided in the vote email thread
> md5sum nifi-1.5.0-source-release.zip
> sha1sum nifi-1.5.0-source-release.zip
> sha256sum nifi-1.5.0-source-release.zip
>
> # Unzip nifi-1.5.0-source-release.zip
>
> # Verify the build works including release audit tool (RAT) checks
> cd nifi-1.5.0
> mvn clean install -Pcontrib-check,include-grpc
>
> # Verify the contents contain a good README, NOTICE, and LICENSE.
>
> # Verify the git commit ID is correct
>
> # Verify the RC was branched off the correct git commit ID
>
> # Look at the resulting convenience binary as found in nifi-assembly/target
>
> # Make sure the README, NOTICE, and LICENSE are present and correct
>
> # Run the resulting convenience binary and make sure it works as expected
>
> # Send a response to the vote thread indicating a +1, 0, -1 based on
> your findings.
>
> Please note that the convenience binaries and artifacts are provided
> except for the tar.gz/zip of the actual nifi build.
> It is simply too large to upload at this time.  But once
> https://issues.apache.org/jira/browse/INFRA-15816 is
> addressed I will upload it.
>
> Thank you for your time and effort to validate the release!
>

Apache NiFi 1.5.0 (RC1) Release Helper Guide

[Joe Witt]

Hello Apache NiFi community,

Please find the associated guidance to help those interested in
validating/verifying the release so they can vote.

# Download latest KEYS file:
https://dist.apache.org/repos/dist/dev/nifi/KEYS

# Import keys file:
gpg --import KEYS

# [optional] Clear out local maven artifact repository

# Pull down nifi-1.5.0 source release artifacts for review:

wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-1.5.0-source-release.zip
wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-1.5.0-source-release.zip.asc
wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-1.5.0-source-release.zip.md5
wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-1.5.0-source-release.zip.sha1
wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.5.0/nifi-1.5.0-source-release.zip.sha256

# Verify the signature
gpg --verify nifi-1.5.0-source-release.zip.asc

# Verify the hashes (md5, sha1, sha256) match the source and what was
provided in the vote email thread
md5sum nifi-1.5.0-source-release.zip
sha1sum nifi-1.5.0-source-release.zip
sha256sum nifi-1.5.0-source-release.zip

# Unzip nifi-1.5.0-source-release.zip

# Verify the build works including release audit tool (RAT) checks
cd nifi-1.5.0
mvn clean install -Pcontrib-check,include-grpc

# Verify the contents contain a good README, NOTICE, and LICENSE.

# Verify the git commit ID is correct

# Verify the RC was branched off the correct git commit ID

# Look at the resulting convenience binary as found in nifi-assembly/target

# Make sure the README, NOTICE, and LICENSE are present and correct

# Run the resulting convenience binary and make sure it works as expected

# Send a response to the vote thread indicating a +1, 0, -1 based on
your findings.

Please note that the convenience binaries and artifacts are provided
except for the tar.gz/zip of the actual nifi build.
It is simply too large to upload at this time.  But once
https://issues.apache.org/jira/browse/INFRA-15816 is
addressed I will upload it.

Thank you for your time and effort to validate the release!

[VOTE] Release Apache NiFi 1.5.0 (RC1)

[Joe Witt]

Hello,

I am pleased to be calling this vote for the source release of Apache
NiFi nifi-1.5.0.

The source zip, including signatures, digests, etc. can be found at:
https://repository.apache.org/content/repositories/orgapachenifi-1116

The Git tag is nifi-1.5.0-RC1
The Git commit ID is 46d30c7e92f0ad034d9b35bf1d05c350ab5547ed
https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=46d30c7e92f0ad034d9b35bf1d05c350ab5547ed

Checksums of nifi-1.5.0-source-release.zip:
MD5: 046f2dde4af592dd8c05e55c2bbb3c4f
SHA1: 63b9a68b9f89200fd31f5561956a15b45b1b9c8c
SHA256: 40b155c4911414907835f2eb0d5a4da798935f27f1e5134218d904fe6c942d13

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/joewitt.asc

KEYS file available here:
https://dist.apache.org/repos/dist/release/nifi/KEYS

195 issues were closed/resolved for this release:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12341668

Release note highlights can be found here:
https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.5.0

The vote will be open for 72 hours.
Please download the release candidate and evaluate the necessary items
including checking hashes, signatures, build
from source, and test.  The please vote:

[ ] +1 Release this package as nifi-1.5.0
[ ] +0 no opinion
[ ] -1 Do not release this package because...

Re: Enrichment plugin for adding attributes from SQL

[Brett Ryan]

> On 7 Jan 2018, at 22:23, Brett Ryan  wrote:
> 
> (1) Write a processor that updates an attribute on a FF from a Record based 
> LookupService. Say I call it UpdateAttributeFromRecordLookup

Ok, I've implemented this now as a PoC and have tested it with the 
MongoDBLookupService which is working great.

The processor takes a record based lookup service and any number of attributes. 
The processor has a required attribute that specifies a key prefix (default = 
"key."), this prefix identifies further attributes as a key lookup, any other 
attributes are considered names to include as attribute/field pairs for the 
resulting FF, if none of these non-key custom attributes exist then all fields 
from the Record are added.

if we take the (LookupRecord with MongoDB) as an example

Key Prefix = ".key"
key.id_store = ${id_store}

We would end up with the following attributes added to the resulting FF

address = 177 Boulevard Haussmann, 75008 Paris
address_city = Paris
capacity = 464600
id_store = 1
manager = Jean Ricca
Given all this, do you think I'm now on the right track for a more general 
purpose processor that could be used for multiple lookups?

I could then implement the SQL Lookup Service to feed this.

  [1]: 
https://community.hortonworks.com/articles/146198/data-flow-enrichment-with-nifi-part-3-lookuprecord.html

> (2) Write a service SQLLookupService which given a query as a lookup 
> coordinate would return a Record for the first found entry.
> (3) Write a processor that updates an attribute from a record lookup, call it 
> UpdateAttributeFromRecordLookup.
> 
> It's a bit more long winded to achieve what I originally had with a single 
> processor, but I guess UpdateAttributeFromRecordLookup would have the benefit 
> of updating attributes from LookupService implementations that do not meet 
> the LookupAttribute criteria.
> 
> If you think this is viable, please let me know and I'll have a look at 
> starting it tomorrow afternoon.



signature.asc
Description: Message signed with OpenPGP