Apache NiFi 0.7.2 RC1 Release Helper Guide

[Andy LoPresto]

Hello Apache NiFi community,

Please find the associated guidance to help those interested in 
validating/verifying the release so they can vote.

# Download latest KEYS file:
https://dist.apache.org/repos/dist/dev/nifi/KEYS

# Import keys file:
gpg --import KEYS

# [optional] Clear out local maven artifact repository

# Pull down nifi-0.7.2 source release artifacts for review:

wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.7.2/nifi-0.7.2-source-release.zip
wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.7.2/nifi-0.7.2-source-release.zip.asc
wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.7.2/nifi-0.7.2-source-release.zip.md5
wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.7.2/nifi-0.7.2-source-release.zip.sha1
wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-0.7.2/nifi-0.7.2-source-release.zip.sha256

# Verify the signature
gpg --verify -v nifi-0.7.2-source-release.zip.asc

# Verify the hashes (md5, sha1, sha256) match the source and what was provided 
in the vote email thread
md5sum nifi-0.7.2-source-release.zip
sha1sum nifi-0.7.2-source-release.zip
sha256sum nifi-0.7.2-source-release.zip

# Unzip nifi-0.7.2-source-release.zip

# Verify the build works including release audit tool (RAT) checks
cd nifi-0.7.2
mvn clean install -Pcontrib-check

# Verify the contents contain a good README, NOTICE, and LICENSE.

# Verify the git commit ID is correct

# Verify the RC was branched off the correct git commit ID

# Look at the resulting convenience binary as found in nifi-assembly/target

# Make sure the README, NOTICE, and LICENSE are present and correct

# Run the resulting convenience binary and make sure it works as expected

# Send a response to the vote thread indicating a +1, 0, -1 based on your 
findings.

Thank you for your time and effort to validate the release!

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69



signature.asc
Description: Message signed with OpenPGP using GPGMail

[VOTE] Release Apache NiFi 0.7.2

[Andy LoPresto]

Hello,

I am pleased to be calling this vote for the source release of Apache NiFi 
nifi-0.7.2.

The source zip, including signatures, digests, etc. can be found at:
https://repository.apache.org/content/repositories/orgapachenifi-1099

The Git tag is nifi-0.7.2-RC1
The Git commit ID is 831ac6939df7d418cadb336eedb4e09fb97541a1
https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=831ac6939df7d418cadb336eedb4e09fb97541a1

Checksums of nifi-0.7.2-source-release.zip:
MD5: efe9ea1cf0698a57a6829fe3829fc136
SHA1: aee51164af34394c7017dae491b239a88b614865
SHA256: 8cd02675003fca837ea0092b6225600a4700b905e5214a751ae7ff833263193b

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/alopresto.asc

KEYS file available here:
https://dist.apache.org/repos/dist/release/nifi/KEYS

2 issues were closed/resolved for this release:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12339601&projectId=12316020

Release note highlights can be found here:
https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version0.7.2

The vote will be open for 96 hours (over holiday weekend).
Please download the release candidate and evaluate the necessary items 
including checking hashes, signatures, build
from source, and test.  The please vote:

[ ] +1 Release this package as nifi-0.7.2
[ ] +0 no opinion
[ ] -1 Do not release this package because because…

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69



signature.asc
Description: Message signed with OpenPGP using GPGMail

Apache NiFi 1.1.2 RC1 Release Helper Guide

[Andy LoPresto]

Hello Apache NiFi community,

Please find the associated guidance to help those interested in 
validating/verifying the release so they can vote.

# Download latest KEYS file:
https://dist.apache.org/repos/dist/dev/nifi/KEYS

# Import keys file:
gpg --import KEYS

# [optional] Clear out local maven artifact repository

# Pull down nifi-1.1.2 source release artifacts for review:

wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.1.2/nifi-1.1.2-source-release.zip
wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.1.2/nifi-1.1.2-source-release.zip.asc
wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.1.2/nifi-1.1.2-source-release.zip.md5
wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.1.2/nifi-1.1.2-source-release.zip.sha1
wget 
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.1.2/nifi-1.1.2-source-release.zip.sha256

# Verify the signature
gpg --verify -v nifi-1.1.2-source-release.zip.asc

# Verify the hashes (md5, sha1, sha256) match the source and what was provided 
in the vote email thread
md5sum nifi-1.1.2-source-release.zip
sha1sum nifi-1.1.2-source-release.zip
sha256sum nifi-1.1.2-source-release.zip

# Unzip nifi-1.1.2-source-release.zip

# Verify the build works including release audit tool (RAT) checks
cd nifi-1.1.2
mvn clean install -Pcontrib-check

# Verify the contents contain a good README, NOTICE, and LICENSE.

# Verify the git commit ID is correct

# Verify the RC was branched off the correct git commit ID

# Look at the resulting convenience binary as found in nifi-assembly/target

# Make sure the README, NOTICE, and LICENSE are present and correct

# Run the resulting convenience binary and make sure it works as expected

# Send a response to the vote thread indicating a +1, 0, -1 based on your 
findings.

Thank you for your time and effort to validate the release!

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69



signature.asc
Description: Message signed with OpenPGP using GPGMail

[VOTE] Release Apache NiFi 1.1.2

[Andy LoPresto]

Hello,

I am pleased to be calling this vote for the source release of Apache NiFi 
nifi-1.1.2.

The source zip, including signatures, digests, etc. can be found at:
https://repository.apache.org/content/repositories/orgapachenifi-1098

The Git tag is nifi-1.1.2-RC1
The Git commit ID is 51fad01f5daf33716b8b5379c32ee932d91c8c63
https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=51fad01f5daf33716b8b5379c32ee932d91c8c63

Checksums of nifi-1.1.2-source-release.zip:
MD5: 31d20a09955fac32d5b4147c497deede
SHA1: 9f8f2ce00397d990dfb0890f9b1ede70dca4f25e
SHA256: 0f57b5ae7f5e1d7f1289d779df39f20d70af0ffd92cb01b265beb90b309b747e

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/alopresto.asc

KEYS file available here:
https://dist.apache.org/repos/dist/release/nifi/KEYS

2 issues were closed/resolved for this release:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12339600&projectId=12316020

Release note highlights can be found here:
https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.1.2

The vote will be open for 96 hours (over holiday weekend).
Please download the release candidate and evaluate the necessary items 
including checking hashes, signatures, build
from source, and test.  The please vote:

[ ] +1 Release this package as nifi-1.1.2
[ ] +0 no opinion
[ ] -1 Do not release this package because because…

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69



signature.asc
Description: Message signed with OpenPGP using GPGMail

Issue with Idle timeout exception with HandleHTTPRequest processor in NiFi-1.1.0

[nairy991]

Hello,

I seem to be running into an issue using the NiFi version NiFi-1.1.0 where
our HandleHttpRequest processor keeps throwing the following error:

ERROR [Timer-Driven Process
Thread-20] o.a.n.p.standard.HandleHttpRequest
HandleHttpRequest[id=11e56f5b-b901-3672-9424-05fc8f846a3e]
HandleHttpRequest[id=11e56f5b-b901-3672-9424-05fc8f846a3e] failed to process
due to org.apache.nifi.processor.exception.FlowFileAccessException: Failed
to
import data from
HttpInputOverHTTP@802fcbf[c=0,s=ERROR:java.util.concurrent.TimeoutException:
Idle timeout expired: 30025/3 ms] for
StandardFlowFileRecord[uuid=3e0a3a13-0d05-44fb-af71-1fa0b2519db8,claim=,offset=0,name=1866449175849578,size=0]
due to org.apache.nifi.processor.exception.FlowFileAccessException: Unable
to
create ContentClaim due to java.io.IOException:
java.util.concurrent.TimeoutException: Idle timeout expired: 30025/3 ms;
rolling back session:
org.apache.nifi.processor.exception.FlowFileAccessException:
Failed to import data from
HttpInputOverHTTP@802fcbf[c=0,s=ERROR:java.util.concurrent.TimeoutException:
Idle timeout expired: 30025/3 ms] for
StandardFlowFileRecord[uuid=3e0a3a13-0d05-44fb-af71-1fa0b2519db8,claim=,offset=0,name=1866449175849578,size=0]
due to org.apache.nifi.processor.exception.FlowFileAccessException: Unable
to
create ContentClaim due to java.io.IOException:
java.util.concurrent.TimeoutException: Idle timeout expired: 30025/3 ms

I been looking through the forums and NiFi community and saw that a fix had
been applied in version NiFi-1.0.0 as mentioned in the following
link:https://issues.apache.org/jira/browse/NIFI-1732
and I see that the fix consisted of hardcoding the default value from 30
seconds to “async.setTimeout(Long.MAX_VALUE); // timeout is handled by
HttpContextMap”. However even though there has been a fix applied, I am
still coming across the error. Does anyone know of any other way to fix this
issue or have an idea of what could cause this error to happen?

Any help would be greatly appreciated!



--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/Issue-with-Idle-timeout-exception-with-HandleHTTPRequest-processor-in-NiFi-1-1-0-tp14769.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: Reporting tasks as processor properties

[Bryan Bende]

Russell,

ReportingTasks are a global element that operate outside the the flow,
meaning they are not related to processors. They are meant to be
scheduled on some time interval to report things like metrics or
provenance to systems outside of NiFi.

The paragraph you referenced from the developer guide was talking
about when a ReportingTask needs to use a ControllerService, there are
two ways to obtain the controller service, and one way is preferred
over another. The only example of this is the
SiteToSiteProvenanceReportingTask which uses an SSLContextService.

Thanks,

Bryan

On Thu, Feb 16, 2017 at 11:55 AM, Russell Bateman  wrote:
> Reading the docs on Reporting Tasks, it seemed to me that they said that
> hooking (my word) a reporting task up as a property to a processor is the
> preferred way [1]. I wanted to confirm this as I have yet to stumble upon a
> coded reporting task sample that does this (or, should I say, a processor
> that has a reporting task as a property to set à la controller service).
>
> I understand controller services, hooking them up, etc., and I can easily
> imagine how to consume a reporting task in similar fashion, but I wanted a)
> to confirm that I'm not crazy and b) feel warm fuzzies that this in fact
> best practice. Also, if you could refer me to a coded processor consuming a
> reporting task from its properties that would make me feel even warmer and
> fuzzier.
>
> Thanks,
>
> Russ
>
> [1] /Developing a Reporting Task/
> ,
> paragraph 2, near end.

Reporting tasks as processor properties

[Russell Bateman]

Reading the docs on Reporting Tasks, it seemed to me that they said that 
hooking (my word) a reporting task up as a property to a processor is 
the preferred way [1]. I wanted to confirm this as I have yet to stumble 
upon a coded reporting task sample that does this (or, should I say, a 
processor that has a reporting task as a property to set à la controller 
service).


I understand controller services, hooking them up, etc., and I can 
easily imagine how to consume a reporting task in similar fashion, but I 
wanted a) to confirm that I'm not crazy and b) feel warm fuzzies that 
this in fact best practice. Also, if you could refer me to a coded 
processor consuming a reporting task from its properties that would make 
me feel even warmer and fuzzier.


Thanks,

Russ

[1] /Developing a Reporting Task/ 
, 
paragraph 2, near end.

Re: Namespaces in MiNiFI-cpp

[Aldrin Piri]

+1

On Thu, Feb 16, 2017 at 9:32 AM, Jeremy Dyer  wrote:

> Marc - I responded in https://issues.apache.org/jira/browse/MINIFI-217
> directly but I think this is a good idea. The sooner we do this the less
> painful it will be. I also like the additional namespaces like *io and
> *processors that you lay out in the JIRA.
>
> On Thu, Feb 16, 2017 at 7:24 AM, Marc  wrote:
>
> > Good timezone appropriate salutation,
> >
> >I'd like to move the MiNiFi-CPP agent to a more controlled namespace,
> > namely org::apache::nifi::minifi . In an upcoming PR I'll introduce a
> > directory for I/O activities and using namespaces will help better
> isolate
> > and encapsulate the duties of the code; however, I wanted to make sure
> > everyone is okay with these namespace changes.
> >
> > First and foremost, the question would be: will this impact anyone?
> >
> >  https://issues.apache.org/jira/browse/MINIFI-217
> >
> > I wouldn't immediately make the change, to avoid impacting devs who are
> > expecting the global namespace. I put an arbitrary estimate in the ticket
> > to reflect notice being given before applying the change to the mainline.
> >
> >  Feel free to comment on the ticket with objections and/or timeline
> > requests. This is pretty low priority, but I wanted to provide sufficient
> > notice.
> >
> >  Best Regards,
> >  Marc Parisi
> >
>

Re: Namespaces in MiNiFI-cpp

[Jeremy Dyer]

Marc - I responded in https://issues.apache.org/jira/browse/MINIFI-217
directly but I think this is a good idea. The sooner we do this the less
painful it will be. I also like the additional namespaces like *io and
*processors that you lay out in the JIRA.

On Thu, Feb 16, 2017 at 7:24 AM, Marc  wrote:

> Good timezone appropriate salutation,
>
>I'd like to move the MiNiFi-CPP agent to a more controlled namespace,
> namely org::apache::nifi::minifi . In an upcoming PR I'll introduce a
> directory for I/O activities and using namespaces will help better isolate
> and encapsulate the duties of the code; however, I wanted to make sure
> everyone is okay with these namespace changes.
>
> First and foremost, the question would be: will this impact anyone?
>
>  https://issues.apache.org/jira/browse/MINIFI-217
>
> I wouldn't immediately make the change, to avoid impacting devs who are
> expecting the global namespace. I put an arbitrary estimate in the ticket
> to reflect notice being given before applying the change to the mainline.
>
>  Feel free to comment on the ticket with objections and/or timeline
> requests. This is pretty low priority, but I wanted to provide sufficient
> notice.
>
>  Best Regards,
>  Marc Parisi
>

Re: NIFI-329 - IRC Processors

[Matt Burgess]

I had looked around a while ago for IRC clients (was thinking of
contributing IRC processors as my first contribution to NiFi, but got
sidetracked), I agree that Kitteh seems like the best choice (others
like Pircbotx, JerkLib, and Martyr don't have friendly licenses).
They all seem to have one thing in common: being event-driven.  This
is not a bad thing and as you pointed out, you could implement it the
way some other processors do (by writing the flow files from the event
handler, or using a LinkedBlockingQueue or something to pass messages
between the event handler and the onTrigger method).

I too am working on a processor that requires an event handler, either
approach seems manageable, just a little different than "normal" NiFi
processors.  I say go for it!

Regards,
Matt

On Thu, Feb 16, 2017 at 9:22 AM, Andre  wrote:
> dev,
>
> I am starting to work on NIFI-329 and plan to soon introduce processors
> capable of interacting with IRC servers.
>
> At this stage I am looking to implement only PublishIRC, however, given the
> processor will use a controller service this may be later on expanded to
> handle IRC message consumption (i.e. ConsumeIRC) and perhaps bot type
> functionality by mimicking the ethos of HandleHttpRequest and
> HandleHttpResponse processors.
>
> Since this is the first time I am trying to write a controller service may
> I ask you some feedback about my first reading?
>
> Currently the "best" ASL2 IRC library seems to be KICL (Kitteh IRC Client
> library[1]) which happens to support IRCv3, SSL and a bunch of other stuff.
>
> KICL Client class seems to fit very well within a controller service
> onEnabled, but I have the impression its event driven API is not a natural
> fit to NiFi processors and will require some work similar to the one
> required to get ListenSMTP working properly.
>
> Is this reading correct? In you experience, is it manageable or should I
> look for another library?
>
>
> Any feedback will be appreciated.
>
>
> [1] -
> http://kicl.kitteh.org/en/latest/#kitteh-irc-client-library-documentation

NIFI-329 - IRC Processors

[Andre]

dev,

I am starting to work on NIFI-329 and plan to soon introduce processors
capable of interacting with IRC servers.

At this stage I am looking to implement only PublishIRC, however, given the
processor will use a controller service this may be later on expanded to
handle IRC message consumption (i.e. ConsumeIRC) and perhaps bot type
functionality by mimicking the ethos of HandleHttpRequest and
HandleHttpResponse processors.

Since this is the first time I am trying to write a controller service may
I ask you some feedback about my first reading?

Currently the "best" ASL2 IRC library seems to be KICL (Kitteh IRC Client
library[1]) which happens to support IRCv3, SSL and a bunch of other stuff.

KICL Client class seems to fit very well within a controller service
onEnabled, but I have the impression its event driven API is not a natural
fit to NiFi processors and will require some work similar to the one
required to get ListenSMTP working properly.

Is this reading correct? In you experience, is it manageable or should I
look for another library?


Any feedback will be appreciated.


[1] -
http://kicl.kitteh.org/en/latest/#kitteh-irc-client-library-documentation

Namespaces in MiNiFI-cpp

[Marc]

Good timezone appropriate salutation,

   I'd like to move the MiNiFi-CPP agent to a more controlled namespace,
namely org::apache::nifi::minifi . In an upcoming PR I'll introduce a
directory for I/O activities and using namespaces will help better isolate
and encapsulate the duties of the code; however, I wanted to make sure
everyone is okay with these namespace changes.

First and foremost, the question would be: will this impact anyone?

 https://issues.apache.org/jira/browse/MINIFI-217

I wouldn't immediately make the change, to avoid impacting devs who are
expecting the global namespace. I put an arbitrary estimate in the ticket
to reflect notice being given before applying the change to the mainline.

 Feel free to comment on the ticket with objections and/or timeline
requests. This is pretty low priority, but I wanted to provide sufficient
notice.

 Best Regards,
 Marc Parisi