[VOTE] Release Apache NiFi Flow Design System 0.1.0 RC3

[Scott Aslan]

Hello,


I am pleased to be calling this vote for the source release of Apache

NiFi Flow Design System 0.1.0.


The source zip, including signatures, etc. can be found at:

https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/


The Git tag is nifi-fds-0.1.0-RC3

The Git commit ID is 4f62f3d4a4626f66011289c7f1346c082a932184

*https://git-wip-us.apache.org/repos/asf?p=nifi-fds.git;a=commit;h=4f62f3d4a4626f66011289c7f1346c082a932184
*


Checksums of nifi-fds-0.1.0-source-release.zip:

SHA1:

ce971d78cc0660299dcee2233bc22fa78e2940af

SHA256:

7cb36f412b4147c0e071508f4f6d2ca76b2fe524382937d6552dbe7efe10a827

SHA512:

11e9cf4a529862da528bd616124e57f567503c25820e4b70e72d3d53f16bfed7c0c97f05f7c390267d2163cdae2c6151dc6d9a9bdf6c11be8a6dca4409b3bd15


Release artifacts are signed with the following key:

https://people.apache.org/keys/committer/scottyaslan.asc


KEYS file available here:

https://dist.apache.org/repos/dist/release/nifi/KEYS


7 issues were closed/resolved for this release:

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12343357


Release note highlights can be found here:

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-NiFiFlowDesignSystem0.1.0


The vote will be open for 72 hours.


Please download the release candidate and evaluate the necessary items

including checking hashes, signatures, build from source, and test.


The please vote:


[ ] +1 Release this package as nifi-fds-0.1.0

[ ] +0 no opinion

[ ] -1 Do not release this package because...

Apache NiFi Flow Design System 0.1.0 RC2 Release Helper Guide RC3

[Scott Aslan]

Hello Apache NiFi community,


Please find the associated guidance to help those interested in

validating/verifying the Apache NiFi Flow Design System release so they can

vote.


# Download latest KEYS file:

https://dist.apache.org/repos/dist/dev/nifi/KEYS


# Import keys file:

gpg --import KEYS


# Pull down nifi-fds-0.1.0 source release artifacts for review:


wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/nifi-fds-0.1.0-source-release.zip

wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/nifi-fds-0.1.0-source-release.zip.asc

wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/nifi-fds-0.1.0-source-release.zip.sha1

wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/nifi-fds-0.1.0-source-release.zip.sha256

wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/nifi-fds-0.1.0-source-release.zip.sha512


# Verify the signature

gpg --verify nifi-fds-0.1.0-source-release.zip.asc


# Verify the hashes (sha1, sha256, sha512) match the source and what was

provided in the vote email thread

sha1sum nifi-fds-0.1.0-source-release.zip

sha256sum nifi-fds-0.1.0-source-release.zip

sha512sum nifi-fds-0.1.0-source-release.zip


# Unzip nifi-fds-0.1.0-source-release.zip


# Verify the build works (npm version >= 5.6.0)

cd nifi-fds-0.1.0

npm run clean:install


# Verify the contents contain a good README, NOTICE, and LICENSE.


# Verify the git commit ID is correct


# Verify the RC was branched off the correct git commit ID


# Run the demo-app and make sure it works as expected
cd target

npm start


# Make sure the README, NOTICE, and LICENSE are present and correct

cd node_modules/@nifi-fds/core

# Send a response to the vote thread indicating a +1, 0, -1 based on

your findings.


Thank you for your time and effort to validate the release!

Re: Issue securing nifi with ca-server.

[Andy LoPresto]

Henk,

Thanks for following up and documenting this. I believe the deployment script 
is something custom you built, and not an artifact we offer, correct? We 
definitely made some improvements to the toolkit in 1.6.0, so the errors are 
clearer now. The minimum password length being enforced should have been called 
out in the migration guidance [1] and release notes [2], and we will try to 
make those types of changes more apparent in the future as well.

[1] https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance 

[2] 
https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.6.0
 



Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Jun 11, 2018, at 12:43 PM, Henk Reder  wrote:
> 
> In response to this
> 
> :
> 
> Hey again,
> 
> I resolved the problem with the ca-server. It was a combination of two
> problems. One, the deployment script was getting the latest version of nifi
> (1.6.0) but an earlier version of TLS Toolkit (1.5.0), that's why the
> message wasn't coming through. There was a version mismatch, and the
> messages just don't come through on the earlier version of the tool.
> 
> Once I got the latest version of TLS toolkit the error message came
> through. There was a new restriction on the passphrase (16 bytes or
> larger). Once I generated a longer passphrase the certs became valid again.
> 
> Thanks for everyones concern, this stuff is definitely not simple but
> having an active and open community helps a ton. Everyones input is very
> much appreciated!
> 
> Henk



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Issue securing nifi with ca-server.

[Henk Reder]

In response to this

:

Hey again,

I resolved the problem with the ca-server. It was a combination of two
problems. One, the deployment script was getting the latest version of nifi
(1.6.0) but an earlier version of TLS Toolkit (1.5.0), that's why the
message wasn't coming through. There was a version mismatch, and the
messages just don't come through on the earlier version of the tool.

Once I got the latest version of TLS toolkit the error message came
through. There was a new restriction on the passphrase (16 bytes or
larger). Once I generated a longer passphrase the certs became valid again.

Thanks for everyones concern, this stuff is definitely not simple but
having an active and open community helps a ton. Everyones input is very
much appreciated!

Henk

Re: [DISCUSS] Release candidate timeframe for Apache NiFi 1.7.0

[Joe Witt]

Thanks Andy.  Will help keep up the review bandwidth as well.  Have
also taken a particular focus again as of late on our license and
notice work as we have more and more contributors/reviewers we need to
really focus there.  Will share any findings.

Thanks

On Mon, Jun 11, 2018 at 2:17 PM, Kevin Doran  wrote:
> This sounds like a good release plan / schedule for the upcoming versions of 
> the sub projects. Thanks, Andy.
>
>
>
> Kevin
>
>
>
> From: Andy LoPresto 
> Reply-To: 
> Date: Monday, June 11, 2018 at 14:03
> To: 
> Subject: Re: [DISCUSS] Release candidate timeframe for Apache NiFi 1.7.0
>
>
>
> Hi folks,
>
>
>
> With the upcoming release of Apache NiFi FDS 0.1.0 and the following release 
> of Apache NiFi Registry 0.2.0, it makes sense to hold off this release a few 
> days to get the latest versions of those projects.
>
>
>
> Here is the anticipated schedule:
>
>
>
> * 06/11 - NiFi FDS RC2 proposed
>
> * 06/14 - NiFi FDS RC2 accepted & released
>
> * 06/15 - NiFi Registry RC1 proposed
>
> * 06/18 - NiFi Registry RC1 accepted & released
>
> * 06/19 - NIFi RC1 proposed
>
> * 06/21 - NiFi RC1 accepted & released
>
>
>
> Obviously, as the version issue in FDS RC1 demonstrated, there can be 
> obstacles to overcome that may slightly affect this timeline, but I think 
> this is a reasonable expectation moving forward.
>
>
>
> As a reminder, this is the filter I am using for visibility into the close 
> out tickets. Please work with each other to get pending PRs reviewed and 
> merged quickly. Thank you.
>
>
>
> project = "Apache NiFi" AND status in ("In Progress", Reopened, "Patch 
> Available") AND updated > '2018-05-28’ ORDER BY updated DESC
>
>
>
> https://issues.apache.org/jira/issues/?filter=12344019=project%20%3D%20%22Apache%20NiFi%22%20AND%20status%20in%20(%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)%20AND%20updated%20%3E%20%272018-05-28%27%20order%20by%20updated%20desc
>
>
>
>
>
> Andy LoPresto
>
> alopre...@apache.org
>
> alopresto.apa...@gmail.com
>
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
>
>
> On Jun 11, 2018, at 7:53 AM, Joe Witt  wrote:
>
>
>
> NIFI-5022 is merged/resolved now Otto.
>
> Thanks
>
> On Mon, Jun 11, 2018 at 6:55 AM, Otto Fowler  wrote:
>
>
> I would really love to get a review for NIFI-5022 for this release.  Please
> let me know if there is a way to make this more likely.
>
>
>
> On June 5, 2018 at 15:09:50, Andy LoPresto (alopre...@apache.org) wrote:
>
> Sending out an update. I am using the following search query to take a look
> at what is still outstanding for 1.7.0. Currently I have visibility on the
> following issues:
>
> project = "Apache NiFi" AND (fixVersion = "1.7.0" OR (status = "Patch
> Available" AND updated > '2018-05-28')) AND status in (Open, "In Progress",
> "Reopened", "Patch Available")
>
> Patch Available (i.e. committers, please review these):
> * NIFI-5241 When calculating stats for components, use synchronized methods
> instead of atomic variables — Mark Payne
> * NIFI-5200 Nested ProcessSession.read resulting in outer stream being
> closed — Mark Payne
> * NIFI-5209 Remove toolkit migration without password functionality — Andy
> LoPresto
> * NIFI-5166 Create deep learning classification and regression processor —
> Mans Singh (Marked as In Progress but there is a PR with some comments)
> * NIFI-5226 Implement a Record API based PutInfluxDB processor — Unassigned
> (Marked as Open but there is a PR with some comments)
> * NIFI-5102 MarkLogic DB Processors — Unassigned (Marked as Open but there
> is a PR with some comments)
> * NIFI-5268 JoltTransformJSON specification property fails with EL — Koji
> Kawamura
> * NIFI-5266 PutElasticsearchHttp processors should sanitize parameters —
> Matt Burgess
> * NIFI-5263 Address auditing of Controller Service Referencing Components —
> Matt Gilman
> * NIFI-5260 Regression in nifi-processor-bundle-archetype — Bryan Bende
> * NIFI-5257 Expand Couchbase Server integration as a cache storage — Koji
> Kawamura
> * NIFI-5237 Wrong redirect from login behind a context path when using
> OpenID authentication — Matt Gilman
> * NIFI-5200 Nested ProcessSession.read resulting in outer stream being
> closed — Mark Payne
> * NIFI-5192 Allow expression language in ‘Schema File’ property for
> ValidateXML processor — Unassigned
> * NIFI-5059 MongoDBLookupService should be able to determine a schema or
> have one provided — Mike Thomsen
> * NIFI-5054 NiFi Couchbase Processors do not support User Authentication —
> Koji Kawamura
> * NIFI-5022 Create an AWS Gateway Web API version of InvokeHTTP — Otto
> Fowler
> * NIFI-4930 Nar-Dependency-Version - timestamped snapshot version problem —
> Bryan Bende
> * NIFI-1321 Support appending files in PutFile — Unassigned
>
> Reopened (i.e. Jeff, Mike, what do you need from the community to get these
> in?):
> * NIFI-5145 MockPropertyValue.evaluateExpressionLanguage(FlowFile) cannot
> handle null inputs — Mike Thomsen
> * NIFI-5175 NiFi 

Re: [VOTE] Release Apache NiFi Flow Design System 0.1.0

[Scott Aslan]

Cancel this RC due to another incorrect version string.

On Mon, Jun 11, 2018 at 1:11 PM Scott Aslan  wrote:

> Hello,
>
>
> I am pleased to be calling this vote for the source release of Apache
>
> NiFi Flow Design System 0.1.0.
>
>
> The source zip, including signatures, etc. can be found at:
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/
>
>
> The Git tag is nifi-fds-0.1.0-RC2
>
> The Git commit ID is 633c0eba1d010fec93c29ec17292e52f60bf944d
>
> *https://git-wip-us.apache.org/repos/asf?p=nifi-fds.git;a=commit;h=633c0eba1d010fec93c29ec17292e52f60bf944d
> *
>
>
> Checksums of nifi-fds-0.1.0-source-release.zip:
>
> SHA1:
>
> 096d4d395c9210caf2dfef5d4050c59600f0f51a
>
> SHA256:
>
> 8e80bb0a5a029c290b2b78b6d4bdafcf613db248fe602c21c35b3e604710cdce
>
> SHA512:
>
>
> 7227fc6ea547463c49a070a79764631cd0e2b84cd50c046706009f5e0a707116d519c572fd49a811d41fd2b37bdd75aa274971a9fd2db9024c225cc7849549d5
>
>
> Release artifacts are signed with the following key:
>
> https://people.apache.org/keys/committer/scottyaslan.asc
>
>
> KEYS file available here:
>
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
>
> 7 issues were closed/resolved for this release:
>
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12343357
>
>
> Release note highlights can be found here:
>
>
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-NiFiFlowDesignSystem0.1.0
>
>
> The vote will be open for 72 hours.
>
>
> Please download the release candidate and evaluate the necessary items
>
> including checking hashes, signatures, build from source, and test.
>
>
> The please vote:
>
>
> [ ] +1 Release this package as nifi-fds-0.1.0
>
> [ ] +0 no opinion
>
> [ ] -1 Do not release this package because...
>
>

Re: [DISCUSS] Release candidate timeframe for Apache NiFi 1.7.0

[Kevin Doran]

This sounds like a good release plan / schedule for the upcoming versions of 
the sub projects. Thanks, Andy.

 

Kevin

 

From: Andy LoPresto 
Reply-To: 
Date: Monday, June 11, 2018 at 14:03
To: 
Subject: Re: [DISCUSS] Release candidate timeframe for Apache NiFi 1.7.0

 

Hi folks,

 

With the upcoming release of Apache NiFi FDS 0.1.0 and the following release of 
Apache NiFi Registry 0.2.0, it makes sense to hold off this release a few days 
to get the latest versions of those projects. 

 

Here is the anticipated schedule:

 

* 06/11 - NiFi FDS RC2 proposed

* 06/14 - NiFi FDS RC2 accepted & released

* 06/15 - NiFi Registry RC1 proposed

* 06/18 - NiFi Registry RC1 accepted & released

* 06/19 - NIFi RC1 proposed

* 06/21 - NiFi RC1 accepted & released

 

Obviously, as the version issue in FDS RC1 demonstrated, there can be obstacles 
to overcome that may slightly affect this timeline, but I think this is a 
reasonable expectation moving forward. 

 

As a reminder, this is the filter I am using for visibility into the close out 
tickets. Please work with each other to get pending PRs reviewed and merged 
quickly. Thank you. 

 

project = "Apache NiFi" AND status in ("In Progress", Reopened, "Patch 
Available") AND updated > '2018-05-28’ ORDER BY updated DESC

 

https://issues.apache.org/jira/issues/?filter=12344019=project%20%3D%20%22Apache%20NiFi%22%20AND%20status%20in%20(%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)%20AND%20updated%20%3E%20%272018-05-28%27%20order%20by%20updated%20desc

 

 

Andy LoPresto

alopre...@apache.org

alopresto.apa...@gmail.com

PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

 

On Jun 11, 2018, at 7:53 AM, Joe Witt  wrote:

 

NIFI-5022 is merged/resolved now Otto.

Thanks

On Mon, Jun 11, 2018 at 6:55 AM, Otto Fowler  wrote:


I would really love to get a review for NIFI-5022 for this release.  Please
let me know if there is a way to make this more likely.



On June 5, 2018 at 15:09:50, Andy LoPresto (alopre...@apache.org) wrote:

Sending out an update. I am using the following search query to take a look
at what is still outstanding for 1.7.0. Currently I have visibility on the
following issues:

project = "Apache NiFi" AND (fixVersion = "1.7.0" OR (status = "Patch
Available" AND updated > '2018-05-28')) AND status in (Open, "In Progress",
"Reopened", "Patch Available")

Patch Available (i.e. committers, please review these):
* NIFI-5241 When calculating stats for components, use synchronized methods
instead of atomic variables — Mark Payne
* NIFI-5200 Nested ProcessSession.read resulting in outer stream being
closed — Mark Payne
* NIFI-5209 Remove toolkit migration without password functionality — Andy
LoPresto
* NIFI-5166 Create deep learning classification and regression processor —
Mans Singh (Marked as In Progress but there is a PR with some comments)
* NIFI-5226 Implement a Record API based PutInfluxDB processor — Unassigned
(Marked as Open but there is a PR with some comments)
* NIFI-5102 MarkLogic DB Processors — Unassigned (Marked as Open but there
is a PR with some comments)
* NIFI-5268 JoltTransformJSON specification property fails with EL — Koji
Kawamura
* NIFI-5266 PutElasticsearchHttp processors should sanitize parameters —
Matt Burgess
* NIFI-5263 Address auditing of Controller Service Referencing Components —
Matt Gilman
* NIFI-5260 Regression in nifi-processor-bundle-archetype — Bryan Bende
* NIFI-5257 Expand Couchbase Server integration as a cache storage — Koji
Kawamura
* NIFI-5237 Wrong redirect from login behind a context path when using
OpenID authentication — Matt Gilman
* NIFI-5200 Nested ProcessSession.read resulting in outer stream being
closed — Mark Payne
* NIFI-5192 Allow expression language in ‘Schema File’ property for
ValidateXML processor — Unassigned
* NIFI-5059 MongoDBLookupService should be able to determine a schema or
have one provided — Mike Thomsen
* NIFI-5054 NiFi Couchbase Processors do not support User Authentication —
Koji Kawamura
* NIFI-5022 Create an AWS Gateway Web API version of InvokeHTTP — Otto
Fowler
* NIFI-4930 Nar-Dependency-Version - timestamped snapshot version problem —
Bryan Bende
* NIFI-1321 Support appending files in PutFile — Unassigned

Reopened (i.e. Jeff, Mike, what do you need from the community to get these
in?):
* NIFI-5145 MockPropertyValue.evaluateExpressionLanguage(FlowFile) cannot
handle null inputs — Mike Thomsen
* NIFI-5175 NiFi built with Java 1.8 needs to run on Java 1.9 — Jeff Storck

Open (i.e. can someone please do an s/2017/2018/ on the docs?):
* NIFI-5106 Update docs to reflect 2018 where applicable — Aldrin Piri

If anyone has other work that they are in the process of doing, please reply
here and update the Jira appropriately. I’ll send out another summary on
Friday. Thanks.

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On Jun 1, 2018, at 10:28 AM, Andy 

Re: [DISCUSS] Release candidate timeframe for Apache NiFi 1.7.0

[Andy LoPresto]

Hi folks,

With the upcoming release of Apache NiFi FDS 0.1.0 and the following release of 
Apache NiFi Registry 0.2.0, it makes sense to hold off this release a few days 
to get the latest versions of those projects.

Here is the anticipated schedule:

* 06/11 - NiFi FDS RC2 proposed
* 06/14 - NiFi FDS RC2 accepted & released
* 06/15 - NiFi Registry RC1 proposed
* 06/18 - NiFi Registry RC1 accepted & released
* 06/19 - NIFi RC1 proposed
* 06/21 - NiFi RC1 accepted & released

Obviously, as the version issue in FDS RC1 demonstrated, there can be obstacles 
to overcome that may slightly affect this timeline, but I think this is a 
reasonable expectation moving forward.

As a reminder, this is the filter I am using for visibility into the close out 
tickets. Please work with each other to get pending PRs reviewed and merged 
quickly. Thank you.

project = "Apache NiFi" AND status in ("In Progress", Reopened, "Patch 
Available") AND updated > '2018-05-28’ ORDER BY updated DESC

https://issues.apache.org/jira/issues/?filter=12344019=project%20%3D%20%22Apache%20NiFi%22%20AND%20status%20in%20(%22In%20Progress%22%2C%20Reopened%2C%20%22Patch%20Available%22)%20AND%20updated%20%3E%20%272018-05-28%27%20order%20by%20updated%20desc
 



Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Jun 11, 2018, at 7:53 AM, Joe Witt  wrote:
> 
> NIFI-5022 is merged/resolved now Otto.
> 
> Thanks
> 
> On Mon, Jun 11, 2018 at 6:55 AM, Otto Fowler  wrote:
>> I would really love to get a review for NIFI-5022 for this release.  Please
>> let me know if there is a way to make this more likely.
>> 
>> 
>> 
>> On June 5, 2018 at 15:09:50, Andy LoPresto (alopre...@apache.org) wrote:
>> 
>> Sending out an update. I am using the following search query to take a look
>> at what is still outstanding for 1.7.0. Currently I have visibility on the
>> following issues:
>> 
>> project = "Apache NiFi" AND (fixVersion = "1.7.0" OR (status = "Patch
>> Available" AND updated > '2018-05-28')) AND status in (Open, "In Progress",
>> "Reopened", "Patch Available")
>> 
>> Patch Available (i.e. committers, please review these):
>> * NIFI-5241 When calculating stats for components, use synchronized methods
>> instead of atomic variables — Mark Payne
>> * NIFI-5200 Nested ProcessSession.read resulting in outer stream being
>> closed — Mark Payne
>> * NIFI-5209 Remove toolkit migration without password functionality — Andy
>> LoPresto
>> * NIFI-5166 Create deep learning classification and regression processor —
>> Mans Singh (Marked as In Progress but there is a PR with some comments)
>> * NIFI-5226 Implement a Record API based PutInfluxDB processor — Unassigned
>> (Marked as Open but there is a PR with some comments)
>> * NIFI-5102 MarkLogic DB Processors — Unassigned (Marked as Open but there
>> is a PR with some comments)
>> * NIFI-5268 JoltTransformJSON specification property fails with EL — Koji
>> Kawamura
>> * NIFI-5266 PutElasticsearchHttp processors should sanitize parameters —
>> Matt Burgess
>> * NIFI-5263 Address auditing of Controller Service Referencing Components —
>> Matt Gilman
>> * NIFI-5260 Regression in nifi-processor-bundle-archetype — Bryan Bende
>> * NIFI-5257 Expand Couchbase Server integration as a cache storage — Koji
>> Kawamura
>> * NIFI-5237 Wrong redirect from login behind a context path when using
>> OpenID authentication — Matt Gilman
>> * NIFI-5200 Nested ProcessSession.read resulting in outer stream being
>> closed — Mark Payne
>> * NIFI-5192 Allow expression language in ‘Schema File’ property for
>> ValidateXML processor — Unassigned
>> * NIFI-5059 MongoDBLookupService should be able to determine a schema or
>> have one provided — Mike Thomsen
>> * NIFI-5054 NiFi Couchbase Processors do not support User Authentication —
>> Koji Kawamura
>> * NIFI-5022 Create an AWS Gateway Web API version of InvokeHTTP — Otto
>> Fowler
>> * NIFI-4930 Nar-Dependency-Version - timestamped snapshot version problem —
>> Bryan Bende
>> * NIFI-1321 Support appending files in PutFile — Unassigned
>> 
>> Reopened (i.e. Jeff, Mike, what do you need from the community to get these
>> in?):
>> * NIFI-5145 MockPropertyValue.evaluateExpressionLanguage(FlowFile) cannot
>> handle null inputs — Mike Thomsen
>> * NIFI-5175 NiFi built with Java 1.8 needs to run on Java 1.9 — Jeff Storck
>> 
>> Open (i.e. can someone please do an s/2017/2018/ on the docs?):
>> * NIFI-5106 Update docs to reflect 2018 where applicable — Aldrin Piri
>> 
>> If anyone has other work that they are in the process of doing, please reply
>> here and update the Jira appropriately. I’ll send out another summary on
>> Friday. Thanks.
>> 
>> Andy LoPresto
>> 

Apache NiFi Flow Design System 0.1.0 RC2 Release Helper Guide

[Scott Aslan]

Hello Apache NiFi community,


Please find the associated guidance to help those interested in

validating/verifying the Apache NiFi Flow Design System release so they can

vote.


# Download latest KEYS file:

https://dist.apache.org/repos/dist/dev/nifi/KEYS


# Import keys file:

gpg --import KEYS


# Pull down nifi-fds-0.1.0 source release artifacts for review:


wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/nifi-fds-0.1.0-source-release.zip

wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/nifi-fds-0.1.0-source-release.zip.asc

wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/nifi-fds-0.1.0-source-release.zip.sha1

wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/nifi-fds-0.1.0-source-release.zip.sha256

wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/nifi-fds-0.1.0-source-release.zip.sha512


# Verify the signature

gpg --verify nifi-fds-0.1.0-source-release.zip.asc


# Verify the hashes (sha1, sha256, sha512) match the source and what was

provided in the vote email thread

sha1sum nifi-fds-0.1.0-source-release.zip

sha256sum nifi-fds-0.1.0-source-release.zip

sha512sum nifi-fds-0.1.0-source-release.zip


# Unzip nifi-fds-0.1.0-source-release.zip


# Verify the build works (npm version >= 5.6.0)

cd nifi-fds-0.1.0

npm run clean:install


# Verify the contents contain a good README, NOTICE, and LICENSE.


# Verify the git commit ID is correct


# Verify the RC was branched off the correct git commit ID


# Run the demo-app and make sure it works as expected
cd target

npm start


# Make sure the README, NOTICE, and LICENSE are present and correct

cd node_modules/@nifi-fds/core

# Send a response to the vote thread indicating a +1, 0, -1 based on

your findings.


Thank you for your time and effort to validate the release!

[VOTE] Release Apache NiFi Flow Design System 0.1.0

[Scott Aslan]

Hello,


I am pleased to be calling this vote for the source release of Apache

NiFi Flow Design System 0.1.0.


The source zip, including signatures, etc. can be found at:

https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/


The Git tag is nifi-fds-0.1.0-RC2

The Git commit ID is 633c0eba1d010fec93c29ec17292e52f60bf944d

*https://git-wip-us.apache.org/repos/asf?p=nifi-fds.git;a=commit;h=633c0eba1d010fec93c29ec17292e52f60bf944d
*


Checksums of nifi-fds-0.1.0-source-release.zip:

SHA1:

096d4d395c9210caf2dfef5d4050c59600f0f51a

SHA256:

8e80bb0a5a029c290b2b78b6d4bdafcf613db248fe602c21c35b3e604710cdce

SHA512:

7227fc6ea547463c49a070a79764631cd0e2b84cd50c046706009f5e0a707116d519c572fd49a811d41fd2b37bdd75aa274971a9fd2db9024c225cc7849549d5


Release artifacts are signed with the following key:

https://people.apache.org/keys/committer/scottyaslan.asc


KEYS file available here:

https://dist.apache.org/repos/dist/release/nifi/KEYS


7 issues were closed/resolved for this release:

https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12343357


Release note highlights can be found here:

https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-NiFiFlowDesignSystem0.1.0


The vote will be open for 72 hours.


Please download the release candidate and evaluate the necessary items

including checking hashes, signatures, build from source, and test.


The please vote:


[ ] +1 Release this package as nifi-fds-0.1.0

[ ] +0 no opinion

[ ] -1 Do not release this package because...

Re: LookupService + flowfile attributes

[Mike Thomsen]

Koji,

After reading Mark's comments on GitHub, it occurred to me that the MongoDB
lookup service and the ES one I have as a PR would be screwed up if we take
the original approach because they blindly build a query from the total
coordinates set. So they'd add flowfile attributes as criteria by default.
I'll update the PR accordingly and make the new method default to the
existing one in all of the lookup services that are already there.

On Sat, Jun 9, 2018 at 8:44 AM Mike Thomsen  wrote:

> https://issues.apache.org/jira/browse/NIFI-5287
>
> On Sat, Jun 9, 2018 at 1:20 AM Koji Kawamura 
> wrote:
>
>> Thanks Mike for starting the discussion.
>>
>> Yes, I believe that will make LookupService and Schema access strategy
>> much easier, reusable, and useful.
>>
>> What I was imagined is not adding new method signature, but simply
>> copy certain FlowFile attributes into the coordinates map.
>> We can add that at LookupRecord.
>> Currently LookupAttribute only uses one coordinate value and can be
>> left as it is.
>>
>> Specifically, by adding new processor property, 'Copy FlowFile
>> Attributes into Coordinates', where user can define RegularExpression
>> to select which attributes to copy.
>> I think it's fine to mix FlowFile attributes and values defined as
>> dynamic properties into the same coordinates map.
>> The put oder should be FlowFile attributes, then dynamic properties,
>> so that user can overwrite attribute values when necessary.
>>
>> Koji
>>
>>
>> On Sat, Jun 9, 2018 at 1:06 AM, Mike Thomsen 
>> wrote:
>> > On the RestLookupService PR I think Koji mentioned the idea of expanding
>> > the lookup capability to include flowfile attributes. That sort of thing
>> > would be immensely useful on two PRs I have already open for lookup
>> service
>> > changes for ES and Mongo. Koji, add your thoughts, but what I'm thinking
>> > would be a new PR that adds:
>> >
>> > T lookup(Map flowfileAttributes, Map
>> > coordinates);
>> >
>> > to the LookupService interface and has the related processors pass in
>> the
>> > flowfile attribute map. Specifically, it would help make the schema
>> access
>> > capabilities really usable with lookup services (see
>> MongoDBLookupService
>> > PR for example; I added a new SchemaRegistryService type for JSON
>> sources)
>>
>

Re: Java 10

[Jeff]

HI all,

I'm a little late on this thread, but wanted to add that running NiFi on
Java 9+ that has been built with Java 1.8 currently works on master.  I am
working through issues getting the compilation on Java 9+ to work, mainly
through a maven profile that auto-activates if Maven detects a Java version
above 1.8.  I've made some good progress...  Most likely it will take me
the rest of the week if not longer to work through the rest of the build
problems.  It's difficult to forecast how many issues remain, but the
majority of the issues so far are due to jaxb modules that were provided by
the JRE not being available on the classpath from Java 9 and beyond.

On Thu, Jun 7, 2018 at 10:53 AM Joe Witt  wrote:

> All,
>
> There is work on-going to allow NiFi to:
>
> A) Run on Java 9, 10, and beyond.
> B) Compile on Java 9, 10, and beyond.
> C) Code leveraging Java 9, 10, and beyond capabilities.
>
> We can make things in category A happen anytime and these should be
> top priority.
> We can make things happen in category B happen and they should be
> lower priority than category A.
> We can make things in Category C be supported in the next major NiFi
> release (ie NiFi 2.x).
>
> No PR can accept Category C things until then.
>
> Thanks
> Joe
>
> On Thu, Jun 7, 2018 at 10:50 AM, Mike Thomsen 
> wrote:
> > I would say that's a safe assumption. Given the newly aggressive release
> > cadence of Java, I would also caution you to be careful because a lot of
> > the transitive dependencies might suddenly break between releases.
> >
> > On Thu, Jun 7, 2018 at 10:38 AM Jon Logan  wrote:
> >
> >> Will any PRs with Java 9 features be rejected until then? (ie.
> preserving
> >> Java-8 compatibility) Is there a plan for supporting multiple Java
> >> versions?
> >>
> >> On Thu, Jun 7, 2018 at 10:18 AM, Mike Thomsen 
> >> wrote:
> >>
> >> > Once that is done, it should run on Java 10. However, under the new
> >> Oracle
> >> > version rules Java 10 is a short term iteration. Java 11 will
> probably be
> >> > the next version officially supported by NiFi because it's supposed
> to be
> >> > Java's next LTS release.
> >> >
> >> > Java 10 features like the var key will also be rejected in PRs until
> that
> >> > point.
> >> >
> >> > On Thu, Jun 7, 2018 at 9:48 AM Otto Fowler 
> >> > wrote:
> >> >
> >> > > You can track progress here:
> >> > > https://issues.apache.org/jira/browse/NIFI-5174
> >> > > There has been some progress very recently.
> >> > >
> >> > >
> >> > >
> >> > > On June 7, 2018 at 09:40:50, Sivaprasanna (
> sivaprasanna...@gmail.com)
> >> > > wrote:
> >> > >
> >> > > Nope. Not yet. AFAIK, there is a Jira to support Java 9.
> >> > >
> >> > > On Thu, Jun 7, 2018 at 7:09 PM, kirilzilla  wrote:
> >> > >
> >> > > > does nifi support java 10 ? or even java 9 ?
> >> > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
> >> > > >
> >> > >
> >> >
> >>
>

Re: [VOTE] Release Apache NiFi Flow Design System 0.1.0

[Scott Aslan]

Good catch MattG!

The Apache NiFi Flow Design System 0.1.0 RC1 is canceled due to having the
incorrect version number in the package-lock.json. I will address this
issue under https://issues.apache.org/jira/browse/NIFI-5240.

I also filed NIFI-5294 and NIFI-5295 to address the demo-app load time and
display the FDS version number in the demo-app.

-Scott

On Mon, Jun 11, 2018 at 9:45 AM Matt Gilman  wrote:

> -1 binding
>
> Signatures, hashes, etc look right. Build was good and the demo application
> started up just fine. The demo application did load a little slowly so we
> may want to file a JIRA to improve this in the future.
>
> However, it appears that the version bump only happened in the package.json
> and not the package-lock.json. Because of this, the built artifacts that
> ended up in the node_modules had a version of 0.0.0 (not 0.1.0). It may
> make sense to render this version in the demo application to prevent this
> in the future. There's an added benefit that we'd also know what version of
> nifi-fds was being shown.
>
> Matt
>
> On Fri, Jun 8, 2018 at 8:03 PM, Scott Aslan  wrote:
>
> > Hello,
> >
> >
> > I am pleased to be calling this vote for the source release of Apache
> >
> > NiFi Flow Design System 0.1.0.
> >
> >
> > The source zip, including signatures, etc. can be found at:
> >
> > https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/
> >
> >
> > The Git tag is nifi-fds-0.1.0-RC1
> >
> > The Git commit ID is 35423f267de0755087a274ece79c81ef1a4ee641
> >
> > https://git-wip-us.apache.org/repos/asf?p=nifi-fds.git;a=commit;h=
> > 35423f267de0755087a274ece79c81ef1a4ee641
> >
> >
> > Checksums of nifi-fds-0.1.0-source-release.zip:
> >
> > SHA1: 389b7bda654cfb9a79efa5f893eb37b9f73caaf5
> >
> > SHA256: b42de408e797df859ab8e11e39bd717c996df775179bfae82bf4392bb346a135
> >
> > SHA512:
> > 3b1cca1d5c3cd1786fb54deaa9e9761feb1b8bd2b365658ecef302ba617d
> > d3fe66766513d76ad8686ce7f87a25d6d6152ccdae1bf2dfb3d5dba3e352d53a4122
> >
> >
> > Release artifacts are signed with the following key:
> >
> > https://people.apache.org/keys/committer/scottyaslan.asc
> >
> >
> > KEYS file available here:
> >
> > https://dist.apache.org/repos/dist/release/nifi/KEYS
> >
> >
> > 7 issues were closed/resolved for this release:
> >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> > projectId=12316020=12343357
> >
> >
> > Release note highlights can be found here:
> >
> > https://cwiki.apache.org/confluence/display/NIFI/
> > Release+Notes#ReleaseNotes-NiFiFlowDesignSystem0.1.0
> >
> >
> > The vote will be open for 72 hours.
> >
> >
> > Please download the release candidate and evaluate the necessary items
> >
> > including checking hashes, signatures, build from source, and test.
> >
> >
> > The please vote:
> >
> >
> > [ ] +1 Release this package as nifi-fds-0.1.0
> >
> > [ ] +0 no opinion
> >
> > [ ] -1 Do not release this package because...
> >
>

Re: [DISCUSS] Release candidate timeframe for Apache NiFi 1.7.0

[Joe Witt]

NIFI-5022 is merged/resolved now Otto.

Thanks

On Mon, Jun 11, 2018 at 6:55 AM, Otto Fowler  wrote:
> I would really love to get a review for NIFI-5022 for this release.  Please
> let me know if there is a way to make this more likely.
>
>
>
> On June 5, 2018 at 15:09:50, Andy LoPresto (alopre...@apache.org) wrote:
>
> Sending out an update. I am using the following search query to take a look
> at what is still outstanding for 1.7.0. Currently I have visibility on the
> following issues:
>
> project = "Apache NiFi" AND (fixVersion = "1.7.0" OR (status = "Patch
> Available" AND updated > '2018-05-28')) AND status in (Open, "In Progress",
> "Reopened", "Patch Available")
>
> Patch Available (i.e. committers, please review these):
> * NIFI-5241 When calculating stats for components, use synchronized methods
> instead of atomic variables — Mark Payne
> * NIFI-5200 Nested ProcessSession.read resulting in outer stream being
> closed — Mark Payne
> * NIFI-5209 Remove toolkit migration without password functionality — Andy
> LoPresto
> * NIFI-5166 Create deep learning classification and regression processor —
> Mans Singh (Marked as In Progress but there is a PR with some comments)
> * NIFI-5226 Implement a Record API based PutInfluxDB processor — Unassigned
> (Marked as Open but there is a PR with some comments)
> * NIFI-5102 MarkLogic DB Processors — Unassigned (Marked as Open but there
> is a PR with some comments)
> * NIFI-5268 JoltTransformJSON specification property fails with EL — Koji
> Kawamura
> * NIFI-5266 PutElasticsearchHttp processors should sanitize parameters —
> Matt Burgess
> * NIFI-5263 Address auditing of Controller Service Referencing Components —
> Matt Gilman
> * NIFI-5260 Regression in nifi-processor-bundle-archetype — Bryan Bende
> * NIFI-5257 Expand Couchbase Server integration as a cache storage — Koji
> Kawamura
> * NIFI-5237 Wrong redirect from login behind a context path when using
> OpenID authentication — Matt Gilman
> * NIFI-5200 Nested ProcessSession.read resulting in outer stream being
> closed — Mark Payne
> * NIFI-5192 Allow expression language in ‘Schema File’ property for
> ValidateXML processor — Unassigned
> * NIFI-5059 MongoDBLookupService should be able to determine a schema or
> have one provided — Mike Thomsen
> * NIFI-5054 NiFi Couchbase Processors do not support User Authentication —
> Koji Kawamura
> * NIFI-5022 Create an AWS Gateway Web API version of InvokeHTTP — Otto
> Fowler
> * NIFI-4930 Nar-Dependency-Version - timestamped snapshot version problem —
> Bryan Bende
> * NIFI-1321 Support appending files in PutFile — Unassigned
>
> Reopened (i.e. Jeff, Mike, what do you need from the community to get these
> in?):
> * NIFI-5145 MockPropertyValue.evaluateExpressionLanguage(FlowFile) cannot
> handle null inputs — Mike Thomsen
> * NIFI-5175 NiFi built with Java 1.8 needs to run on Java 1.9 — Jeff Storck
>
> Open (i.e. can someone please do an s/2017/2018/ on the docs?):
> * NIFI-5106 Update docs to reflect 2018 where applicable — Aldrin Piri
>
> If anyone has other work that they are in the process of doing, please reply
> here and update the Jira appropriately. I’ll send out another summary on
> Friday. Thanks.
>
> Andy LoPresto
> alopre...@apache.org
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On Jun 1, 2018, at 10:28 AM, Andy LoPresto  wrote:
>
> Hi folks,
>
> It’s been a little bit since 1.6.0 was released, and there have been a ton
> of new features and bug fixes that have made it in to master. With all this
> great work, I think it’s a good time to consider the next release. I
> volunteer to RM for this one.
>
> I’m thinking the next couple weeks are a good timeframe, and I wanted to put
> this out there early so anyone who is working on something they want in can
> be aware of the approaching release and communicate on the list to keep
> everyone apprised. Thanks for all your hard work.
>
> Andy LoPresto
> alopre...@apache.org
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
>

Re: [VOTE] Release Apache NiFi Flow Design System 0.1.0

[Matt Gilman]

-1 binding

Signatures, hashes, etc look right. Build was good and the demo application
started up just fine. The demo application did load a little slowly so we
may want to file a JIRA to improve this in the future.

However, it appears that the version bump only happened in the package.json
and not the package-lock.json. Because of this, the built artifacts that
ended up in the node_modules had a version of 0.0.0 (not 0.1.0). It may
make sense to render this version in the demo application to prevent this
in the future. There's an added benefit that we'd also know what version of
nifi-fds was being shown.

Matt

On Fri, Jun 8, 2018 at 8:03 PM, Scott Aslan  wrote:

> Hello,
>
>
> I am pleased to be calling this vote for the source release of Apache
>
> NiFi Flow Design System 0.1.0.
>
>
> The source zip, including signatures, etc. can be found at:
>
> https://dist.apache.org/repos/dist/dev/nifi/nifi-fds/nifi-fds-0.1.0/
>
>
> The Git tag is nifi-fds-0.1.0-RC1
>
> The Git commit ID is 35423f267de0755087a274ece79c81ef1a4ee641
>
> https://git-wip-us.apache.org/repos/asf?p=nifi-fds.git;a=commit;h=
> 35423f267de0755087a274ece79c81ef1a4ee641
>
>
> Checksums of nifi-fds-0.1.0-source-release.zip:
>
> SHA1: 389b7bda654cfb9a79efa5f893eb37b9f73caaf5
>
> SHA256: b42de408e797df859ab8e11e39bd717c996df775179bfae82bf4392bb346a135
>
> SHA512:
> 3b1cca1d5c3cd1786fb54deaa9e9761feb1b8bd2b365658ecef302ba617d
> d3fe66766513d76ad8686ce7f87a25d6d6152ccdae1bf2dfb3d5dba3e352d53a4122
>
>
> Release artifacts are signed with the following key:
>
> https://people.apache.org/keys/committer/scottyaslan.asc
>
>
> KEYS file available here:
>
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
>
> 7 issues were closed/resolved for this release:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12316020=12343357
>
>
> Release note highlights can be found here:
>
> https://cwiki.apache.org/confluence/display/NIFI/
> Release+Notes#ReleaseNotes-NiFiFlowDesignSystem0.1.0
>
>
> The vote will be open for 72 hours.
>
>
> Please download the release candidate and evaluate the necessary items
>
> including checking hashes, signatures, build from source, and test.
>
>
> The please vote:
>
>
> [ ] +1 Release this package as nifi-fds-0.1.0
>
> [ ] +0 no opinion
>
> [ ] -1 Do not release this package because...
>

Re: Should the elastic search client service impl get renamed before 1.7?

[Bryan Bende]

Lets make sure this is mentioned in the migration notes wiki for 1.7.0
since it will create a ghost component for existing flows that have
the ES service with the original name.

On Mon, Jun 11, 2018 at 9:16 AM, Mike Thomsen  wrote:
> Should mention that I did not check the L for the ES 6.X version. That
> needs to be reviewed as part of the PR.
>
> On Mon, Jun 11, 2018 at 7:54 AM Mike Thomsen  wrote:
>
>> https://github.com/apache/nifi/pull/2782
>>
>> Renamed the service and added a clone for 6.X. The integration tests use a
>> Maven ES plugin that downloads ES, bootstraps it with test data and runs
>> the integration tests against that. The 6.X version uses the latest stable
>> build (6.2.4) and passed all integration tests.
>>
>> On Sun, Jun 10, 2018 at 11:35 AM Mark Payne  wrote:
>>
>>> I think it’s a good idea to go ahead and rename it. Even if 6.x doesn’t
>>> make breaking changes, 7.x may. And I’d rather not have an
>>> ElasticSearchClient that works for 5.x and 6.x and then an
>>> ElasticSearchClient_7 that deals with 7. It makes it look like the simply
>>> named ElasticSearchClient is generic and will work with any version. This
>>> is what we did with the PublishKafka* processors and it has only caused
>>> confusion.
>>>
>>> Sent from my iPhone
>>>
>>> > On Jun 10, 2018, at 8:20 AM, Mike Thomsen 
>>> wrote:
>>> >
>>> > I think types are required in 5.X and optional in 6.X. By 8.X types are
>>> > going to be firmly eliminated, so that's at least one area of research
>>> that
>>> > needs to be done I suppose.
>>> >
>>> > On Sun, Jun 10, 2018 at 7:53 AM Sivaprasanna >> >
>>> > wrote:
>>> >
>>> >> I feel if the upgrade from 5.x to 6.x client doesn't introduce any
>>> breaking
>>> >> changes, we can continue with the name that we are having now.
>>> >>
>>> >> -
>>> >> Sivaprasanna
>>> >>
>>> >> On Sun, Jun 10, 2018 at 5:16 PM, Mike Thomsen 
>>> >> wrote:
>>> >>
>>> >>> The current implementation uses the last stable release of the 5.X
>>> client
>>> >>> from Elastic, and 6.X is already mature so we'll need to be able to
>>> have
>>> >>> room to create a new implementation copy that uses that client if
>>> there
>>> >> are
>>> >>> things we have to change between them. So does it make sense to throw
>>> in
>>> >> a
>>> >>> new ticket to rename the service to something that indicates that this
>>> >>> implementation is officially for 5.X? As of 1.6 I think only one
>>> >> processor,
>>> >>> JsonQueryElasticSearch, uses it so not many uses would likely be
>>> impacted
>>> >>> yet.
>>> >>>
>>> >>> Thanks,
>>> >>>
>>> >>> Mike
>>> >>>
>>> >>
>>>
>>

Re: Should the elastic search client service impl get renamed before 1.7?

[Mike Thomsen]

Should mention that I did not check the L for the ES 6.X version. That
needs to be reviewed as part of the PR.

On Mon, Jun 11, 2018 at 7:54 AM Mike Thomsen  wrote:

> https://github.com/apache/nifi/pull/2782
>
> Renamed the service and added a clone for 6.X. The integration tests use a
> Maven ES plugin that downloads ES, bootstraps it with test data and runs
> the integration tests against that. The 6.X version uses the latest stable
> build (6.2.4) and passed all integration tests.
>
> On Sun, Jun 10, 2018 at 11:35 AM Mark Payne  wrote:
>
>> I think it’s a good idea to go ahead and rename it. Even if 6.x doesn’t
>> make breaking changes, 7.x may. And I’d rather not have an
>> ElasticSearchClient that works for 5.x and 6.x and then an
>> ElasticSearchClient_7 that deals with 7. It makes it look like the simply
>> named ElasticSearchClient is generic and will work with any version. This
>> is what we did with the PublishKafka* processors and it has only caused
>> confusion.
>>
>> Sent from my iPhone
>>
>> > On Jun 10, 2018, at 8:20 AM, Mike Thomsen 
>> wrote:
>> >
>> > I think types are required in 5.X and optional in 6.X. By 8.X types are
>> > going to be firmly eliminated, so that's at least one area of research
>> that
>> > needs to be done I suppose.
>> >
>> > On Sun, Jun 10, 2018 at 7:53 AM Sivaprasanna > >
>> > wrote:
>> >
>> >> I feel if the upgrade from 5.x to 6.x client doesn't introduce any
>> breaking
>> >> changes, we can continue with the name that we are having now.
>> >>
>> >> -
>> >> Sivaprasanna
>> >>
>> >> On Sun, Jun 10, 2018 at 5:16 PM, Mike Thomsen 
>> >> wrote:
>> >>
>> >>> The current implementation uses the last stable release of the 5.X
>> client
>> >>> from Elastic, and 6.X is already mature so we'll need to be able to
>> have
>> >>> room to create a new implementation copy that uses that client if
>> there
>> >> are
>> >>> things we have to change between them. So does it make sense to throw
>> in
>> >> a
>> >>> new ticket to rename the service to something that indicates that this
>> >>> implementation is officially for 5.X? As of 1.6 I think only one
>> >> processor,
>> >>> JsonQueryElasticSearch, uses it so not many uses would likely be
>> impacted
>> >>> yet.
>> >>>
>> >>> Thanks,
>> >>>
>> >>> Mike
>> >>>
>> >>
>>
>

Re: Should the elastic search client service impl get renamed before 1.7?

[Mike Thomsen]

https://github.com/apache/nifi/pull/2782

Renamed the service and added a clone for 6.X. The integration tests use a
Maven ES plugin that downloads ES, bootstraps it with test data and runs
the integration tests against that. The 6.X version uses the latest stable
build (6.2.4) and passed all integration tests.

On Sun, Jun 10, 2018 at 11:35 AM Mark Payne  wrote:

> I think it’s a good idea to go ahead and rename it. Even if 6.x doesn’t
> make breaking changes, 7.x may. And I’d rather not have an
> ElasticSearchClient that works for 5.x and 6.x and then an
> ElasticSearchClient_7 that deals with 7. It makes it look like the simply
> named ElasticSearchClient is generic and will work with any version. This
> is what we did with the PublishKafka* processors and it has only caused
> confusion.
>
> Sent from my iPhone
>
> > On Jun 10, 2018, at 8:20 AM, Mike Thomsen 
> wrote:
> >
> > I think types are required in 5.X and optional in 6.X. By 8.X types are
> > going to be firmly eliminated, so that's at least one area of research
> that
> > needs to be done I suppose.
> >
> > On Sun, Jun 10, 2018 at 7:53 AM Sivaprasanna 
> > wrote:
> >
> >> I feel if the upgrade from 5.x to 6.x client doesn't introduce any
> breaking
> >> changes, we can continue with the name that we are having now.
> >>
> >> -
> >> Sivaprasanna
> >>
> >> On Sun, Jun 10, 2018 at 5:16 PM, Mike Thomsen 
> >> wrote:
> >>
> >>> The current implementation uses the last stable release of the 5.X
> client
> >>> from Elastic, and 6.X is already mature so we'll need to be able to
> have
> >>> room to create a new implementation copy that uses that client if there
> >> are
> >>> things we have to change between them. So does it make sense to throw
> in
> >> a
> >>> new ticket to rename the service to something that indicates that this
> >>> implementation is officially for 5.X? As of 1.6 I think only one
> >> processor,
> >>> JsonQueryElasticSearch, uses it so not many uses would likely be
> impacted
> >>> yet.
> >>>
> >>> Thanks,
> >>>
> >>> Mike
> >>>
> >>
>

Re: Why "ExecuteSQL" processor serving DELETE and UPDATE SQL queries.

[Mike Thomsen]

If you really need to restrict the commands that a particular user can
execute, it should be done via the database users that NiFi's services are
using. In 1.7, there's a new database service coming along that wraps two
or more database pools. It could probably be used in scenarios like this to
associate a particular pool with a particular limited database user account.

On Mon, Jun 11, 2018 at 3:13 AM Sivaprasanna 
wrote:

> I understand that. I linked that Jira to initiate a discussion (so that the
> team can pitch in their thoughts) on to have necessary changes done on
> ExecuteSQL to enable support for both SELECT and DELETE operations and make
> the necessary documentation changes, explaining that this processor
> supports SELECT and DELETE.
>
> For your case of restricting to accept "SELECT" statement alone:
> IMHO, this sounds like a case specific requirement and it may not be
> possible or seem logical to enforce this restriction for the whole user
> base so you can have two things:
>
>- If you have authorization management tools like Apache Ranger, you can
>have the restriction by having policies that allows only certain people
> to
>do DELETE operations so even if someone using NiFi tries to delete and
> that
>person doesn't have the necessary privileges, it will fail. This is
> kinda
>complex solution and the changes are externalized to NiFi
>- The simple solution is to customize the ExecuteSQL to accept only
>SELECT statements.
>
> Thanks,
> Sivaprasanna
>
> On Mon, Jun 11, 2018 at 12:19 PM, Brajendra Mishra <
> brajendra_mis...@persistent.com> wrote:
>
> > Hi Sivaprasanna,
> >
> > Thanks for prompt response.
> > Mentioned Jira defect (https://issues.apache.org/jira/browse/NIFI-4843)
> > talks about, to support delete queries through ExecuteSQL processor, but
> in
> > our case it works fine for 'Delete' as well as 'Update' queries.
> >
> > IN documentation is mentioned only about catering 'Select' SQL queries:
> >
> > "SQL select query: The SQL select query to execute. The query can be
> > empty, a constant value, or built from attributes using Expression
> > Language. If this property is specified, it will be used regardless of
> the
> > content of incoming flowfiles. If this property is empty, the content of
> > the incoming flow file is expected to contain a valid SQL select query,
> to
> > be issued by the processor to the database. Note that Expression Language
> > is not evaluated for flow file contents.
> > Supports Expression Language: true"
> >
> > Is there a way to restrict user to allow only select statement/queries
> > through expression language?
> >
> > Brajendra Mishra
> > Persistent Systems Ltd.
> >
> > -Original Message-
> > From: Sivaprasanna 
> > Sent: Monday, June 11, 2018 11:55 AM
> > To: dev@nifi.apache.org
> > Subject: Re: Why "ExecuteSQL" processor serving DELETE and UPDATE SQL
> > queries.
> >
> > Brajendra,
> >
> > As you have said, even though the documentation for ExecuteSQL mentions,
> > "It is meant to execute SELECT", it ultimately accepts and executes DML
> > commands. Looking at the code, there are no way of restricting it to
> accept
> > & execute SELECT query only. There is this Jira NIFI-4843 <
> > https://issues.apache.org/jira/browse/NIFI-4843> which mentions
> something
> > similar. Either we can do one thing: Make ExecuteSQL support two types of
> > operation "SELECT" and "DELETE" and be it exposed as a property and in
> the
> > code we do the check and perform the execution. Thoughts?
> >
> > Thanks,
> > Sivaprasanna
> >
> > On Mon, Jun 11, 2018 at 11:43 AM, Brajendra Mishra <
> > brajendra_mis...@persistent.com> wrote:
> >
> > > Hi Team,
> > >
> > >
> > >
> > > We are currently using ExecuteSQL processor in our flow.
> > >
> > > As per the documentation, ExecuteSQL processor should only be worked
> > > for SELECT queries, but it is working for other SQL commands as well
> > > for DELETE and UPDATE queries.
> > >
> > >
> > >
> > > In our current flow implementation we want to restrict user to execute
> > > only SELECT queries. How we can achieve this requirement?
> > >
> > >
> > >
> > > For your reference, here I have attached screen prints of ‘ExecuteSQL’
> > > processor:
> > >
> > >
> > >
> > > Brajendra Mishra
> > >
> > > Persistent Systems Ltd.
> > >
> > >
> > > DISCLAIMER
> > > ==
> > > This e-mail may contain privileged and confidential information which
> > > is the property of Persistent Systems Ltd. It is intended only for the
> > > use of the individual or entity to which it is addressed. If you are
> > > not the intended recipient, you are not authorized to read, retain,
> > > copy, print, distribute or use this message. If you have received this
> > > communication in error, please notify the sender and delete all copies
> > of this message.
> > > Persistent Systems Ltd. does not accept any liability for virus
> > > infected mails.
> > >
> > DISCLAIMER
> > ==
> > This e-mail may contain 

Re: [DISCUSS] Release candidate timeframe for Apache NiFi 1.7.0

[Otto Fowler]

I would really love to get a review for NIFI-5022 for this release.  Please let 
me know if there is a way to make this more likely.



On June 5, 2018 at 15:09:50, Andy LoPresto (alopre...@apache.org) wrote:

Sending out an update. I am using the following search query to take a look at 
what is still outstanding for 1.7.0. Currently I have visibility on the 
following issues:

project = "Apache NiFi" AND (fixVersion = "1.7.0" OR (status = "Patch 
Available" AND updated > '2018-05-28')) AND status in (Open, "In Progress", 
"Reopened", "Patch Available")

Patch Available (i.e. committers, please review these):
* NIFI-5241 When calculating stats for components, use synchronized methods 
instead of atomic variables — Mark Payne
* NIFI-5200 Nested ProcessSession.read resulting in outer stream being closed — 
Mark Payne
* NIFI-5209 Remove toolkit migration without password functionality — Andy 
LoPresto
* NIFI-5166 Create deep learning classification and regression processor — Mans 
Singh (Marked as In Progress but there is a PR with some comments)
* NIFI-5226 Implement a Record API based PutInfluxDB processor — Unassigned 
(Marked as Open but there is a PR with some comments)
* NIFI-5102 MarkLogic DB Processors — Unassigned (Marked as Open but there is a 
PR with some comments)
* NIFI-5268 JoltTransformJSON specification property fails with EL — Koji 
Kawamura
* NIFI-5266 PutElasticsearchHttp processors should sanitize parameters — Matt 
Burgess
* NIFI-5263 Address auditing of Controller Service Referencing Components — 
Matt Gilman
* NIFI-5260 Regression in nifi-processor-bundle-archetype — Bryan Bende
* NIFI-5257 Expand Couchbase Server integration as a cache storage — Koji 
Kawamura
* NIFI-5237 Wrong redirect from login behind a context path when using OpenID 
authentication — Matt Gilman
* NIFI-5200 Nested ProcessSession.read resulting in outer stream being closed — 
Mark Payne
* NIFI-5192 Allow expression language in ‘Schema File’ property for ValidateXML 
processor — Unassigned
* NIFI-5059 MongoDBLookupService should be able to determine a schema or have 
one provided — Mike Thomsen
* NIFI-5054 NiFi Couchbase Processors do not support User Authentication — Koji 
Kawamura
* NIFI-5022 Create an AWS Gateway Web API version of InvokeHTTP — Otto Fowler
* NIFI-4930 Nar-Dependency-Version - timestamped snapshot version problem — 
Bryan Bende
* NIFI-1321 Support appending files in PutFile — Unassigned

Reopened (i.e. Jeff, Mike, what do you need from the community to get these 
in?):
* NIFI-5145 MockPropertyValue.evaluateExpressionLanguage(FlowFile) cannot 
handle null inputs — Mike Thomsen
* NIFI-5175 NiFi built with Java 1.8 needs to run on Java 1.9 — Jeff Storck

Open (i.e. can someone please do an s/2017/2018/ on the docs?):
* NIFI-5106 Update docs to reflect 2018 where applicable — Aldrin Piri

If anyone has other work that they are in the process of doing, please reply 
here and update the Jira appropriately. I’ll send out another summary on 
Friday. Thanks. 

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

On Jun 1, 2018, at 10:28 AM, Andy LoPresto  wrote:

Hi folks,

It’s been a little bit since 1.6.0 was released, and there have been a ton of 
new features and bug fixes that have made it in to master. With all this great 
work, I think it’s a good time to consider the next release. I volunteer to RM 
for this one. 

I’m thinking the next couple weeks are a good timeframe, and I wanted to put 
this out there early so anyone who is working on something they want in can be 
aware of the approaching release and communicate on the list to keep everyone 
apprised. Thanks for all your hard work. 

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69




signature.asc
Description: Message signed with OpenPGP using AMPGpg

Re: Why "ExecuteSQL" processor serving DELETE and UPDATE SQL queries.

[Sivaprasanna]

I understand that. I linked that Jira to initiate a discussion (so that the
team can pitch in their thoughts) on to have necessary changes done on
ExecuteSQL to enable support for both SELECT and DELETE operations and make
the necessary documentation changes, explaining that this processor
supports SELECT and DELETE.

For your case of restricting to accept "SELECT" statement alone:
IMHO, this sounds like a case specific requirement and it may not be
possible or seem logical to enforce this restriction for the whole user
base so you can have two things:

   - If you have authorization management tools like Apache Ranger, you can
   have the restriction by having policies that allows only certain people to
   do DELETE operations so even if someone using NiFi tries to delete and that
   person doesn't have the necessary privileges, it will fail. This is kinda
   complex solution and the changes are externalized to NiFi
   - The simple solution is to customize the ExecuteSQL to accept only
   SELECT statements.

Thanks,
Sivaprasanna

On Mon, Jun 11, 2018 at 12:19 PM, Brajendra Mishra <
brajendra_mis...@persistent.com> wrote:

> Hi Sivaprasanna,
>
> Thanks for prompt response.
> Mentioned Jira defect (https://issues.apache.org/jira/browse/NIFI-4843)
> talks about, to support delete queries through ExecuteSQL processor, but in
> our case it works fine for 'Delete' as well as 'Update' queries.
>
> IN documentation is mentioned only about catering 'Select' SQL queries:
>
> "SQL select query: The SQL select query to execute. The query can be
> empty, a constant value, or built from attributes using Expression
> Language. If this property is specified, it will be used regardless of the
> content of incoming flowfiles. If this property is empty, the content of
> the incoming flow file is expected to contain a valid SQL select query, to
> be issued by the processor to the database. Note that Expression Language
> is not evaluated for flow file contents.
> Supports Expression Language: true"
>
> Is there a way to restrict user to allow only select statement/queries
> through expression language?
>
> Brajendra Mishra
> Persistent Systems Ltd.
>
> -Original Message-
> From: Sivaprasanna 
> Sent: Monday, June 11, 2018 11:55 AM
> To: dev@nifi.apache.org
> Subject: Re: Why "ExecuteSQL" processor serving DELETE and UPDATE SQL
> queries.
>
> Brajendra,
>
> As you have said, even though the documentation for ExecuteSQL mentions,
> "It is meant to execute SELECT", it ultimately accepts and executes DML
> commands. Looking at the code, there are no way of restricting it to accept
> & execute SELECT query only. There is this Jira NIFI-4843 <
> https://issues.apache.org/jira/browse/NIFI-4843> which mentions something
> similar. Either we can do one thing: Make ExecuteSQL support two types of
> operation "SELECT" and "DELETE" and be it exposed as a property and in the
> code we do the check and perform the execution. Thoughts?
>
> Thanks,
> Sivaprasanna
>
> On Mon, Jun 11, 2018 at 11:43 AM, Brajendra Mishra <
> brajendra_mis...@persistent.com> wrote:
>
> > Hi Team,
> >
> >
> >
> > We are currently using ExecuteSQL processor in our flow.
> >
> > As per the documentation, ExecuteSQL processor should only be worked
> > for SELECT queries, but it is working for other SQL commands as well
> > for DELETE and UPDATE queries.
> >
> >
> >
> > In our current flow implementation we want to restrict user to execute
> > only SELECT queries. How we can achieve this requirement?
> >
> >
> >
> > For your reference, here I have attached screen prints of ‘ExecuteSQL’
> > processor:
> >
> >
> >
> > Brajendra Mishra
> >
> > Persistent Systems Ltd.
> >
> >
> > DISCLAIMER
> > ==
> > This e-mail may contain privileged and confidential information which
> > is the property of Persistent Systems Ltd. It is intended only for the
> > use of the individual or entity to which it is addressed. If you are
> > not the intended recipient, you are not authorized to read, retain,
> > copy, print, distribute or use this message. If you have received this
> > communication in error, please notify the sender and delete all copies
> of this message.
> > Persistent Systems Ltd. does not accept any liability for virus
> > infected mails.
> >
> DISCLAIMER
> ==
> This e-mail may contain privileged and confidential information which is
> the property of Persistent Systems Ltd. It is intended only for the use of
> the individual or entity to which it is addressed. If you are not the
> intended recipient, you are not authorized to read, retain, copy, print,
> distribute or use this message. If you have received this communication in
> error, please notify the sender and delete all copies of this message.
> Persistent Systems Ltd. does not accept any liability for virus infected
> mails.
>

RE: Why "ExecuteSQL" processor serving DELETE and UPDATE SQL queries.

[Brajendra Mishra]

Hi Sivaprasanna, 

Thanks for prompt response. 
Mentioned Jira defect (https://issues.apache.org/jira/browse/NIFI-4843) talks 
about, to support delete queries through ExecuteSQL processor, but in our case 
it works fine for 'Delete' as well as 'Update' queries.

IN documentation is mentioned only about catering 'Select' SQL queries:

"SQL select query: The SQL select query to execute. The query can be empty, a 
constant value, or built from attributes using Expression Language. If this 
property is specified, it will be used regardless of the content of incoming 
flowfiles. If this property is empty, the content of the incoming flow file is 
expected to contain a valid SQL select query, to be issued by the processor to 
the database. Note that Expression Language is not evaluated for flow file 
contents.
Supports Expression Language: true"

Is there a way to restrict user to allow only select statement/queries through 
expression language?

Brajendra Mishra
Persistent Systems Ltd.

-Original Message-
From: Sivaprasanna  
Sent: Monday, June 11, 2018 11:55 AM
To: dev@nifi.apache.org
Subject: Re: Why "ExecuteSQL" processor serving DELETE and UPDATE SQL queries.

Brajendra,

As you have said, even though the documentation for ExecuteSQL mentions, "It is 
meant to execute SELECT", it ultimately accepts and executes DML commands. 
Looking at the code, there are no way of restricting it to accept & execute 
SELECT query only. There is this Jira NIFI-4843 
 which mentions something 
similar. Either we can do one thing: Make ExecuteSQL support two types of 
operation "SELECT" and "DELETE" and be it exposed as a property and in the code 
we do the check and perform the execution. Thoughts?

Thanks,
Sivaprasanna

On Mon, Jun 11, 2018 at 11:43 AM, Brajendra Mishra < 
brajendra_mis...@persistent.com> wrote:

> Hi Team,
>
>
>
> We are currently using ExecuteSQL processor in our flow.
>
> As per the documentation, ExecuteSQL processor should only be worked 
> for SELECT queries, but it is working for other SQL commands as well 
> for DELETE and UPDATE queries.
>
>
>
> In our current flow implementation we want to restrict user to execute 
> only SELECT queries. How we can achieve this requirement?
>
>
>
> For your reference, here I have attached screen prints of ‘ExecuteSQL’
> processor:
>
>
>
> Brajendra Mishra
>
> Persistent Systems Ltd.
>
>
> DISCLAIMER
> ==
> This e-mail may contain privileged and confidential information which 
> is the property of Persistent Systems Ltd. It is intended only for the 
> use of the individual or entity to which it is addressed. If you are 
> not the intended recipient, you are not authorized to read, retain, 
> copy, print, distribute or use this message. If you have received this 
> communication in error, please notify the sender and delete all copies of 
> this message.
> Persistent Systems Ltd. does not accept any liability for virus 
> infected mails.
>
DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Persistent Systems Ltd. It is intended only for the use of the 
individual or entity to which it is addressed. If you are not the intended 
recipient, you are not authorized to read, retain, copy, print, distribute or 
use this message. If you have received this communication in error, please 
notify the sender and delete all copies of this message. Persistent Systems 
Ltd. does not accept any liability for virus infected mails.

Re: Why "ExecuteSQL" processor serving DELETE and UPDATE SQL queries.

[Sivaprasanna]

Brajendra,

As you have said, even though the documentation for ExecuteSQL mentions,
"It is meant to execute SELECT", it ultimately accepts and executes DML
commands. Looking at the code, there are no way of restricting it to accept
& execute SELECT query only. There is this Jira NIFI-4843
 which mentions something
similar. Either we can do one thing: Make ExecuteSQL support two types of
operation "SELECT" and "DELETE" and be it exposed as a property and in the
code we do the check and perform the execution. Thoughts?

Thanks,
Sivaprasanna

On Mon, Jun 11, 2018 at 11:43 AM, Brajendra Mishra <
brajendra_mis...@persistent.com> wrote:

> Hi Team,
>
>
>
> We are currently using ExecuteSQL processor in our flow.
>
> As per the documentation, ExecuteSQL processor should only be worked for
> SELECT queries, but it is working for other SQL commands as well for DELETE
> and UPDATE queries.
>
>
>
> In our current flow implementation we want to restrict user to execute
> only SELECT queries. How we can achieve this requirement?
>
>
>
> For your reference, here I have attached screen prints of ‘ExecuteSQL’
> processor:
>
>
>
> Brajendra Mishra
>
> Persistent Systems Ltd.
>
>
> DISCLAIMER
> ==
> This e-mail may contain privileged and confidential information which is
> the property of Persistent Systems Ltd. It is intended only for the use of
> the individual or entity to which it is addressed. If you are not the
> intended recipient, you are not authorized to read, retain, copy, print,
> distribute or use this message. If you have received this communication in
> error, please notify the sender and delete all copies of this message.
> Persistent Systems Ltd. does not accept any liability for virus infected
> mails.
>

Why "ExecuteSQL" processor serving DELETE and UPDATE SQL queries.

[Brajendra Mishra]

Hi Team,

We are currently using ExecuteSQL processor in our flow.
As per the documentation, ExecuteSQL processor should only be worked for SELECT 
queries, but it is working for other SQL commands as well for DELETE and UPDATE 
queries.

In our current flow implementation we want to restrict user to execute only 
SELECT queries. How we can achieve this requirement?

For your reference, here I have attached screen prints of 'ExecuteSQL' 
processor:
[cid:image001.jpg@01D40179.6CF43000]

[cid:image002.png@01D40179.6CF43000]
Brajendra Mishra
Persistent Systems Ltd.

DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Persistent Systems Ltd. It is intended only for the use of the 
individual or entity to which it is addressed. If you are not the intended 
recipient, you are not authorized to read, retain, copy, print, distribute or 
use this message. If you have received this communication in error, please 
notify the sender and delete all copies of this message. Persistent Systems 
Ltd. does not accept any liability for virus infected mails.