date:20170410


[ 
https://issues.apache.org/jira/browse/NIFI-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963848#comment-15963848
 ] 

Joseph Witt commented on NIFI-3690:
---

here are the jars the nifi-gcp-nar includes:

aopalliance-1.0.jar   
google-oauth-client-appengine-1.21.0.jar
api-common-0.0.2.jar  
google-oauth-client-servlet-1.21.0.jar
auto-value-1.2.jar
grpc-auth-1.0.1.jar
bcpkix-jdk15on-1.55.jar   
grpc-context-1.0.1.jar
bcprov-jdk15on-1.55.jar   
grpc-core-1.0.1.jar
commons-codec-1.10.jar
grpc-google-cloud-error-reporting-v1beta1-0.1.3.jar
commons-fileupload-1.3.1.jar  
grpc-google-cloud-language-v1-0.1.3.jar
commons-io-2.5.jar
grpc-google-cloud-logging-v2-0.1.4.jar
commons-lang3-3.4.jar 
grpc-google-cloud-monitoring-v3-0.1.3.jar
commons-logging-1.2.jar   
grpc-google-cloud-pubsub-v1-0.1.3.jar
datastore-v1-proto-client-1.3.0.jar   
grpc-google-cloud-speech-v1beta1-0.1.3.jar
datastore-v1-protos-1.3.0.jar 
grpc-google-cloud-trace-v1-0.1.3.jar
gax-0.0.25.jar
grpc-google-cloud-vision-v1-0.1.3.jar
google-api-client-1.22.0.jar  
grpc-google-common-protos-0.1.3.jar
google-api-client-appengine-1.21.0.jar
grpc-google-iam-v1-0.1.3.jar
google-api-client-servlet-1.21.0.jar  
grpc-netty-1.0.1.jar
google-api-services-bigquery-v2-rev330-1.22.0.jar 
grpc-protobuf-1.0.1.jar
google-api-services-cloudresourcemanager-v1beta1-rev10-1.21.0.jar 
grpc-protobuf-lite-1.0.1.jar
google-api-services-compute-v1-rev103-1.21.0.jar  
grpc-stub-1.0.1.jar
google-api-services-dns-v1-rev7-1.21.0.jargson-2.7.jar
google-api-services-storage-v1-rev85-1.22.0.jar   guava-18.0.jar
google-api-services-translate-v2-rev47-1.22.0.jar guice-4.0.jar
google-auth-library-credentials-0.6.0.jar 
httpclient-4.4.1.jar
google-auth-library-oauth2-http-0.6.0.jar 
httpcore-4.4.1.jar
google-cloud-0.8.0.jar
jackson-core-2.1.3.jar
google-cloud-bigquery-0.8.0-beta.jar  
jackson-core-asl-1.9.11.jar
google-cloud-compute-0.8.0.jar
javax.inject-1.jar
google-cloud-core-0.8.0.jar   
jdo2-api-2.3-eb.jar
google-cloud-datastore-0.8.0-beta.jar 
joda-time-2.8.2.jar
google-cloud-dns-0.8.0.jar
json-20151123.jar
google-cloud-errorreporting-0.8.0.jar 
jsr305-1.3.9.jar
google-cloud-language-0.8.0.jar   
netty-buffer-4.1.3.Final.jar
google-cloud-logging-0.8.0-beta.jar   
netty-codec-4.1.3.Final.jar
google-cloud-monitoring-0.8.0.jar 
netty-codec-http-4.1.3.Final.jar
google-cloud-pubsub-0.8.0.jar 
netty-codec-http2-4.1.3.Final.jar
google-cloud-resourcemanager-0.8.0.jar
netty-common-4.1.3.Final.jar
google-cloud-speech-0.8.0.jar 
netty-handler-4.1.3.Final.jar
google-cloud-storage-0.8.0-beta.jar   
netty-resolver-4.1.3.Final.jar
google-cloud-trace-0.8.0.jar  
netty-tcnative-boringssl-static-1.1.33.Fork19.jar
google-cloud-translate-0.8.0.jar  
netty-transport-4.1.3.Final.jar
google-cloud-vision-0.8.0.jar 
nifi-gcp-processors-1.2.0-SNAPSHOT.jar
google-http-client-1.19.0.jar 
nifi-processor-utils-1.2.0-SNAPSHOT.jar
google-http-client-appengine-1.21.0.jar   
nifi-security-utils-1.2.0-SNAPSHOT.jar
google-http-client-jackson-1.21.0.jar 
nifi-utils-1.2.0-SNAPSHOT.jar
google-http-client-jackson2-1.19.0.jar
protobuf-java-3.0.0.jar
google-http-client-jdo-1.21.0.jar 
protobuf-java-util-3.0.0.jar
google-http-client-protobuf-1.20.0.jar
servlet-api-2.5.jar
google-oauth-client-1.21.0.jar
transaction-api-1.1.jar

> nifi-gcp-bundle contains undeclared (L) dependencies
>

[jira] [Created] (NIFI-3690) nifi-gcp-bundle contains undeclared (L) dependencies

Joseph Witt created NIFI-3690:
-

 Summary: nifi-gcp-bundle contains undeclared (L) dependencies
 Key: NIFI-3690
 URL: https://issues.apache.org/jira/browse/NIFI-3690
 Project: Apache NiFi
  Issue Type: Bug
  Components: Extensions
Reporter: Joseph Witt
Priority: Blocker
 Fix For: 1.2.0


[~gene-telligentdata] [~jameswing] In working NIFI-3089 I found the 
nifi-gcp-nar. It contained a transitive dependency on org.json library which is 
Category X for apache so must be removed.  I've got that address in NIFI-3089 
and will have a PR for that soon.  

However, in looking at the L for the nar itself there are a ton of 
dependencies in the nar and they're not referenced in the nar's L  We must 
resolve this before nifi-gcp-nar can be released w/NiFi.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3089) Eliminate any usage of json.org Cat-X License


[ 
https://issues.apache.org/jira/browse/NIFI-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963836#comment-15963836
 ] 

Joseph Witt commented on NIFI-3089:
---

This comes in transitively via twitter4j-hbc -> twitter4j.  It is a source 
dependency in twitter4j therefore Dunnings alternative binary cleanroom 
implementation is not an option.  We will need to remove the twitter processor 
until an alternative can be provided.

Also, we have to address this one that snuck in:
  
./nar/extensions/nifi-gcp-nar-1.2.0-SNAPSHOT.nar-unpacked/META-INF/bundled-dependencies/json-20151123.jar
For that Ted Dunnings alternative is likely sufficient:
  https://mvnrepository.com/artifact/com.tdunning/json/1.8

> Eliminate any usage of json.org Cat-X License
> -
>
> Key: NIFI-3089
> URL: https://issues.apache.org/jira/browse/NIFI-3089
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Witt
>Priority: Blocker
> Fix For: 1.2.0
>
>
> In NIFI-2991 we did the bulk of the work to eliminate Cat-X usage of the 
> json.org library.
> In a thread on 23 Nov 2016 VP Legal of the ASF approved pre-existing usage of 
> the json.org library provided sufficient NOTICE of usage existed (we have 
> that) until April 2017.
> We must ensure twitter4j changes their usage of this by then or we must find 
> an alternative or we must remove this processor/nar again.
> Will place this on the 1.2 release but it might be later than that.  
> Ultimately this must be done by April.
> Please do NOT remove the fix version.  We must ensure we don't lose track of 
> this issue.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (NIFI-3089) Eliminate any usage of json.org Cat-X License


 [ 
https://issues.apache.org/jira/browse/NIFI-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt reassigned NIFI-3089:
-

Assignee: Joseph Witt

> Eliminate any usage of json.org Cat-X License
> -
>
> Key: NIFI-3089
> URL: https://issues.apache.org/jira/browse/NIFI-3089
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Witt
>Assignee: Joseph Witt
>Priority: Blocker
> Fix For: 1.2.0
>
>
> In NIFI-2991 we did the bulk of the work to eliminate Cat-X usage of the 
> json.org library.
> In a thread on 23 Nov 2016 VP Legal of the ASF approved pre-existing usage of 
> the json.org library provided sufficient NOTICE of usage existed (we have 
> that) until April 2017.
> We must ensure twitter4j changes their usage of this by then or we must find 
> an alternative or we must remove this processor/nar again.
> Will place this on the 1.2 release but it might be later than that.  
> Ultimately this must be done by April.
> Please do NOT remove the fix version.  We must ensure we don't lose track of 
> this issue.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3089) Eliminate any usage of json.org Cat-X License


 [ 
https://issues.apache.org/jira/browse/NIFI-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-3089:
--
Summary: Eliminate any usage of json.org Cat-X License  (was: Ensure 
GetTwitter processor does not leverage json.org Cat-X License)

> Eliminate any usage of json.org Cat-X License
> -
>
> Key: NIFI-3089
> URL: https://issues.apache.org/jira/browse/NIFI-3089
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joseph Witt
>Priority: Blocker
> Fix For: 1.2.0
>
>
> In NIFI-2991 we did the bulk of the work to eliminate Cat-X usage of the 
> json.org library.
> In a thread on 23 Nov 2016 VP Legal of the ASF approved pre-existing usage of 
> the json.org library provided sufficient NOTICE of usage existed (we have 
> that) until April 2017.
> We must ensure twitter4j changes their usage of this by then or we must find 
> an alternative or we must remove this processor/nar again.
> Will place this on the 1.2 release but it might be later than that.  
> Ultimately this must be done by April.
> Please do NOT remove the fix version.  We must ensure we don't lose track of 
> this issue.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3633) Create an HttpNotifierService implementation of the Bootstrap Notification Service

2017-04-10 Thread Joseph Percivall (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963825#comment-15963825
 ] 

Joseph Percivall commented on NIFI-3633:


Thanks for the review [~joewitt], very much appreciated.

> Create an HttpNotifierService implementation of the Bootstrap Notification 
> Service
> --
>
> Key: NIFI-3633
> URL: https://issues.apache.org/jira/browse/NIFI-3633
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joseph Percivall
>Assignee: Joseph Percivall
> Fix For: 1.2.0
>
>
> In order to notify "when the NiFi bootstrap starts or stops NiFi, or detects 
> that it has died unexpectedly" the bootstrap has the notification service but 
> currently there is only one implementation, the EmailNotifierService[1]. 
> An HttpNotifierService should be created in order to more generically post 
> notifications. 
> [1] 
> https://github.com/mcgilman/nifi/blob/3a7ddc6a35f143c1d4a735226fe7467a83d04583/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/notification/email/EmailNotificationService.java#L47-L47



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3633) Create an HttpNotifierService implementation of the Bootstrap Notification Service


 [ 
https://issues.apache.org/jira/browse/NIFI-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-3633:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

+1 merged to master.  Thanks

> Create an HttpNotifierService implementation of the Bootstrap Notification 
> Service
> --
>
> Key: NIFI-3633
> URL: https://issues.apache.org/jira/browse/NIFI-3633
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joseph Percivall
>Assignee: Joseph Witt
> Fix For: 1.2.0
>
>
> In order to notify "when the NiFi bootstrap starts or stops NiFi, or detects 
> that it has died unexpectedly" the bootstrap has the notification service but 
> currently there is only one implementation, the EmailNotifierService[1]. 
> An HttpNotifierService should be created in order to more generically post 
> notifications. 
> [1] 
> https://github.com/mcgilman/nifi/blob/3a7ddc6a35f143c1d4a735226fe7467a83d04583/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/notification/email/EmailNotificationService.java#L47-L47



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (NIFI-3633) Create an HttpNotifierService implementation of the Bootstrap Notification Service


 [ 
https://issues.apache.org/jira/browse/NIFI-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt reassigned NIFI-3633:
-

Assignee: Joseph Percivall  (was: Joseph Witt)

> Create an HttpNotifierService implementation of the Bootstrap Notification 
> Service
> --
>
> Key: NIFI-3633
> URL: https://issues.apache.org/jira/browse/NIFI-3633
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joseph Percivall
>Assignee: Joseph Percivall
> Fix For: 1.2.0
>
>
> In order to notify "when the NiFi bootstrap starts or stops NiFi, or detects 
> that it has died unexpectedly" the bootstrap has the notification service but 
> currently there is only one implementation, the EmailNotifierService[1]. 
> An HttpNotifierService should be created in order to more generically post 
> notifications. 
> [1] 
> https://github.com/mcgilman/nifi/blob/3a7ddc6a35f143c1d4a735226fe7467a83d04583/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/notification/email/EmailNotificationService.java#L47-L47



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3633) Create an HttpNotifierService implementation of the Bootstrap Notification Service

2017-04-10 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963821#comment-15963821
 ] 

ASF GitHub Bot commented on NIFI-3633:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1610


> Create an HttpNotifierService implementation of the Bootstrap Notification 
> Service
> --
>
> Key: NIFI-3633
> URL: https://issues.apache.org/jira/browse/NIFI-3633
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joseph Percivall
>Assignee: Joseph Witt
> Fix For: 1.2.0
>
>
> In order to notify "when the NiFi bootstrap starts or stops NiFi, or detects 
> that it has died unexpectedly" the bootstrap has the notification service but 
> currently there is only one implementation, the EmailNotifierService[1]. 
> An HttpNotifierService should be created in order to more generically post 
> notifications. 
> [1] 
> https://github.com/mcgilman/nifi/blob/3a7ddc6a35f143c1d4a735226fe7467a83d04583/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/notification/email/EmailNotificationService.java#L47-L47



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3633) Create an HttpNotifierService implementation of the Bootstrap Notification Service


[ 
https://issues.apache.org/jira/browse/NIFI-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963820#comment-15963820
 ] 

ASF subversion and git services commented on NIFI-3633:
---

Commit 5419891fe760746f3fd4bb5c397d9a43b79f8281 in nifi's branch 
refs/heads/master from Joe Percivall
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=5419891 ]

NIFI-3633 This closes #1610. Adding HttpNotificationService.

Signed-off-by: joewitt 


> Create an HttpNotifierService implementation of the Bootstrap Notification 
> Service
> --
>
> Key: NIFI-3633
> URL: https://issues.apache.org/jira/browse/NIFI-3633
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joseph Percivall
>Assignee: Joseph Witt
> Fix For: 1.2.0
>
>
> In order to notify "when the NiFi bootstrap starts or stops NiFi, or detects 
> that it has died unexpectedly" the bootstrap has the notification service but 
> currently there is only one implementation, the EmailNotifierService[1]. 
> An HttpNotifierService should be created in order to more generically post 
> notifications. 
> [1] 
> https://github.com/mcgilman/nifi/blob/3a7ddc6a35f143c1d4a735226fe7467a83d04583/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/notification/email/EmailNotificationService.java#L47-L47



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi pull request #1610: NIFI-3633 Adding HttpNotificationService.

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1610


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3679) Add Content Viewer for Avro


[ 
https://issues.apache.org/jira/browse/NIFI-3679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963802#comment-15963802
 ] 

ASF GitHub Bot commented on NIFI-3679:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1655


> Add Content Viewer for Avro
> ---
>
> Key: NIFI-3679
> URL: https://issues.apache.org/jira/browse/NIFI-3679
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Joseph Witt
> Fix For: 1.2.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3679) Add Content Viewer for Avro


 [ 
https://issues.apache.org/jira/browse/NIFI-3679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-3679:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Add Content Viewer for Avro
> ---
>
> Key: NIFI-3679
> URL: https://issues.apache.org/jira/browse/NIFI-3679
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Joseph Witt
> Fix For: 1.2.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi pull request #1655: NIFI-3679: Added Avro Content Viewer

2017-04-10 Thread ASF subversion and git services (JIRA)

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1655


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3679) Add Content Viewer for Avro


[ 
https://issues.apache.org/jira/browse/NIFI-3679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963801#comment-15963801
 ] 

ASF subversion and git services commented on NIFI-3679:
---

Commit 8a5398eba771f0cb7733eb552c91e072c1ed40f8 in nifi's branch 
refs/heads/master from [~markap14]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=8a5398e ]

NIFI-3679: This closes #1655. Added Avro Content Viewer

Signed-off-by: joewitt 


> Add Content Viewer for Avro
> ---
>
> Key: NIFI-3679
> URL: https://issues.apache.org/jira/browse/NIFI-3679
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Joseph Witt
> Fix For: 1.2.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi-site pull request #17: MiNiFi-260: Create a "Getting Started" Guide for...

2017-04-10 Thread randerzander

Github user randerzander commented on a diff in the pull request:

https://github.com/apache/nifi-site/pull/17#discussion_r110802366
  
--- Diff: src/pages/markdown/minifi/getting-started.md ---
@@ -0,0 +1,122 @@
+---
+title: Apache NiFi MiNiFi: Getting Started
+---
+
+# Getting started with MiNiFi
+
+This page explains how to configure and deploy MiNiFi agents.
+
+The Java agent is able to run most of [NiFi's available 
processors](http://nifi.apache.org/docs.html), but is a larger binary 
distribution (49MB) and consumes greater system resources (24MB max JVM 
heapsize by default). If you need maximum flexibility to make routing and 
processing decisions at your data's point of origin, the Java agent is a good 
fit.
+
+The C++ agent is a smaller binary (3.2MB), consumes low system memory 
(about 5MB at idle) but has [a limited subset of 
processors](https://github.com/apache/nifi-minifi-cpp#caveats). If your primary 
concern is gathering and pushing data to downstream consumers and minimizing 
system impact, the C++ agent is a good fit.
+
+
+1. Install the appropriate OS level dependencies:
+
+   MiNiFi Java:
+
+  - Java 1.8+
+
+   MiNiFi C++:
--- End diff --

Two items of note here, both in the context of the as-is, currently 
released minifi-cpp binary:

1. I tested in Docker "FROM centos", which is CentOS 7.3.1611, and only 
installed the two listed yum packages, after which the binary runs 
successfully. Perhaps the boost libraries were installed as deps of the leveldb 
package?

2. I tested the binary in Docker "FROM ubuntu" which is Ubuntu 16.04.2. 
While libxml2 may not be used as a dependency, attempting to run the binary 
without having it installed yielded errors about missing that library. That 
said, the binary still segfaults, albeit not with errors about libxml2

I was under the impression that docs are intended to reflect how released 
artifacts work. If I'm mistaken, I'll update the PR to show how things work 
from a recent master branch build. Which would you suggest?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-site pull request #17: MiNiFi-260: Create a "Getting Started" Guide for...

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-site/pull/17#discussion_r110800166
  
--- Diff: src/pages/markdown/minifi/getting-started.md ---
@@ -0,0 +1,122 @@
+---
+title: Apache NiFi MiNiFi: Getting Started
+---
+
+# Getting started with MiNiFi
+
+This page explains how to configure and deploy MiNiFi agents.
+
+The Java agent is able to run most of [NiFi's available 
processors](http://nifi.apache.org/docs.html), but is a larger binary 
distribution (49MB) and consumes greater system resources (24MB max JVM 
heapsize by default). If you need maximum flexibility to make routing and 
processing decisions at your data's point of origin, the Java agent is a good 
fit.
+
+The C++ agent is a smaller binary (3.2MB), consumes low system memory 
(about 5MB at idle) but has [a limited subset of 
processors](https://github.com/apache/nifi-minifi-cpp#caveats). If your primary 
concern is gathering and pushing data to downstream consumers and minimizing 
system impact, the C++ agent is a good fit.
+
+
+1. Install the appropriate OS level dependencies:
+
+   MiNiFi Java:
+
+  - Java 1.8+
+
+   MiNiFi C++:
--- End diff --

I did indeed conflate those, apologies.  We do have some considerations in 
the current build in terms of dynamically linked items though, such as the 
libboost items.  I do not believe the current build makes these static.

libxml2 should also no longer be needed.  I noticed this was a remnant in 
our Travis build and created MINIFI-264 to clean that up.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi-cpp pull request #77: MINIFI-264 Remove libxml2 from travis buil...

GitHub user apiri opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/77

MINIFI-264 Remove libxml2 from travis build as it is no longer needed.

MINIFI-264 Remove libxml2 from travis build as it is no longer needed.

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
 in the commit message?

- [X] Does your PR title start with MINIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [X] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [X] Is your initial contribution a single, squashed commit?


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apiri/nifi-minifi-cpp remove-libxml

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/77.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #77


commit 0c0e8f4bdefbeb8c7e4a8629c8ad6fecb11634c9
Author: Aldrin Piri 
Date:   2017-04-11T01:31:12Z

MINIFI-264 Remove libxml2 from travis build as it is no longer needed.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-site pull request #17: MiNiFi-260: Create a "Getting Started" Guide for...

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-site/pull/17#discussion_r110799319
  
--- Diff: src/includes/minifi/minifi-topbar.hbs ---
@@ -26,16 +26,17 @@
 
 Documentation
 
-FAQ
+Getting Started
 Administrator's Guide
 Converter Toolkit
+FAQ
 https://cwiki.apache.org/confluence/display/MiNiFi;>Wiki
 
 
 
-Download
--- End diff --

Great, thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (NIFI-3689) TestWriteAheadStorePartition frequently causes travis failures

Andre F de Miranda created NIFI-3689:


 Summary: TestWriteAheadStorePartition frequently causes travis 
failures
 Key: NIFI-3689
 URL: https://issues.apache.org/jira/browse/NIFI-3689
 Project: Apache NiFi
  Issue Type: Bug
Reporter: Andre F de Miranda


Apologies if this happens to be a duplicate (I did a search but could not find 
anything similar)

I notice a number of travis builds seem to fail during the execution of 
TestWriteAheadStorePartition

https://travis-ci.org/apache/nifi/jobs/220619894
https://travis-ci.org/apache/nifi/jobs/220671755
https://travis-ci.org/apache/nifi/jobs/217435194



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3600) Improve logging and relationship routing for failures in DeleteHDFS


[ 
https://issues.apache.org/jira/browse/NIFI-3600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963658#comment-15963658
 ] 

ASF GitHub Bot commented on NIFI-3600:
--

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1595
  
:rofl: 


> Improve logging and relationship routing for failures in DeleteHDFS
> ---
>
> Key: NIFI-3600
> URL: https://issues.apache.org/jira/browse/NIFI-3600
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Jeremy Dyer
>Assignee: Jeremy Dyer
>
> Currently if DeleteHDFS attempts to delete a file and does not have 
> permissions to delete that file the flowfile session is rolled back and 
> penalized. This causes the incoming connection into DeleteHDFS to build up 
> over time and not process at all.
> This enhancement would add a new relationship to DeleteHDFS where files 
> failures to delete a file strictly tied to invalid permissions would be 
> routed given the end user the opportunity to react to those invalid 
> permissions through another approach.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi issue #1595: NIFI-3600 Improve logging and relationship routing for fai...

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1595
  
:rofl: 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3688) Create extended groovy scripting processor


[ 
https://issues.apache.org/jira/browse/NIFI-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963549#comment-15963549
 ] 

ASF GitHub Bot commented on NIFI-3688:
--

Github user dlukyanov commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1662#discussion_r110773203
  
--- Diff: nifi-nar-bundles/nifi-groovyx-bundle/nifi-groovyx-nar/pom.xml ---
@@ -0,0 +1,44 @@
+
+
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+4.0.0
+
+
+org.apache.nifi
+nifi-groovyx-bundle
+1.2.0-SNAPSHOT
+
+
+nifi-groovyx-nar
+nar
+
+true
--- End diff --

no reason. i'll fix it.


> Create extended groovy scripting processor
> --
>
> Key: NIFI-3688
> URL: https://issues.apache.org/jira/browse/NIFI-3688
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Dmitry Lukyanov
>Priority: Minor
>
> The idea is to simplify groovy scripting.
> Main targets:
> - to be compatible with existing groovy scripting
> - simplify read/write attributes
> - simplify read/write content
> - avoid closure casting to nifi types like `StreamCallback`
> - simplify and provide visibility when accessing to controller services from 
> script



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi pull request #1662: NIFI-3688 Extended Groovy Nifi Processor

2017-04-10 Thread dlukyanov

Github user dlukyanov commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1662#discussion_r110773203
  
--- Diff: nifi-nar-bundles/nifi-groovyx-bundle/nifi-groovyx-nar/pom.xml ---
@@ -0,0 +1,44 @@
+
+
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+4.0.0
+
+
+org.apache.nifi
+nifi-groovyx-bundle
+1.2.0-SNAPSHOT
+
+
+nifi-groovyx-nar
+nar
+
+true
--- End diff --

no reason. i'll fix it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3688) Create extended groovy scripting processor


[ 
https://issues.apache.org/jira/browse/NIFI-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963547#comment-15963547
 ] 

ASF GitHub Bot commented on NIFI-3688:
--

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1662#discussion_r110773031
  
--- Diff: nifi-nar-bundles/nifi-groovyx-bundle/nifi-groovyx-nar/pom.xml ---
@@ -0,0 +1,44 @@
+
+
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+4.0.0
+
+
+org.apache.nifi
+nifi-groovyx-bundle
+1.2.0-SNAPSHOT
+
+
+nifi-groovyx-nar
+nar
+
+true
--- End diff --

I don't think this is compatible with our coding style guide. Is there a 
reason Javadoc needs to be skipped?


> Create extended groovy scripting processor
> --
>
> Key: NIFI-3688
> URL: https://issues.apache.org/jira/browse/NIFI-3688
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Dmitry Lukyanov
>Priority: Minor
>
> The idea is to simplify groovy scripting.
> Main targets:
> - to be compatible with existing groovy scripting
> - simplify read/write attributes
> - simplify read/write content
> - avoid closure casting to nifi types like `StreamCallback`
> - simplify and provide visibility when accessing to controller services from 
> script



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi pull request #1662: NIFI-3688 Extended Groovy Nifi Processor

2017-04-10 Thread alopresto

Github user alopresto commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1662#discussion_r110773031
  
--- Diff: nifi-nar-bundles/nifi-groovyx-bundle/nifi-groovyx-nar/pom.xml ---
@@ -0,0 +1,44 @@
+
+
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+4.0.0
+
+
+org.apache.nifi
+nifi-groovyx-bundle
+1.2.0-SNAPSHOT
+
+
+nifi-groovyx-nar
+nar
+
+true
--- End diff --

I don't think this is compatible with our coding style guide. Is there a 
reason Javadoc needs to be skipped?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1662: NIFI-3688 Extended Groovy Nifi Processor

2017-04-10 Thread dlukyanov

GitHub user dlukyanov opened a pull request:

https://github.com/apache/nifi/pull/1662

NIFI-3688 Extended Groovy Nifi Processor 

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [YES] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?
 in last one only

- [YES] Does your PR title start with NIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [YES] Has your PR been rebased against the latest commit within the 
target branch (typically master)?

- [YES?] Is your initial contribution a single, squashed commit?
 
### For code changes:
- [YES] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [NO] Have you written or updated unit tests to verify your changes?
 no changes done to existing code. but no tests to groovyx processor.
- [YES] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [YES?] If applicable, have you updated the LICENSE file, including the 
main LICENSE file under nifi-assembly?
- [YES?] If applicable, have you updated the NOTICE file, including the 
main NOTICE file found under nifi-assembly?
- [NO] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?
 not applicable?
### For documentation related changes:
- [YES] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dlukyanov/nifi master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1662.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1662


commit 279e0502deef66d2a2934a211430aac52d3a5b28
Author: dlukyanov 
Date:   2017-04-02T07:55:22Z

groovyx initial version

commit 0bb6f9f970b186b6ec018d70e5afa68c9345b7c1
Author: dlukyanov 
Date:   2017-04-10T21:09:54Z

https://issues.apache.org/jira/browse/NIFI-3688

commit 8d142dd629fee4f42434f754c16c897edfa08447
Author: dlukyanov 
Date:   2017-04-10T21:22:38Z

NIFI-3688 license update to ASF

commit cd26cb1f3f2ebcbdc2ebda8904c424c62f2e5298
Author: dlukyanov 
Date:   2017-04-10T21:24:30Z

NIFI-3688 add groovyx dependency




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3688) Create extended groovy scripting processor


[ 
https://issues.apache.org/jira/browse/NIFI-3688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963504#comment-15963504
 ] 

ASF GitHub Bot commented on NIFI-3688:
--

GitHub user dlukyanov opened a pull request:

https://github.com/apache/nifi/pull/1662

NIFI-3688 Extended Groovy Nifi Processor 

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [YES] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?
 in last one only

- [YES] Does your PR title start with NIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [YES] Has your PR been rebased against the latest commit within the 
target branch (typically master)?

- [YES?] Is your initial contribution a single, squashed commit?
 
### For code changes:
- [YES] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [NO] Have you written or updated unit tests to verify your changes?
 no changes done to existing code. but no tests to groovyx processor.
- [YES] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [YES?] If applicable, have you updated the LICENSE file, including the 
main LICENSE file under nifi-assembly?
- [YES?] If applicable, have you updated the NOTICE file, including the 
main NOTICE file found under nifi-assembly?
- [NO] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?
 not applicable?
### For documentation related changes:
- [YES] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dlukyanov/nifi master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1662.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1662


commit 279e0502deef66d2a2934a211430aac52d3a5b28
Author: dlukyanov 
Date:   2017-04-02T07:55:22Z

groovyx initial version

commit 0bb6f9f970b186b6ec018d70e5afa68c9345b7c1
Author: dlukyanov 
Date:   2017-04-10T21:09:54Z

https://issues.apache.org/jira/browse/NIFI-3688

commit 8d142dd629fee4f42434f754c16c897edfa08447
Author: dlukyanov 
Date:   2017-04-10T21:22:38Z

NIFI-3688 license update to ASF

commit cd26cb1f3f2ebcbdc2ebda8904c424c62f2e5298
Author: dlukyanov 
Date:   2017-04-10T21:24:30Z

NIFI-3688 add groovyx dependency




> Create extended groovy scripting processor
> --
>
> Key: NIFI-3688
> URL: https://issues.apache.org/jira/browse/NIFI-3688
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Dmitry Lukyanov
>Priority: Minor
>
> The idea is to simplify groovy scripting.
> Main targets:
> - to be compatible with existing groovy scripting
> - simplify read/write attributes
> - simplify read/write content
> - avoid closure casting to nifi types like `StreamCallback`
> - simplify and provide visibility when accessing to controller services from 
> script



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3688) Create extended groovy scripting processor

2017-04-10 Thread Dmitry Lukyanov (JIRA)

Dmitry Lukyanov created NIFI-3688:
-

 Summary: Create extended groovy scripting processor
 Key: NIFI-3688
 URL: https://issues.apache.org/jira/browse/NIFI-3688
 Project: Apache NiFi
  Issue Type: New Feature
  Components: Extensions
Reporter: Dmitry Lukyanov
Priority: Minor


The idea is to simplify groovy scripting.
Main targets:
- to be compatible with existing groovy scripting
- simplify read/write attributes
- simplify read/write content
- avoid closure casting to nifi types like `StreamCallback`
- simplify and provide visibility when accessing to controller services from 
script




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3687) Failed to connect node to cluster due to: java.lang.NullPointerException

2017-04-10 Thread Joseph Niemiec (JIRA)

Joseph Niemiec created NIFI-3687:


 Summary: Failed to connect node to cluster due to: 
java.lang.NullPointerException
 Key: NIFI-3687
 URL: https://issues.apache.org/jira/browse/NIFI-3687
 Project: Apache NiFi
  Issue Type: Bug
 Environment: SUSE 11 SP3, NIFI 1.1.0.2.1.2.0-10, Java jdk1.8.0_60, 2 
node clustered Nifi
Reporter: Joseph Niemiec


We had a server fill its root directory, we stopped NiFi, moved the 
repositories to mounts with space, updated the configs and now we have a single 
node that will not join the cluster. There are no major errors but we do run 
into some WARNS talking about an NPE before Jetty explodes. 


2017-04-10 14:55:57,293 ERROR [main] o.a.nifi.controller.StandardFlowService 
Failed to load flow from cluster due to: 
org.apache.nifi.cluster.ConnectionException: Failed to connect node to cluster 
due to: java.lang.NullPointerException
org.apache.nifi.cluster.ConnectionException: Failed to connect node to cluster 
due to: java.lang.NullPointerException
at 
org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:901)
 ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at 
org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:493)
 ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:831) 
[nifi-jetty-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at org.apache.nifi.NiFi.(NiFi.java:156) 
[nifi-runtime-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at org.apache.nifi.NiFi.main(NiFi.java:262) 
[nifi-runtime-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
Caused by: java.lang.NullPointerException: null
at 
org.apache.nifi.controller.repository.SchemaRepositoryRecordSerde.deserializeRecord(SchemaRepositoryRecordSerde.java:119)
 ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at 
org.apache.nifi.controller.repository.SchemaRepositoryRecordSerde.deserializeEdit(SchemaRepositoryRecordSerde.java:109)
 ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at 
org.apache.nifi.controller.repository.SchemaRepositoryRecordSerde.deserializeEdit(SchemaRepositoryRecordSerde.java:46)
 ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at 
org.wali.MinimalLockingWriteAheadLog$Partition.recoverNextTransaction(MinimalLockingWriteAheadLog.java:1072)
 ~[nifi-write-ahead-log-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at 
org.wali.MinimalLockingWriteAheadLog.recoverFromEdits(MinimalLockingWriteAheadLog.java:459)
 ~[nifi-write-ahead-log-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at 
org.wali.MinimalLockingWriteAheadLog.recoverRecords(MinimalLockingWriteAheadLog.java:301)
 ~[nifi-write-ahead-log-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at 
org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.loadFlowFiles(WriteAheadFlowFileRepository.java:346)
 ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at 
org.apache.nifi.controller.FlowController.initializeFlow(FlowController.java:699)
 ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at 
org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:701)
 ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
at 
org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:872)
 ~[nifi-framework-core-1.1.0.2.1.2.0-10.jar:1.1.0.2.1.2.0-10]
... 4 common frames omitted
2017-04-10 14:55:57,293 INFO [main] o.a.n.c.c.node.NodeClusterCoordinator 
hpce2r01n11.abc.com:9090 requested disconnection from cluster due to 
org.apache.nifi.cluster.ConnectionException: Failed to connect node to cluster 
due to: java.lang.NullPointerException
2017-04-10 14:55:57,293 INFO [main] o.a.n.c.c.node.NodeClusterCoordinator 
Status of hpce2r01n11.abc.com:9090 changed from 
NodeConnectionStatus[nodeId=hpce2r01n11.abc.com:9090, state=CONNECTING, 
updateId=60] to NodeConnectionStatus[nodeId=hpce2r01n11.abc.com:9090, 
state=DISCONNECTED, Disconnect Code=Node Failed to Startup Properly, Disconnect 
Reason=org.apache.nifi.cluster.ConnectionException: Failed to connect node to 
cluster due to: java.lang.NullPointerException, updateId=60]
2017-04-10 14:55:57,293 DEBUG [main] o.a.n.c.c.node.NodeClusterCoordinator 
State of cluster nodes is now 
{hpce2r01n12.abc.com:9090=NodeConnectionStatus[nodeId=hpce2r01n12.abc.com:9090, 
state=CONNECTED, updateId=54], 
hpce2r01n11.abc.com:9090=NodeConnectionStatus[nodeId=hpce2r01n11.abc.com:9090, 
state=DISCONNECTED, Disconnect Code=Node Failed to Startup Properly, Disconnect 
Reason=org.apache.nifi.cluster.ConnectionException: Failed to connect node to 
cluster due to: java.lang.NullPointerException, updateId=60]}
2017-04-10 14:55:57,294 DEBUG [main]

[jira] [Commented] (NIFI-3686) EOFException on swap in causes tight loop in polling for flowfiles

2017-04-10 Thread Michael Moser (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-3686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963355#comment-15963355
 ] 

Michael Moser commented on NIFI-3686:
-

Note: I didn't encounter this on a production system, I simulated this 
happening by truncating a swap file while NiFi was not running.

I have a simple code patch to StandardFlowFileQueue that will remove the swap 
contents from the swapQueue if the swap summary is valid.  This fixes the user 
experience by logging the EOFException ERROR to the nifi-app.log, then the 
queue size goes to 0 and the processor reading from this queue is not 
triggered.  On the next NiFi restart, if the corrupt swap file is still there, 
the EOFException ERROR happens again.  I'm not sure this is the desired 
approach, though.

[~markap14] if you can ponder this, please let me know if I should submit this 
as a PR or if it should be resolved in another way.  Thanks!

> EOFException on swap in causes tight loop in polling for flowfiles
> --
>
> Key: NIFI-3686
> URL: https://issues.apache.org/jira/browse/NIFI-3686
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.1.1
>Reporter: Michael Moser
>
> If flowfile_repository partition fills 100% while swapping files out to a new 
> swap file, then this swap file becomes corrupt (partially written).  When 
> NiFi tries to swap this file in, EOFException happens and we get following 
> ERROR, which is nice.
> 2017-04-10 18:02:58,855 ERROR [Timer-Driven Process Thread-3] 
> o.a.n.controller.StandardFlowFileQueue Failed to swap in FlowFiles from Swap 
> File 
> /local/mwmoser/nifi-1.2.0-SNAPSHOT/./flowfile_repository/swap/1491574631605-2840b630-57fc-4f49-615b-0b37d77bec66-5dbc0ad0-921c-483e-a05d-5c65d014fa48.swap;
>  Swap File appears to be corrupt!
> However, once all other dataflow stops, the queue now shows 1 flowfiles 
> in it.  The processor reading from this queue constantly has its onTrigger() 
> called, and session.get() polls the queue and gets 0 files returned.  This 
> happens in a tight loop, with no other errors.
> To a user it appears that the processor is doing lots of work but just not 
> processing those 1 files.  The error message above only appears once in 
> the nifi-app.log, so you don't see anything wrong if you tail the log. 
>  When you restart NiFi, the error message above appears again, but the user 
> experience of 1 files not processing remains.
> The new SchemaSwapDeserializer does not (and perhaps cannot) implement the 
> IncompleteSwapFileException that the old SimpleSwapDeserializer does.  So, 
> reading a swap file is currently all-or-nothing.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3686) EOFException on swap in causes tight loop in polling for flowfiles

2017-04-10 Thread Michael Moser (JIRA)

Michael Moser created NIFI-3686:
---

Summary: EOFException on swap in causes tight loop in polling for
flowfiles
Key: NIFI-3686
URL: https://issues.apache.org/jira/browse/NIFI-3686
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 1.1.1
Reporter: Michael Moser

If flowfile_repository partition fills 100% while swapping files out to a new
swap file, then this swap file becomes corrupt (partially written). When NiFi
tries to swap this file in, EOFException happens and we get following ERROR,
which is nice.

2017-04-10 18:02:58,855 ERROR [Timer-Driven Process Thread-3]
o.a.n.controller.StandardFlowFileQueue Failed to swap in FlowFiles from Swap
File
/local/mwmoser/nifi-1.2.0-SNAPSHOT/./flowfile_repository/swap/1491574631605-2840b630-57fc-4f49-615b-0b37d77bec66-5dbc0ad0-921c-483e-a05d-5c65d014fa48.swap;
Swap File appears to be corrupt!

However, once all other dataflow stops, the queue now shows 1 flowfiles in
it. The processor reading from this queue constantly has its onTrigger()
called, and session.get() polls the queue and gets 0 files returned. This
happens in a tight loop, with no other errors.

To a user it appears that the processor is doing lots of work but just not
processing those 1 files. The error message above only appears once in the
nifi-app.log, so you don't see anything wrong if you tail the log.
When you restart NiFi, the error message above appears again, but the user
experience of 1 files not processing remains.

The new SchemaSwapDeserializer does not (and perhaps cannot) implement the
IncompleteSwapFileException that the old SimpleSwapDeserializer does. So,
reading a swap file is currently all-or-nothing.

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi pull request #1659: [NIFI-3035] use api supported .entries() to iterate...

2017-04-10 Thread scottyaslan

Github user scottyaslan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1659#discussion_r110735389
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-canvas-utils.js
 ---
@@ -412,9 +412,9 @@
 
 // create object whose keys are the parameter name and the 
values are the parameter values
 var paramsObject = {};
-params.forEach(function (v, k) {
-paramsObject[k] = v;
-});
+for(var pair of params.entries()) {
--- End diff --

Good catch! As it turns out we dont need to loop over the 
URLSearchParams...we can just use the .toString(). I have updated this PR to 
reflect this change. Please test and let me know if it works for you.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-3414) Implement an EnforceOrder processor

2017-04-10 Thread Matt Burgess (JIRA)


 [ 
https://issues.apache.org/jira/browse/NIFI-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess updated NIFI-3414:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Implement an EnforceOrder processor
> ---
>
> Key: NIFI-3414
> URL: https://issues.apache.org/jira/browse/NIFI-3414
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Assignee: Koji Kawamura
> Fix For: 1.2.0
>
>
> For some flows, it is imperative that the flow files are processed in a 
> certain order.  The PriorityAttributePrioritizer can be used on a connection 
> to ensure that flow files going through that connection are in priority 
> order, but depending on error-handling, branching, and other flow designs, it 
> is possible for flow files to get out-of-order.
> I propose an EnforceOrder processor, which would be single-threaded and have 
> (at a minimum) the following properties:
> 1) Order Attribute: This would be the name of a flow file attribute from 
> which the current value will be retrieved.
> 2) Initial Value: This property specifies an initial value for the order. The 
> processor is stateful, however, so this property is only used when there is 
> no entry in the state map for current value.
> The processor would store the Initial Value into the state map (if no state 
> map entry exists), then for each incoming flow file, it checks the value in 
> the Order Attribute against the current value.  If the attribute value 
> matches the current value, the flow file is transferred to the "success" 
> relationship, and the current value is incremented in the state map. If the 
> attribute value does not match the current value, the session will be rolled 
> back.
> Using this processor, along with a PriorityAttributePrioritizer on the 
> incoming connection, will allow for out-of-order flow files to have a sort of 
> "barrier", thereby guaranteeing that flow files transferred to the "success" 
> relationship are in the specified order.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3414) Implement an EnforceOrder processor

2017-04-10 Thread Matt Burgess (JIRA)


 [ 
https://issues.apache.org/jira/browse/NIFI-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess updated NIFI-3414:
---
Fix Version/s: 1.2.0

> Implement an EnforceOrder processor
> ---
>
> Key: NIFI-3414
> URL: https://issues.apache.org/jira/browse/NIFI-3414
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Assignee: Koji Kawamura
> Fix For: 1.2.0
>
>
> For some flows, it is imperative that the flow files are processed in a 
> certain order.  The PriorityAttributePrioritizer can be used on a connection 
> to ensure that flow files going through that connection are in priority 
> order, but depending on error-handling, branching, and other flow designs, it 
> is possible for flow files to get out-of-order.
> I propose an EnforceOrder processor, which would be single-threaded and have 
> (at a minimum) the following properties:
> 1) Order Attribute: This would be the name of a flow file attribute from 
> which the current value will be retrieved.
> 2) Initial Value: This property specifies an initial value for the order. The 
> processor is stateful, however, so this property is only used when there is 
> no entry in the state map for current value.
> The processor would store the Initial Value into the state map (if no state 
> map entry exists), then for each incoming flow file, it checks the value in 
> the Order Attribute against the current value.  If the attribute value 
> matches the current value, the flow file is transferred to the "success" 
> relationship, and the current value is incremented in the state map. If the 
> attribute value does not match the current value, the session will be rolled 
> back.
> Using this processor, along with a PriorityAttributePrioritizer on the 
> incoming connection, will allow for out-of-order flow files to have a sort of 
> "barrier", thereby guaranteeing that flow files transferred to the "success" 
> relationship are in the specified order.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3414) Implement an EnforceOrder processor

2017-04-10 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963332#comment-15963332
 ] 

ASF GitHub Bot commented on NIFI-3414:
--

Github user mattyb149 commented on the issue:

https://github.com/apache/nifi/pull/1496
  
+1 LGTM, ran with the latest, full build and a number of use cases. All 
looks great, thanks very much! Merging to master


> Implement an EnforceOrder processor
> ---
>
> Key: NIFI-3414
> URL: https://issues.apache.org/jira/browse/NIFI-3414
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Assignee: Koji Kawamura
> Fix For: 1.2.0
>
>
> For some flows, it is imperative that the flow files are processed in a 
> certain order.  The PriorityAttributePrioritizer can be used on a connection 
> to ensure that flow files going through that connection are in priority 
> order, but depending on error-handling, branching, and other flow designs, it 
> is possible for flow files to get out-of-order.
> I propose an EnforceOrder processor, which would be single-threaded and have 
> (at a minimum) the following properties:
> 1) Order Attribute: This would be the name of a flow file attribute from 
> which the current value will be retrieved.
> 2) Initial Value: This property specifies an initial value for the order. The 
> processor is stateful, however, so this property is only used when there is 
> no entry in the state map for current value.
> The processor would store the Initial Value into the state map (if no state 
> map entry exists), then for each incoming flow file, it checks the value in 
> the Order Attribute against the current value.  If the attribute value 
> matches the current value, the flow file is transferred to the "success" 
> relationship, and the current value is incremented in the state map. If the 
> attribute value does not match the current value, the session will be rolled 
> back.
> Using this processor, along with a PriorityAttributePrioritizer on the 
> incoming connection, will allow for out-of-order flow files to have a sort of 
> "barrier", thereby guaranteeing that flow files transferred to the "success" 
> relationship are in the specified order.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi issue #1496: NIFI-3414: Added EnforceOrder processor

2017-04-10 Thread mattyb149

Github user mattyb149 commented on the issue:

https://github.com/apache/nifi/pull/1496
  
+1 LGTM, ran with the latest, full build and a number of use cases. All 
looks great, thanks very much! Merging to master


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3414) Implement an EnforceOrder processor


[ 
https://issues.apache.org/jira/browse/NIFI-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963318#comment-15963318
 ] 

ASF subversion and git services commented on NIFI-3414:
---

Commit 9583ca99c1e4b2d3c43511939a2a148d20f8e1ac in nifi's branch 
refs/heads/master from [~ijokarumawak]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9583ca9 ]

NIFI-3414: Added EnforceOrder processor

Use it with FirstInFirstOutPrioritizer, it can enforce original ordering
of 'out-of-order' FlowFiles.

nifi-mock is modified to support FlowFile assertion using Prioritizer.

Signed-off-by: Matt Burgess 

NIFI-3414: Added EnforceOrder processor

Incorporated review comments, added displayNames.

Signed-off-by: Matt Burgess 

NIFI-3414: Added EnforceOrder processor

Incorporate review comments:

- Moved nifi-standard-prioritizers dependency to top level nifi/pom.xml.
- Changed default initial order from 1 to 0.
- Fixed typos.
- Use session.get(batchCount).

Signed-off-by: Matt Burgess 

NIFI-3414: Added EnforceOrder processor

When a FlowFile is transferred to success, remove attributes previously set 
when it was transferred to wait or failure.

Signed-off-by: Matt Burgess 

This closes #1496


> Implement an EnforceOrder processor
> ---
>
> Key: NIFI-3414
> URL: https://issues.apache.org/jira/browse/NIFI-3414
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Assignee: Koji Kawamura
>
> For some flows, it is imperative that the flow files are processed in a 
> certain order.  The PriorityAttributePrioritizer can be used on a connection 
> to ensure that flow files going through that connection are in priority 
> order, but depending on error-handling, branching, and other flow designs, it 
> is possible for flow files to get out-of-order.
> I propose an EnforceOrder processor, which would be single-threaded and have 
> (at a minimum) the following properties:
> 1) Order Attribute: This would be the name of a flow file attribute from 
> which the current value will be retrieved.
> 2) Initial Value: This property specifies an initial value for the order. The 
> processor is stateful, however, so this property is only used when there is 
> no entry in the state map for current value.
> The processor would store the Initial Value into the state map (if no state 
> map entry exists), then for each incoming flow file, it checks the value in 
> the Order Attribute against the current value.  If the attribute value 
> matches the current value, the flow file is transferred to the "success" 
> relationship, and the current value is incremented in the state map. If the 
> attribute value does not match the current value, the session will be rolled 
> back.
> Using this processor, along with a PriorityAttributePrioritizer on the 
> incoming connection, will allow for out-of-order flow files to have a sort of 
> "barrier", thereby guaranteeing that flow files transferred to the "success" 
> relationship are in the specified order.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3414) Implement an EnforceOrder processor

2017-04-10 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963320#comment-15963320
 ] 

ASF GitHub Bot commented on NIFI-3414:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1496


> Implement an EnforceOrder processor
> ---
>
> Key: NIFI-3414
> URL: https://issues.apache.org/jira/browse/NIFI-3414
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Assignee: Koji Kawamura
>
> For some flows, it is imperative that the flow files are processed in a 
> certain order.  The PriorityAttributePrioritizer can be used on a connection 
> to ensure that flow files going through that connection are in priority 
> order, but depending on error-handling, branching, and other flow designs, it 
> is possible for flow files to get out-of-order.
> I propose an EnforceOrder processor, which would be single-threaded and have 
> (at a minimum) the following properties:
> 1) Order Attribute: This would be the name of a flow file attribute from 
> which the current value will be retrieved.
> 2) Initial Value: This property specifies an initial value for the order. The 
> processor is stateful, however, so this property is only used when there is 
> no entry in the state map for current value.
> The processor would store the Initial Value into the state map (if no state 
> map entry exists), then for each incoming flow file, it checks the value in 
> the Order Attribute against the current value.  If the attribute value 
> matches the current value, the flow file is transferred to the "success" 
> relationship, and the current value is incremented in the state map. If the 
> attribute value does not match the current value, the session will be rolled 
> back.
> Using this processor, along with a PriorityAttributePrioritizer on the 
> incoming connection, will allow for out-of-order flow files to have a sort of 
> "barrier", thereby guaranteeing that flow files transferred to the "success" 
> relationship are in the specified order.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3414) Implement an EnforceOrder processor


[ 
https://issues.apache.org/jira/browse/NIFI-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963316#comment-15963316
 ] 

ASF subversion and git services commented on NIFI-3414:
---

Commit 9583ca99c1e4b2d3c43511939a2a148d20f8e1ac in nifi's branch 
refs/heads/master from [~ijokarumawak]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9583ca9 ]

NIFI-3414: Added EnforceOrder processor

Use it with FirstInFirstOutPrioritizer, it can enforce original ordering
of 'out-of-order' FlowFiles.

nifi-mock is modified to support FlowFile assertion using Prioritizer.

Signed-off-by: Matt Burgess 

NIFI-3414: Added EnforceOrder processor

Incorporated review comments, added displayNames.

Signed-off-by: Matt Burgess 

NIFI-3414: Added EnforceOrder processor

Incorporate review comments:

- Moved nifi-standard-prioritizers dependency to top level nifi/pom.xml.
- Changed default initial order from 1 to 0.
- Fixed typos.
- Use session.get(batchCount).

Signed-off-by: Matt Burgess 

NIFI-3414: Added EnforceOrder processor

When a FlowFile is transferred to success, remove attributes previously set 
when it was transferred to wait or failure.

Signed-off-by: Matt Burgess 

This closes #1496


> Implement an EnforceOrder processor
> ---
>
> Key: NIFI-3414
> URL: https://issues.apache.org/jira/browse/NIFI-3414
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Assignee: Koji Kawamura
>
> For some flows, it is imperative that the flow files are processed in a 
> certain order.  The PriorityAttributePrioritizer can be used on a connection 
> to ensure that flow files going through that connection are in priority 
> order, but depending on error-handling, branching, and other flow designs, it 
> is possible for flow files to get out-of-order.
> I propose an EnforceOrder processor, which would be single-threaded and have 
> (at a minimum) the following properties:
> 1) Order Attribute: This would be the name of a flow file attribute from 
> which the current value will be retrieved.
> 2) Initial Value: This property specifies an initial value for the order. The 
> processor is stateful, however, so this property is only used when there is 
> no entry in the state map for current value.
> The processor would store the Initial Value into the state map (if no state 
> map entry exists), then for each incoming flow file, it checks the value in 
> the Order Attribute against the current value.  If the attribute value 
> matches the current value, the flow file is transferred to the "success" 
> relationship, and the current value is incremented in the state map. If the 
> attribute value does not match the current value, the session will be rolled 
> back.
> Using this processor, along with a PriorityAttributePrioritizer on the 
> incoming connection, will allow for out-of-order flow files to have a sort of 
> "barrier", thereby guaranteeing that flow files transferred to the "success" 
> relationship are in the specified order.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3414) Implement an EnforceOrder processor

2017-04-10 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963319#comment-15963319
 ] 

ASF subversion and git services commented on NIFI-3414:
---

Commit 9583ca99c1e4b2d3c43511939a2a148d20f8e1ac in nifi's branch 
refs/heads/master from [~ijokarumawak]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9583ca9 ]

NIFI-3414: Added EnforceOrder processor

Use it with FirstInFirstOutPrioritizer, it can enforce original ordering
of 'out-of-order' FlowFiles.

nifi-mock is modified to support FlowFile assertion using Prioritizer.

Signed-off-by: Matt Burgess 

NIFI-3414: Added EnforceOrder processor

Incorporated review comments, added displayNames.

Signed-off-by: Matt Burgess 

NIFI-3414: Added EnforceOrder processor

Incorporate review comments:

- Moved nifi-standard-prioritizers dependency to top level nifi/pom.xml.
- Changed default initial order from 1 to 0.
- Fixed typos.
- Use session.get(batchCount).

Signed-off-by: Matt Burgess 

NIFI-3414: Added EnforceOrder processor

When a FlowFile is transferred to success, remove attributes previously set 
when it was transferred to wait or failure.

Signed-off-by: Matt Burgess 

This closes #1496


> Implement an EnforceOrder processor
> ---
>
> Key: NIFI-3414
> URL: https://issues.apache.org/jira/browse/NIFI-3414
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Assignee: Koji Kawamura
>
> For some flows, it is imperative that the flow files are processed in a 
> certain order.  The PriorityAttributePrioritizer can be used on a connection 
> to ensure that flow files going through that connection are in priority 
> order, but depending on error-handling, branching, and other flow designs, it 
> is possible for flow files to get out-of-order.
> I propose an EnforceOrder processor, which would be single-threaded and have 
> (at a minimum) the following properties:
> 1) Order Attribute: This would be the name of a flow file attribute from 
> which the current value will be retrieved.
> 2) Initial Value: This property specifies an initial value for the order. The 
> processor is stateful, however, so this property is only used when there is 
> no entry in the state map for current value.
> The processor would store the Initial Value into the state map (if no state 
> map entry exists), then for each incoming flow file, it checks the value in 
> the Order Attribute against the current value.  If the attribute value 
> matches the current value, the flow file is transferred to the "success" 
> relationship, and the current value is incremented in the state map. If the 
> attribute value does not match the current value, the session will be rolled 
> back.
> Using this processor, along with a PriorityAttributePrioritizer on the 
> incoming connection, will allow for out-of-order flow files to have a sort of 
> "barrier", thereby guaranteeing that flow files transferred to the "success" 
> relationship are in the specified order.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3414) Implement an EnforceOrder processor

2017-04-10 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963317#comment-15963317
 ] 

ASF subversion and git services commented on NIFI-3414:
---

Commit 9583ca99c1e4b2d3c43511939a2a148d20f8e1ac in nifi's branch 
refs/heads/master from [~ijokarumawak]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9583ca9 ]

NIFI-3414: Added EnforceOrder processor

Use it with FirstInFirstOutPrioritizer, it can enforce original ordering
of 'out-of-order' FlowFiles.

nifi-mock is modified to support FlowFile assertion using Prioritizer.

Signed-off-by: Matt Burgess 

NIFI-3414: Added EnforceOrder processor

Incorporated review comments, added displayNames.

Signed-off-by: Matt Burgess 

NIFI-3414: Added EnforceOrder processor

Incorporate review comments:

- Moved nifi-standard-prioritizers dependency to top level nifi/pom.xml.
- Changed default initial order from 1 to 0.
- Fixed typos.
- Use session.get(batchCount).

Signed-off-by: Matt Burgess 

NIFI-3414: Added EnforceOrder processor

When a FlowFile is transferred to success, remove attributes previously set 
when it was transferred to wait or failure.

Signed-off-by: Matt Burgess 

This closes #1496


> Implement an EnforceOrder processor
> ---
>
> Key: NIFI-3414
> URL: https://issues.apache.org/jira/browse/NIFI-3414
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Matt Burgess
>Assignee: Koji Kawamura
>
> For some flows, it is imperative that the flow files are processed in a 
> certain order.  The PriorityAttributePrioritizer can be used on a connection 
> to ensure that flow files going through that connection are in priority 
> order, but depending on error-handling, branching, and other flow designs, it 
> is possible for flow files to get out-of-order.
> I propose an EnforceOrder processor, which would be single-threaded and have 
> (at a minimum) the following properties:
> 1) Order Attribute: This would be the name of a flow file attribute from 
> which the current value will be retrieved.
> 2) Initial Value: This property specifies an initial value for the order. The 
> processor is stateful, however, so this property is only used when there is 
> no entry in the state map for current value.
> The processor would store the Initial Value into the state map (if no state 
> map entry exists), then for each incoming flow file, it checks the value in 
> the Order Attribute against the current value.  If the attribute value 
> matches the current value, the flow file is transferred to the "success" 
> relationship, and the current value is incremented in the state map. If the 
> attribute value does not match the current value, the session will be rolled 
> back.
> Using this processor, along with a PriorityAttributePrioritizer on the 
> incoming connection, will allow for out-of-order flow files to have a sort of 
> "barrier", thereby guaranteeing that flow files transferred to the "success" 
> relationship are in the specified order.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi pull request #1496: NIFI-3414: Added EnforceOrder processor

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1496


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3600) Improve logging and relationship routing for failures in DeleteHDFS


[ 
https://issues.apache.org/jira/browse/NIFI-3600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963261#comment-15963261
 ] 

ASF GitHub Bot commented on NIFI-3600:
--

Github user jdye64 commented on the issue:

https://github.com/apache/nifi/pull/1595
  
@trixpan I tried talking it out but that didn't seem to solve the 
conflict?? Will try more forceful procedures when I get off the road and near a 
computer ;) thanks for taking a look at this!


> Improve logging and relationship routing for failures in DeleteHDFS
> ---
>
> Key: NIFI-3600
> URL: https://issues.apache.org/jira/browse/NIFI-3600
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Jeremy Dyer
>Assignee: Jeremy Dyer
>
> Currently if DeleteHDFS attempts to delete a file and does not have 
> permissions to delete that file the flowfile session is rolled back and 
> penalized. This causes the incoming connection into DeleteHDFS to build up 
> over time and not process at all.
> This enhancement would add a new relationship to DeleteHDFS where files 
> failures to delete a file strictly tied to invalid permissions would be 
> routed given the end user the opportunity to react to those invalid 
> permissions through another approach.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi issue #1595: NIFI-3600 Improve logging and relationship routing for fai...

2017-04-10 Thread jdye64

Github user jdye64 commented on the issue:

https://github.com/apache/nifi/pull/1595
  
@trixpan I tried talking it out but that didn't seem to solve the 
conflict?? Will try more forceful procedures when I get off the road and near a 
computer ;) thanks for taking a look at this!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-site pull request #17: MiNiFi-260: Create a "Getting Started" Guide for...

2017-04-10 Thread randerzander

Github user randerzander commented on a diff in the pull request:

https://github.com/apache/nifi-site/pull/17#discussion_r110705173
  
--- Diff: src/includes/minifi/minifi-topbar.hbs ---
@@ -26,16 +26,17 @@
 
 Documentation
 
-FAQ
+Getting Started
 Administrator's Guide
 Converter Toolkit
+FAQ
 https://cwiki.apache.org/confluence/display/MiNiFi;>Wiki
 
 
 
-Download
--- End diff --

I changed to Download_s_ for MiNiFi since there are 3 distinct binaries for 
different purposes. 

For NiFi you typically download only the latest binary release, but there 
are several different links. I will change NiFi's topbar to "Downloads" as well.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-site pull request #17: MiNiFi-260: Create a "Getting Started" Guide for...

2017-04-10 Thread randerzander

Github user randerzander commented on a diff in the pull request:

https://github.com/apache/nifi-site/pull/17#discussion_r110702518
  
--- Diff: src/pages/markdown/minifi/getting-started.md ---
@@ -0,0 +1,122 @@
+---
+title: Apache NiFi MiNiFi: Getting Started
+---
+
+# Getting started with MiNiFi
+
+This page explains how to configure and deploy MiNiFi agents.
+
+The Java agent is able to run most of [NiFi's available 
processors](http://nifi.apache.org/docs.html), but is a larger binary 
distribution (49MB) and consumes greater system resources (24MB max JVM 
heapsize by default). If you need maximum flexibility to make routing and 
processing decisions at your data's point of origin, the Java agent is a good 
fit.
+
+The C++ agent is a smaller binary (3.2MB), consumes low system memory 
(about 5MB at idle) but has [a limited subset of 
processors](https://github.com/apache/nifi-minifi-cpp#caveats). If your primary 
concern is gathering and pushing data to downstream consumers and minimizing 
system impact, the C++ agent is a good fit.
+
+
+1. Install the appropriate OS level dependencies:
+
+   MiNiFi Java:
+
+  - Java 1.8+
+
+   MiNiFi C++:
--- End diff --

It's only needed if you plan to do a build from source, not if you're just 
running the released binary.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3600) Improve logging and relationship routing for failures in DeleteHDFS


[ 
https://issues.apache.org/jira/browse/NIFI-3600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963113#comment-15963113
 ] 

ASF GitHub Bot commented on NIFI-3600:
--

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1595
  
@jdye64 can you please solve the conflicts?

Cheers!


> Improve logging and relationship routing for failures in DeleteHDFS
> ---
>
> Key: NIFI-3600
> URL: https://issues.apache.org/jira/browse/NIFI-3600
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Jeremy Dyer
>Assignee: Jeremy Dyer
>
> Currently if DeleteHDFS attempts to delete a file and does not have 
> permissions to delete that file the flowfile session is rolled back and 
> penalized. This causes the incoming connection into DeleteHDFS to build up 
> over time and not process at all.
> This enhancement would add a new relationship to DeleteHDFS where files 
> failures to delete a file strictly tied to invalid permissions would be 
> routed given the end user the opportunity to react to those invalid 
> permissions through another approach.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi issue #1595: NIFI-3600 Improve logging and relationship routing for fai...

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1595
  
@jdye64 can you please solve the conflicts?

Cheers!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information


[ 
https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963091#comment-15963091
 ] 

ASF GitHub Bot commented on NIFI-1613:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/293


> ConvertJSONToSQL Drops Type Information
> ---
>
> Key: NIFI-1613
> URL: https://issues.apache.org/jira/browse/NIFI-1613
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 0.4.1, 0.5.1
> Environment: Ubuntu 14.04 LTS
>Reporter: Aaron Stephens
>Assignee: Toivo Adams
>  Labels: ConvertJSONToSQL, Phoenix, SQL
>
> It appears that the ConvertJSONToSQL processor is turning Boolean (and 
> possibly Integer and Float) values into Strings.  This is okay for some 
> drivers (like PostgreSQL) which can coerce a String back into a Boolean, but 
> it causes issues for others (specifically Phoenix in my case).
> {noformat}
> org.apache.phoenix.schema.ConstraintViolationException: 
> org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type 
> mismatch. VARCHAR cannot be coerced to BOOLEAN
> at 
> org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282)
>  ~[na:na]
> at 
> org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na]
> at 
> org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442)
>  ~[na:na]
> at 
> org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166)
>  ~[na:na]
> at 
> org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166)
>  ~[na:na]
> at 
> org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) 
> ~[na:na]
> at 
> org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) 
> ~[na:na]
> at 
> org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na]
> at 
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>  ~[nifi-api-0.4.1.jar:0.4.1]
> at 
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146)
>  ~[nifi-framework-core-0.4.1.jar:0.4.1]
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139)
>  [nifi-framework-core-0.4.1.jar:0.4.1]
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
>  [nifi-framework-core-0.4.1.jar:0.4.1]
> at 
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119)
>  [nifi-framework-core-0.4.1.jar:0.4.1]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
> [na:1.7.0_79]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) 
> [na:1.7.0_79]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>  [na:1.7.0_79]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>  [na:1.7.0_79]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  [na:1.7.0_79]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  [na:1.7.0_79]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
> Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 
> (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN
> at 
> org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71)
>  ~[na:na]
> at 
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
>  ~[na:na]
> ... 20 common frames omitted
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-1922) Add support for WASB / Azure Blob Storage with HDFS processors


[ 
https://issues.apache.org/jira/browse/NIFI-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963094#comment-15963094
 ] 

ASF GitHub Bot commented on NIFI-1922:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/626


> Add support for WASB / Azure Blob Storage with HDFS processors
> --
>
> Key: NIFI-1922
> URL: https://issues.apache.org/jira/browse/NIFI-1922
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Joseph Witt
>
> https://hadoop.apache.org/docs/stable2/hadoop-azure/index.html
> Received two mailing list questions on this in the past couple weeks.
> We should look at what things we'd need to do in the processors to make this 
> possible.  Adding the dependency is one, anything else?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-2629) Hide add property "+" in Processors, Controller Services and Reporting Tasks that don't allow dynamic properties


[ 
https://issues.apache.org/jira/browse/NIFI-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963092#comment-15963092
 ] 

ASF GitHub Bot commented on NIFI-2629:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/969


> Hide add property "+" in Processors, Controller Services and Reporting Tasks 
> that don't allow dynamic properties
> 
>
> Key: NIFI-2629
> URL: https://issues.apache.org/jira/browse/NIFI-2629
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework, Core UI
>Affects Versions: 0.7.0
>Reporter: Kirk Tarou
>Priority: Minor
> Attachments: Screen Shot 2016-09-07 at 3.20.04 PM.png, Screen Shot 
> 2016-09-07 at 3.20.20 PM.png
>
>
> A few Processors, Controller Services and Reporting Tasks expect or allow 
> extra properties to be added on the 'Properties' tab in the Detail view. The 
> rest will show a "not a supported property" error.
> To avoid confusion, the new property "+" icon should be hidden if dynamic 
> properties are not allowed.
> The existence of a "DynamicProperty" annotation could be used as an indicator.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-1583) Convert JSON format to CSV


[ 
https://issues.apache.org/jira/browse/NIFI-1583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963093#comment-15963093
 ] 

ASF GitHub Bot commented on NIFI-1583:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/254


> Convert JSON format to CSV
> --
>
> Key: NIFI-1583
> URL: https://issues.apache.org/jira/browse/NIFI-1583
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 0.4.1
>Reporter: Adis Cesir
>Priority: Minor
>
> Processor with the ability to flatten out multi nested JSON Objects and 
> Arrays into tabular format / CSV. An example of a use for this is the common 
> server responses and log events producing JSON which can be converted into 
> fixed delimited structure to ingest into RDBMS,HDFS/Hive and any other system 
> preferring 2 dimensional tabular formats.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi pull request #293: NIFI-1613 Initial version, try to improve conversion...

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/293


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #626: NIFI-1922 added support for HDInsight wasb file stor...

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/626


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #254: NIFI-1583 Added ConvertJSONtoCSV processor

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/254


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #969: NIFI-2629 - Hide add property "+" in Processors, Con...

2017-04-10 Thread ASF subversion and git services (JIRA)

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/969


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3304) In the Administration Guide, there is an incorrect JAVA 8 parameter for bootstrap.conf file


[ 
https://issues.apache.org/jira/browse/NIFI-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963078#comment-15963078
 ] 

ASF subversion and git services commented on NIFI-3304:
---

Commit 2589df80ab1eb70327c6ed836f269967caee35fe in nifi's branch 
refs/heads/master from [~boardm26]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2589df8 ]

NIFI-3304 removed unneeded Java 8 codecache info from Admin Guide

This closes #1621

Signed-off-by: Andre F de Miranda 


> In the Administration Guide, there is an incorrect JAVA 8 parameter for 
> bootstrap.conf file
> ---
>
> Key: NIFI-3304
> URL: https://issues.apache.org/jira/browse/NIFI-3304
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Documentation & Website
>Reporter: David A. Wynne
>Assignee: Michael Moser
>Priority: Minor
> Fix For: 1.2.0
>
>
> In the Administration guide, section Bootstrap properties, there is a 
> paragraph about Java 8 handling of codecache, the listed properties are 
> accurate for Java version 7, but for Java version 8, one has changed
> This is correct for Java version 7:
> java.arg.8=-XX:CodeCacheFlushingMinimumFreeSpace=10m
> But for Java version 8, it should be
> java.arg.8=-XX:CodeCacheMinimumFreeSpace=10m



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3304) In the Administration Guide, there is an incorrect JAVA 8 parameter for bootstrap.conf file


 [ 
https://issues.apache.org/jira/browse/NIFI-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andre F de Miranda updated NIFI-3304:
-
   Resolution: Fixed
Fix Version/s: 1.2.0
   Status: Resolved  (was: Patch Available)

> In the Administration Guide, there is an incorrect JAVA 8 parameter for 
> bootstrap.conf file
> ---
>
> Key: NIFI-3304
> URL: https://issues.apache.org/jira/browse/NIFI-3304
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Documentation & Website
>Reporter: David A. Wynne
>Assignee: Michael Moser
>Priority: Minor
> Fix For: 1.2.0
>
>
> In the Administration guide, section Bootstrap properties, there is a 
> paragraph about Java 8 handling of codecache, the listed properties are 
> accurate for Java version 7, but for Java version 8, one has changed
> This is correct for Java version 7:
> java.arg.8=-XX:CodeCacheFlushingMinimumFreeSpace=10m
> But for Java version 8, it should be
> java.arg.8=-XX:CodeCacheMinimumFreeSpace=10m



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3304) In the Administration Guide, there is an incorrect JAVA 8 parameter for bootstrap.conf file


[ 
https://issues.apache.org/jira/browse/NIFI-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963080#comment-15963080
 ] 

ASF GitHub Bot commented on NIFI-3304:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1621


> In the Administration Guide, there is an incorrect JAVA 8 parameter for 
> bootstrap.conf file
> ---
>
> Key: NIFI-3304
> URL: https://issues.apache.org/jira/browse/NIFI-3304
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Documentation & Website
>Reporter: David A. Wynne
>Assignee: Michael Moser
>Priority: Minor
> Fix For: 1.2.0
>
>
> In the Administration guide, section Bootstrap properties, there is a 
> paragraph about Java 8 handling of codecache, the listed properties are 
> accurate for Java version 7, but for Java version 8, one has changed
> This is correct for Java version 7:
> java.arg.8=-XX:CodeCacheFlushingMinimumFreeSpace=10m
> But for Java version 8, it should be
> java.arg.8=-XX:CodeCacheMinimumFreeSpace=10m



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi pull request #1621: NIFI-3304 removed unneeded Java 8 codecache info fr...

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1621


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2724) JMX Processor


[ 
https://issues.apache.org/jira/browse/NIFI-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963076#comment-15963076
 ] 

ASF GitHub Bot commented on NIFI-2724:
--

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1016
  
@brianburnett there's a minor conflict but I am more concerned with 
consistent failures when I run the test units?

Have you been able to build this with `-Pcontrib-check` ?


> JMX Processor
> -
>
> Key: NIFI-2724
> URL: https://issues.apache.org/jira/browse/NIFI-2724
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Affects Versions: 1.0.0
> Environment: All platforms with Java RMI support for JMX
>Reporter: Brian Burnett
>Assignee: Andre F de Miranda
>Priority: Minor
>  Labels: processor
> Attachments: 0001-NIFI-2724-New-JMX-Processor.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The JMX Processor feature addition includes only GetJMX without 
> SecurityManager capabilities at this time.  The processor in its current 
> state is capable of pulling MBean Property and Attribute values from a remote 
> RMI Server.  Each set of Mbean data is wrapped in a JSON formatted FlowFile 
> for downstream processing.  It has the ability to control content with 
> whitelist and blacklist properties.
> Possible use for this processor and the reason it was created is to help make 
> sense of Kafka server metrics.
> Will followup with a SecurityManager Context Service and PutJMX Processor.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi issue #1016: NIFI-2724 New JMX Processor

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1016
  
@brianburnett there's a minor conflict but I am more concerned with 
consistent failures when I run the test units?

Have you been able to build this with `-Pcontrib-check` ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (NIFI-3685) Azure EventHub processor tests fail

2017-04-10 Thread Jeff Storck (JIRA)


 [ 
https://issues.apache.org/jira/browse/NIFI-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Storck reassigned NIFI-3685:
-

Assignee: Jeff Storck

> Azure EventHub processor tests fail
> ---
>
> Key: NIFI-3685
> URL: https://issues.apache.org/jira/browse/NIFI-3685
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.1.1
>Reporter: Jeff Storck
>Assignee: Jeff Storck
>
> GetAzureEventHubTest fails with the following exception:
> {quote}
> ---
>  T E S T S
> ---
> Running org.apache.nifi.processors.azure.eventhub.GetAzureEventHubTest
> Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.22 sec <<< 
> FAILURE! - in org.apache.nifi.processors.azure.eventhub.GetAzureEventHubTest
> testNormalFlow(org.apache.nifi.processors.azure.eventhub.GetAzureEventHubTest)
>   Time elapsed: 0.195 sec  <<< FAILURE!
> java.lang.AssertionError: 
> org.powermock.reflect.exceptions.FieldNotFoundException: No instance field 
> named "isReceivedEvent" could be found in the class hierarchy of 
> com.microsoft.azure.eventhubs.EventData.
>   at 
> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:193)
>   at 
> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:152)
>   at 
> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:147)
>   at 
> org.apache.nifi.processors.azure.eventhub.GetAzureEventHubTest.testNormalFlow(GetAzureEventHubTest.java:108)
> Caused by: org.powermock.reflect.exceptions.FieldNotFoundException: No 
> instance field named "isReceivedEvent" could be found in the class hierarchy 
> of com.microsoft.azure.eventhubs.EventData.
>   at 
> org.powermock.reflect.internal.matcherstrategies.FieldNameMatcherStrategy.notFound(FieldNameMatcherStrategy.java:40)
>   at 
> org.powermock.reflect.internal.WhiteboxImpl.findSingleFieldUsingStrategy(WhiteboxImpl.java:492)
>   at 
> org.powermock.reflect.internal.WhiteboxImpl.findFieldInHierarchy(WhiteboxImpl.java:445)
>   at 
> org.powermock.reflect.internal.WhiteboxImpl.findFieldInHierarchy(WhiteboxImpl.java:433)
>   at 
> org.powermock.reflect.internal.WhiteboxImpl.setInternalState(WhiteboxImpl.java:294)
>   at org.powermock.reflect.Whitebox.setInternalState(Whitebox.java:174)
>   at 
> org.apache.nifi.processors.azure.eventhub.GetAzureEventHubTest$MockGetAzureEventHub.receiveEvents(GetAzureEventHubTest.java:144)
>   at 
> org.apache.nifi.processors.azure.eventhub.GetAzureEventHub.onTrigger(GetAzureEventHub.java:316)
>   at 
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>   at 
> org.apache.nifi.util.StandardProcessorTestRunner$RunProcessor.call(StandardProcessorTestRunner.java:243)
>   at 
> org.apache.nifi.util.StandardProcessorTestRunner$RunProcessor.call(StandardProcessorTestRunner.java:237)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> Running org.apache.nifi.processors.azure.eventhub.PutAzureEventHubTest
> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 sec - 
> in org.apache.nifi.processors.azure.eventhub.PutAzureEventHubTest
> Results :
> Failed tests:
>   GetAzureEventHubTest.testNormalFlow:108 
> org.powermock.reflect.exceptions.FieldNotFoundException: No instance field 
> named "isReceivedEvent" could be found in the class hierarchy of 
> com.microsoft.azure.eventhubs.EventData.
> {quote}
> The tests for the processors also need to be moved to src/test/java 
> (currently in src/test) so that the tests are detected during the build.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3685) Azure EventHub processor tests fail

2017-04-10 Thread Jeff Storck (JIRA)

Jeff Storck created NIFI-3685:
-

 Summary: Azure EventHub processor tests fail
 Key: NIFI-3685
 URL: https://issues.apache.org/jira/browse/NIFI-3685
 Project: Apache NiFi
  Issue Type: Bug
  Components: Extensions
Affects Versions: 1.1.1
Reporter: Jeff Storck


GetAzureEventHubTest fails with the following exception:
{quote}
---
 T E S T S
---
Running org.apache.nifi.processors.azure.eventhub.GetAzureEventHubTest
Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.22 sec <<< 
FAILURE! - in org.apache.nifi.processors.azure.eventhub.GetAzureEventHubTest
testNormalFlow(org.apache.nifi.processors.azure.eventhub.GetAzureEventHubTest)  
Time elapsed: 0.195 sec  <<< FAILURE!
java.lang.AssertionError: 
org.powermock.reflect.exceptions.FieldNotFoundException: No instance field 
named "isReceivedEvent" could be found in the class hierarchy of 
com.microsoft.azure.eventhubs.EventData.
at 
org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:193)
at 
org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:152)
at 
org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:147)
at 
org.apache.nifi.processors.azure.eventhub.GetAzureEventHubTest.testNormalFlow(GetAzureEventHubTest.java:108)
Caused by: org.powermock.reflect.exceptions.FieldNotFoundException: No instance 
field named "isReceivedEvent" could be found in the class hierarchy of 
com.microsoft.azure.eventhubs.EventData.
at 
org.powermock.reflect.internal.matcherstrategies.FieldNameMatcherStrategy.notFound(FieldNameMatcherStrategy.java:40)
at 
org.powermock.reflect.internal.WhiteboxImpl.findSingleFieldUsingStrategy(WhiteboxImpl.java:492)
at 
org.powermock.reflect.internal.WhiteboxImpl.findFieldInHierarchy(WhiteboxImpl.java:445)
at 
org.powermock.reflect.internal.WhiteboxImpl.findFieldInHierarchy(WhiteboxImpl.java:433)
at 
org.powermock.reflect.internal.WhiteboxImpl.setInternalState(WhiteboxImpl.java:294)
at org.powermock.reflect.Whitebox.setInternalState(Whitebox.java:174)
at 
org.apache.nifi.processors.azure.eventhub.GetAzureEventHubTest$MockGetAzureEventHub.receiveEvents(GetAzureEventHubTest.java:144)
at 
org.apache.nifi.processors.azure.eventhub.GetAzureEventHub.onTrigger(GetAzureEventHub.java:316)
at 
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at 
org.apache.nifi.util.StandardProcessorTestRunner$RunProcessor.call(StandardProcessorTestRunner.java:243)
at 
org.apache.nifi.util.StandardProcessorTestRunner$RunProcessor.call(StandardProcessorTestRunner.java:237)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Running org.apache.nifi.processors.azure.eventhub.PutAzureEventHubTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 sec - in 
org.apache.nifi.processors.azure.eventhub.PutAzureEventHubTest

Results :

Failed tests:
  GetAzureEventHubTest.testNormalFlow:108 
org.powermock.reflect.exceptions.FieldNotFoundException: No instance field 
named "isReceivedEvent" could be found in the class hierarchy of 
com.microsoft.azure.eventhubs.EventData.
{quote}

The tests for the processors also need to be moved to src/test/java (currently 
in src/test) so that the tests are detected during the build.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3304) In the Administration Guide, there is an incorrect JAVA 8 parameter for bootstrap.conf file


[ 
https://issues.apache.org/jira/browse/NIFI-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963069#comment-15963069
 ] 

Andre F de Miranda commented on NIFI-3304:
--

seems fine.

> In the Administration Guide, there is an incorrect JAVA 8 parameter for 
> bootstrap.conf file
> ---
>
> Key: NIFI-3304
> URL: https://issues.apache.org/jira/browse/NIFI-3304
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Documentation & Website
>Reporter: David A. Wynne
>Assignee: Michael Moser
>Priority: Minor
>
> In the Administration guide, section Bootstrap properties, there is a 
> paragraph about Java 8 handling of codecache, the listed properties are 
> accurate for Java version 7, but for Java version 8, one has changed
> This is correct for Java version 7:
> java.arg.8=-XX:CodeCacheFlushingMinimumFreeSpace=10m
> But for Java version 8, it should be
> java.arg.8=-XX:CodeCacheMinimumFreeSpace=10m



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-1833) Add support for Azure Blob Storage and Table Storage


[ 
https://issues.apache.org/jira/browse/NIFI-1833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963045#comment-15963045
 ] 

ASF GitHub Bot commented on NIFI-1833:
--

Github user jtstorck commented on the issue:

https://github.com/apache/nifi/pull/1636
  
@aperepel Integration tests are written for the processors, but aren't 
included in the build.  There should probably be some tests that do get run 
with the build, using the TestRunner.  The CloudStorageAccount would have to be 
mocked, which at a minimum you could do by making an extension of the class 
under test in which the createStorageAccount method used by the test returns a 
mock.


> Add support for Azure Blob Storage and Table Storage
> 
>
> Key: NIFI-1833
> URL: https://issues.apache.org/jira/browse/NIFI-1833
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 0.6.1
>Reporter: Simon Elliston Ball
>Priority: Minor
>
> It would be useful to have an Azure equivalent of the current S3 capability. 
> Azure also provides a Table storage mechanism, providing simple key value 
> storage. Since the Azure SDKs are Apache Licensed, this should be reasonably 
> straightforward. A first cut is available as an addition to the existing 
> azure bundle.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi issue #1636: NIFI-1833 - Azure Storage processors

2017-04-10 Thread jtstorck

Github user jtstorck commented on the issue:

https://github.com/apache/nifi/pull/1636
  
@aperepel Integration tests are written for the processors, but aren't 
included in the build.  There should probably be some tests that do get run 
with the build, using the TestRunner.  The CloudStorageAccount would have to be 
mocked, which at a minimum you could do by making an extension of the class 
under test in which the createStorageAccount method used by the test returns a 
mock.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2747) Add Processor support SSDeep (SpamSum) hashes (and possibly others e.g. TLSH)


[ 
https://issues.apache.org/jira/browse/NIFI-2747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963042#comment-15963042
 ] 

ASF GitHub Bot commented on NIFI-2747:
--

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1619
  
no worries. rebased


> Add Processor support SSDeep (SpamSum) hashes (and possibly others e.g. TLSH)
> -
>
> Key: NIFI-2747
> URL: https://issues.apache.org/jira/browse/NIFI-2747
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.0.0
>Reporter: Andre F de Miranda
>Assignee: Andre F de Miranda
>
> It would be great if we had a processor to support fuzzy hashing algorithms 
> such as SpamSum/SSDeep



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi issue #1619: NIFI-2747 - Introduce FuzzyHashContent processor

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1619
  
no worries. rebased


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Comment Edited] (NIFI-1131) InvokeHttp relationships don't match other processors


[ 
https://issues.apache.org/jira/browse/NIFI-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963024#comment-15963024
 ] 

Andre F de Miranda edited comment on NIFI-1131 at 4/10/17 3:29 PM:
---

-[~josephxsxn] I suspect [~joewitt] was referring to adding {{.displayName}} to 
the property builder allows you to discretionary change a property rendering 
value without causing impact to its underlying relationships (as within 
flow.xml)?-

-If so, all you need it to change your PR so that the property look a bit like 
this:-

-{{https://github.com/apache/nifi/blob/b7cdc6b382b09d601b1e9ecf81ef16b9246309b3/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ConsumeEWS.java#L161}}-


was (Author: trixpan):
[~josephxsxn] I suspect [~joewitt] was referring to adding {{.displayName}} to 
the property builder allows you to discretionary change a property rendering 
value without causing impact to its underlying relationships (as within 
flow.xml)?

If so, all you need it to change your PR so that the property look a bit like 
this:

https://github.com/apache/nifi/blob/b7cdc6b382b09d601b1e9ecf81ef16b9246309b3/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ConsumeEWS.java#L161

> InvokeHttp relationships don't match other processors
> -
>
> Key: NIFI-1131
> URL: https://issues.apache.org/jira/browse/NIFI-1131
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.0.0
>Reporter: Joseph Percivall
>Assignee: Joseph Niemiec
>Priority: Trivial
>  Labels: newbie
>
> The current InvokeHttp relationships are "Original", "Response", "Retry", "No 
> Retry" and "Failure". They should be renamed as such:
> "Original" -> "Success Request"
> "Response" -> stays same
> "Retry" -> "Retry Request"
> "No Retry" -> "No Retry Request"
> "Failure" -> "Failure Request"
> In other processors the "Original" relationship is the unmodified original 
> flowfile but in InvokeHttp it is used as a destination for a modified 
> request. In InvokeHttp the request flowfile coming in is routed to different 
> relationships depending on the status code received and is modified with new 
> attributes.
> This would be a breaking change and would be slated for 1.0.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-1131) InvokeHttp relationships don't match other processors


[ 
https://issues.apache.org/jira/browse/NIFI-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963034#comment-15963034
 ] 

Andre F de Miranda commented on NIFI-1131:
--

[~josephxsxn] discard my last comment. Seems you are on the right track!

> InvokeHttp relationships don't match other processors
> -
>
> Key: NIFI-1131
> URL: https://issues.apache.org/jira/browse/NIFI-1131
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.0.0
>Reporter: Joseph Percivall
>Assignee: Joseph Niemiec
>Priority: Trivial
>  Labels: newbie
>
> The current InvokeHttp relationships are "Original", "Response", "Retry", "No 
> Retry" and "Failure". They should be renamed as such:
> "Original" -> "Success Request"
> "Response" -> stays same
> "Retry" -> "Retry Request"
> "No Retry" -> "No Retry Request"
> "Failure" -> "Failure Request"
> In other processors the "Original" relationship is the unmodified original 
> flowfile but in InvokeHttp it is used as a destination for a modified 
> request. In InvokeHttp the request flowfile coming in is routed to different 
> relationships depending on the status code received and is modified with new 
> attributes.
> This would be a breaking change and would be slated for 1.0.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3591) Relationship should support a DisplayName and Name like PropertyDescriptor


[ 
https://issues.apache.org/jira/browse/NIFI-3591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963033#comment-15963033
 ] 

Andre F de Miranda commented on NIFI-3591:
--

Ah Ok. Ok. I get it now. 

> Relationship should support a DisplayName and Name like PropertyDescriptor
> --
>
> Key: NIFI-3591
> URL: https://issues.apache.org/jira/browse/NIFI-3591
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joseph Niemiec
>Assignee: Joseph Niemiec
>Priority: Minor
>
> Today the PropertyDescriptor supports a display name which allows UI updates 
> without breaking of the flow.xml.gz file in terms of downstream relationship 
> binding. If we attempted to just update the NAME processors my break, but by 
> enabling a decoupled displayname this can get updated at will.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3591) Relationship should support a DisplayName and Name like PropertyDescriptor

2017-04-10 Thread Joseph Niemiec (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-3591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963025#comment-15963025
 ] 

Joseph Niemiec commented on NIFI-3591:
--

Hi Andre,

I think its the Properties.java class that supports Name and DisplayName, but 
Relationship only supports Name.  This isnt really an issue more of just like 
for like so NIFI-1131 can be done without breaking changes in a minor release. 
But its possible I misunderstood Joe's commits in 1131. I'll look back over and 
follow up. 

https://github.com/apache/nifi/blob/d838f61291d2582592754a37314911b701c6891b/nifi-api/src/main/java/org/apache/nifi/processor/Relationship.java



> Relationship should support a DisplayName and Name like PropertyDescriptor
> --
>
> Key: NIFI-3591
> URL: https://issues.apache.org/jira/browse/NIFI-3591
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joseph Niemiec
>Assignee: Joseph Niemiec
>Priority: Minor
>
> Today the PropertyDescriptor supports a display name which allows UI updates 
> without breaking of the flow.xml.gz file in terms of downstream relationship 
> binding. If we attempted to just update the NAME processors my break, but by 
> enabling a decoupled displayname this can get updated at will.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-1131) InvokeHttp relationships don't match other processors


[ 
https://issues.apache.org/jira/browse/NIFI-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963024#comment-15963024
 ] 

Andre F de Miranda commented on NIFI-1131:
--

[~josephxsxn] I suspect [~joewitt] was referring to adding {{.displayName}} to 
the property builder allows you to discretionary change a property rendering 
value without causing impact to its underlying relationships (as within 
flow.xml)?

If so, all you need it to change your PR so that the property look a bit like 
this:

https://github.com/apache/nifi/blob/b7cdc6b382b09d601b1e9ecf81ef16b9246309b3/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/ConsumeEWS.java#L161

> InvokeHttp relationships don't match other processors
> -
>
> Key: NIFI-1131
> URL: https://issues.apache.org/jira/browse/NIFI-1131
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.0.0
>Reporter: Joseph Percivall
>Assignee: Joseph Niemiec
>Priority: Trivial
>  Labels: newbie
>
> The current InvokeHttp relationships are "Original", "Response", "Retry", "No 
> Retry" and "Failure". They should be renamed as such:
> "Original" -> "Success Request"
> "Response" -> stays same
> "Retry" -> "Retry Request"
> "No Retry" -> "No Retry Request"
> "Failure" -> "Failure Request"
> In other processors the "Original" relationship is the unmodified original 
> flowfile but in InvokeHttp it is used as a destination for a modified 
> request. In InvokeHttp the request flowfile coming in is routed to different 
> relationships depending on the status code received and is modified with new 
> attributes.
> This would be a breaking change and would be slated for 1.0.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3591) Relationship should support a DisplayName and Name like PropertyDescriptor


[ 
https://issues.apache.org/jira/browse/NIFI-3591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963011#comment-15963011
 ] 

Andre F de Miranda commented on NIFI-3591:
--

[~josephxsxn] - mind sharing a bit about what the issue is? I would imagine 
{{.displayName}} should already cover this?

> Relationship should support a DisplayName and Name like PropertyDescriptor
> --
>
> Key: NIFI-3591
> URL: https://issues.apache.org/jira/browse/NIFI-3591
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Joseph Niemiec
>Assignee: Joseph Niemiec
>Priority: Minor
>
> Today the PropertyDescriptor supports a display name which allows UI updates 
> without breaking of the flow.xml.gz file in terms of downstream relationship 
> binding. If we attempted to just update the NAME processors my break, but by 
> enabling a decoupled displayname this can get updated at will.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3439) Making password property in HiveConnectionPool a hidden show.


[ 
https://issues.apache.org/jira/browse/NIFI-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962981#comment-15962981
 ] 

ASF GitHub Bot commented on NIFI-3439:
--

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1546
  
forget it. Seen what @mcgilman was referring to. Possible to play with text 
area but it becomes single line. :-(


> Making password property in HiveConnectionPool a hidden show.
> -
>
> Key: NIFI-3439
> URL: https://issues.apache.org/jira/browse/NIFI-3439
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.1.1
> Environment: Google Chrome browser
>Reporter: Ramakrishnan V
>Assignee: Scott Aslan
>Priority: Minor
>  Labels: ControllerService, Hive, security
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> When user types the password in the password attribute in hive connection 
> pool controller service, the characters are visible. Once completed the 
> password is encrypted and no longer visible.
> shouldn't be me made as normal hidden show password box ?
> The user always has the option to delete the controller service and 
> re-configure if he has entered a wrong password.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi issue #1546: [NIFI-3439] when users type into sensitive property fields...

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1546
  
forget it. Seen what @mcgilman was referring to. Possible to play with text 
area but it becomes single line. :-(


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2747) Add Processor support SSDeep (SpamSum) hashes (and possibly others e.g. TLSH)


[ 
https://issues.apache.org/jira/browse/NIFI-2747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962974#comment-15962974
 ] 

ASF GitHub Bot commented on NIFI-2747:
--

Github user apiri commented on the issue:

https://github.com/apache/nifi/pull/1619
  
@trixpan sorry, been traveling and have had a bit of a hectic schedule.  
would you please be able to rebase and then I will look to get this closed out 
in the next day or so?


> Add Processor support SSDeep (SpamSum) hashes (and possibly others e.g. TLSH)
> -
>
> Key: NIFI-2747
> URL: https://issues.apache.org/jira/browse/NIFI-2747
> Project: Apache NiFi
>  Issue Type: Improvement
>Affects Versions: 1.0.0
>Reporter: Andre F de Miranda
>Assignee: Andre F de Miranda
>
> It would be great if we had a processor to support fuzzy hashing algorithms 
> such as SpamSum/SSDeep



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi issue #1619: NIFI-2747 - Introduce FuzzyHashContent processor

Github user apiri commented on the issue:

https://github.com/apache/nifi/pull/1619
  
@trixpan sorry, been traveling and have had a bit of a hectic schedule.  
would you please be able to rebase and then I will look to get this closed out 
in the next day or so?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-site pull request #17: MiNiFi-260: Create a "Getting Started" Guide for...

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-site/pull/17#discussion_r110674818
  
--- Diff: src/pages/markdown/minifi/getting-started.md ---
@@ -0,0 +1,122 @@
+---
+title: Apache NiFi MiNiFi: Getting Started
+---
+
+# Getting started with MiNiFi
+
+This page explains how to configure and deploy MiNiFi agents.
+
+The Java agent is able to run most of [NiFi's available 
processors](http://nifi.apache.org/docs.html), but is a larger binary 
distribution (49MB) and consumes greater system resources (24MB max JVM 
heapsize by default). If you need maximum flexibility to make routing and 
processing decisions at your data's point of origin, the Java agent is a good 
fit.
+
+The C++ agent is a smaller binary (3.2MB), consumes low system memory 
(about 5MB at idle) but has [a limited subset of 
processors](https://github.com/apache/nifi-minifi-cpp#caveats). If your primary 
concern is gathering and pushing data to downstream consumers and minimizing 
system impact, the C++ agent is a good fit.
+
+
+1. Install the appropriate OS level dependencies:
+
+   MiNiFi Java:
+
+  - Java 1.8+
+
+   MiNiFi C++:
--- End diff --

Should also note C++ 11 needed 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-site pull request #17: MiNiFi-260: Create a "Getting Started" Guide for...

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-site/pull/17#discussion_r110673346
  
--- Diff: src/pages/markdown/minifi/getting-started.md ---
@@ -0,0 +1,120 @@
+---
+title: Apache NiFi MiNiFi: Getting Started
+---
+
+# Getting started with MiNiFi
+
+This page explains how to configure and deploy MiNiFi agents.
+
+The Java agent is able to run most of [NiFi's available 
processors](http://nifi.apache.org/docs.html), but is a larger binary 
distribution (49MB) and consumes greater system resources (24MB max JVM 
heapsize by default). If you need maximum flexibility to make routing and 
processing decisions at your data's point of origin, the Java agent is a good 
fit.
+
+The C++ agent is a smaller binary (3.2MB), consumes low system memory 
(about 5MB at idle) but has [a limited subset of 
processors](https://github.com/apache/nifi-minifi-cpp#caveats). If your primary 
concern is gathering and pushing data to downstream consumers and minimizing 
system impact, the C++ agent is a good fit.
+
+
+1. Install the appropriate OS level dependencies:
+
+   MiNiFi Java:
+
+  - Java 1.8+
+
+   MiNiFi C++:
+   ## RHEL/CentOS:
+   - yum install -y epel-release
+   - yum install -y leveldb
+
+   ## Debian/Ubuntu:
+   - apt install -y libleveldb-dev
+   - apt install -y libxml2
+2. Download the relevant compressed binary from the 
[Downloads](download.html) page 
+3. Copy and decompress the binary to your target deployment environment
+4. Set the MINIFI_HOME environment variable to your decompressed binary 
directory
+5. From $MINIFI_HOME, ./bin/minifi.sh 
{start|stop|run|restart|status|flowStatus|dump|install}
+
+*flowStatus* and *dump* are available only for MiNiFi Java.
+
+For MiNiFi Java, the agent logs to $MINIFI_HOME/logs/minifi-app.log.
+
+For MiNiFi C++, the agent logs to $MINIFI_HOME/minifi-app.log.txt
+
+# Configuring MiNiFi Agent Flows
+
+Agents execute the flow defined in $MINIFI_HOME/conf/config.yml. The 
shipped config.yml file has an "empty" flow.
+
+While it's possible to create and modify config.yml programmatically or by 
hand, the more common scenario is to:
+1. Use NiFi's web interface to define a flow
+2. Save the flow as a template and download its XML file
+3. Use the [Converter Toolkit](minifi-toolkit.html) to convert the 
template XML to an equivalent config.yml
+4. Put the config.yml file in $MINIFI_HOME/conf and start the agent.
+
+# Pushing Data From MiNiFi to NiFi
+
+What follows is an illustrated walkthrough of configuring MiNiFi and NiFi 
to generate and collect log data.
+
+If you're new to NiFi, watch the [NiFi introductory video 
series](http://nifi.apache.org/videos.html) before proceeding. If you don't 
already have NiFi, you can download it from 
[here](http://nifi.apache.org/download.html). 
+
+## Configuring NiFi to Receive Data
+
+First, check that $NIFI_HOME/conf/nifi.properties has a Site to Site input 
socket port specified:
+```
+# Site to Site properties
+nifi.remote.input.host=
+nifi.remote.input.secure=false
+nifi.remote.input.socket.port=1026
+nifi.remote.input.http.enabled=true
+nifi.remote.input.http.transaction.ttl=30 sec
+```
+
+Restart NiFi so changes take effect.
+
+On NiFi's web UI, place an Input Port "From MiNiFi" and connect it to a 
downstream processor:
+![](../assets/images/minifi/nifi-receive.png)
+
+## Creating the MiNiFi Flow
+
+Create a new process group, and double click on it to enter it. Working 
inside this new processor group will let us create a clean template that 
includes flow components meant to run in MiNiFi.
+
+Inside the process group, create a Remote Process Group (RPG) and give it 
the URL of your NiFi instance. The RPG will connect to NiFi and discover the 
input port created earlier.
+
+Create a TailFile processor and connect it to the RPG, selecting the "From 
MiNiFi" input port.
+![](../assets/images/minifi/minifi-flow.png)
+
+Now create a template, download it as an XML file, and download the 
[Converter ToolKit](download.html).
+
+From the toolkit directory:
+```
+[root@host minifi-toolkit-0.0.1]# ./bin/config.sh transform 
/path/to/template.xml /output/path/config.yml
+
+Java home: /usr/lib/jvm/jre-1.8.0-openjdk
+MiNiFi Toolkit home: /minifi-toolkit-0.0.1
+
+
+No validation errors found in template.
--- End diff --

We likely have some room for improvement on this front (from a tooling 
perspective, less so this doc patch).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and

[GitHub] nifi-site pull request #17: MiNiFi-260: Create a "Getting Started" Guide for...

Github user apiri commented on a diff in the pull request:

https://github.com/apache/nifi-site/pull/17#discussion_r110674033
  
--- Diff: src/includes/minifi/minifi-topbar.hbs ---
@@ -26,16 +26,17 @@
 
 Documentation
 
-FAQ
+Getting Started
 Administrator's Guide
 Converter Toolkit
+FAQ
 https://cwiki.apache.org/confluence/display/MiNiFi;>Wiki
 
 
 
-Download
--- End diff --

Pretty minor but would like some consistency between NiFi's nav and 
MiNiFi's.  Could we either roll these back or update NIFi's to be similar?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3439) Making password property in HiveConnectionPool a hidden show.


[ 
https://issues.apache.org/jira/browse/NIFI-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962967#comment-15962967
 ] 

ASF GitHub Bot commented on NIFI-3439:
--

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1546
  
The approach linked above seems to work on chrome without the javascript 
but firefox required the js code.


> Making password property in HiveConnectionPool a hidden show.
> -
>
> Key: NIFI-3439
> URL: https://issues.apache.org/jira/browse/NIFI-3439
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.1.1
> Environment: Google Chrome browser
>Reporter: Ramakrishnan V
>Assignee: Scott Aslan
>Priority: Minor
>  Labels: ControllerService, Hive, security
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> When user types the password in the password attribute in hive connection 
> pool controller service, the characters are visible. Once completed the 
> password is encrypted and no longer visible.
> shouldn't be me made as normal hidden show password box ?
> The user always has the option to delete the controller service and 
> re-configure if he has entered a wrong password.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi issue #1546: [NIFI-3439] when users type into sensitive property fields...

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1546
  
The approach linked above seems to work on chrome without the javascript 
but firefox required the js code.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi-minifi-cpp pull request #76: MINIFI-261: Allow std::string to be used f...

2017-04-10 Thread phrocker

GitHub user phrocker opened a pull request:

https://github.com/apache/nifi-minifi-cpp/pull/76

MINIFI-261: Allow std::string to be used for log messages

Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
 in the commit message?

- [ ] Does your PR title start with MINIFI- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.

- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [ ] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file?
- [ ] If applicable, have you updated the NOTICE file?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-261

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi-minifi-cpp/pull/76.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #76






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3439) Making password property in HiveConnectionPool a hidden show.


[ 
https://issues.apache.org/jira/browse/NIFI-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962960#comment-15962960
 ] 

ASF GitHub Bot commented on NIFI-3439:
--

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1546
  
@scottyaslan - have you tested this?

http://jsfiddle.net/Ldr07h2r/6/

Source:

http://stackoverflow.com/questions/25822682/webkit-text-security-compatibility


> Making password property in HiveConnectionPool a hidden show.
> -
>
> Key: NIFI-3439
> URL: https://issues.apache.org/jira/browse/NIFI-3439
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.1.1
> Environment: Google Chrome browser
>Reporter: Ramakrishnan V
>Assignee: Scott Aslan
>Priority: Minor
>  Labels: ControllerService, Hive, security
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> When user types the password in the password attribute in hive connection 
> pool controller service, the characters are visible. Once completed the 
> password is encrypted and no longer visible.
> shouldn't be me made as normal hidden show password box ?
> The user always has the option to delete the controller service and 
> re-configure if he has entered a wrong password.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi issue #1546: [NIFI-3439] when users type into sensitive property fields...

Github user trixpan commented on the issue:

https://github.com/apache/nifi/pull/1546
  
@scottyaslan - have you tested this?

http://jsfiddle.net/Ldr07h2r/6/

Source:

http://stackoverflow.com/questions/25822682/webkit-text-security-compatibility


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-2952) ReplaceText: Search String Validation Error


 [ 
https://issues.apache.org/jira/browse/NIFI-2952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andre F de Miranda updated NIFI-2952:
-
   Resolution: Fixed
Fix Version/s: 1.2.0
   Status: Resolved  (was: Patch Available)

> ReplaceText: Search String Validation Error
> ---
>
> Key: NIFI-2952
> URL: https://issues.apache.org/jira/browse/NIFI-2952
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Justin M Cavinee
>Assignee: Pierre Villard
>Priority: Minor
> Fix For: 1.2.0
>
>
> The ReplaceText Processor's SearchText Validator always uses a Regex 
> Validator, even when the Literal Replace is chosen as the Replacement 
> Strategy.
> The Replace Works properly, we're just unable to use a search string that 
> fails regex validation.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3035) URL to display a particular process group in UI


[ 
https://issues.apache.org/jira/browse/NIFI-3035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962947#comment-15962947
 ] 

ASF GitHub Bot commented on NIFI-3035:
--

Github user gresockj commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1659#discussion_r110669589
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-canvas-utils.js
 ---
@@ -412,9 +412,9 @@
 
 // create object whose keys are the parameter name and the 
values are the parameter values
 var paramsObject = {};
-params.forEach(function (v, k) {
-paramsObject[k] = v;
-});
+for(var pair of params.entries()) {
--- End diff --

Interesting, I see what you mean.  When I build the nifi-web-ui module with 
the "of" in place, I get the following error:
[ERROR] 
/var/local/workspace/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-canvas-utils.js
 [1:0]: Compilation produced 301 syntax errors.
[ERROR] ...nf-canvas-utils.js:line 1:column 0:Compilation produced 301 
syntax errors.

However, when I change "of " to "in", the error goes away and it is able to 
build the nar.  How did you get around this issue?


> URL to display a particular process group in UI
> ---
>
> Key: NIFI-3035
> URL: https://issues.apache.org/jira/browse/NIFI-3035
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core UI
>Reporter: Christine Draper
>Assignee: Scott Aslan
> Fix For: 1.2.0
>
>
> Our use case has multiple teams of users working on specific process groups. 
> We would like to be able to give them a URL that will launch the UI on the 
> specific group they are working on, rather than them having to navigate to it 
> from the root group.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3414) Implement an EnforceOrder processor


[ 
https://issues.apache.org/jira/browse/NIFI-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962938#comment-15962938
 ] 

ASF GitHub Bot commented on NIFI-3414:
--

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1496#discussion_r110668232
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EnforceOrder.java
 ---
@@ -0,0 +1,543 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.Stateful;
+import org.apache.nifi.annotation.behavior.TriggerSerially;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.state.Scope;
+import org.apache.nifi.components.state.StateMap;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.TreeMap;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import static org.apache.commons.lang3.StringUtils.isBlank;
+
+@EventDriven
+@Tags({"sort", "order"})
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@TriggerSerially
+@CapabilityDescription("Enforces expected ordering of FlowFiles those 
belong to the same data group. " +
+" Although PriorityAttributePrioritizer can be used on a 
connection to ensure that flow files going through that connection are in 
priority order," +
+" depending on error-handling, branching, and other flow designs, 
it is possible for FlowFiles to get out-of-order." +
+" EnforceOrder can be used to enforce original ordering for those 
FlowFiles." +
+" [IMPORTANT] In order to take effect of EnforceOrder, 
FirstInFirstOutPrioritizer should be used at EVERY downstream relationship" +
+" UNTIL the order of FlowFiles physically get FIXED by operation 
such as MergeContent or being stored to the final destination.")
+@Stateful(scopes = Scope.LOCAL, description = "EnforceOrder uses following 
states per ordering group:" +
+" '.target' is a order number which is being waited to 
arrive next." +
+" When a FlowFile with a matching order arrives, or a FlowFile 
overtakes the FlowFile being waited for because of wait timeout," +
+" target order will be updated to (FlowFile.order + 1)." +
+" '.max is the maximum order number for a group." +
+" '.updatedAt' is a timestamp when the order of a group 
was updated

[GitHub] nifi pull request #1496: NIFI-3414: Added EnforceOrder processor

2017-04-10 Thread ijokarumawak

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1496#discussion_r110668232
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EnforceOrder.java
 ---
@@ -0,0 +1,543 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.Stateful;
+import org.apache.nifi.annotation.behavior.TriggerSerially;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.state.Scope;
+import org.apache.nifi.components.state.StateMap;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.TreeMap;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import static org.apache.commons.lang3.StringUtils.isBlank;
+
+@EventDriven
+@Tags({"sort", "order"})
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@TriggerSerially
+@CapabilityDescription("Enforces expected ordering of FlowFiles those 
belong to the same data group. " +
+" Although PriorityAttributePrioritizer can be used on a 
connection to ensure that flow files going through that connection are in 
priority order," +
+" depending on error-handling, branching, and other flow designs, 
it is possible for FlowFiles to get out-of-order." +
+" EnforceOrder can be used to enforce original ordering for those 
FlowFiles." +
+" [IMPORTANT] In order to take effect of EnforceOrder, 
FirstInFirstOutPrioritizer should be used at EVERY downstream relationship" +
+" UNTIL the order of FlowFiles physically get FIXED by operation 
such as MergeContent or being stored to the final destination.")
+@Stateful(scopes = Scope.LOCAL, description = "EnforceOrder uses following 
states per ordering group:" +
+" '.target' is a order number which is being waited to 
arrive next." +
+" When a FlowFile with a matching order arrives, or a FlowFile 
overtakes the FlowFile being waited for because of wait timeout," +
+" target order will be updated to (FlowFile.order + 1)." +
+" '.max is the maximum order number for a group." +
+" '.updatedAt' is a timestamp when the order of a group 
was updated last time." +
+" These managed states will be removed automatically once a group 
is determined as inactive, see 'Inactive Timeout' for detail.")
+@WritesAttributes({
+@WritesAttribute(attribute =

[GitHub] nifi pull request #1659: [NIFI-3035] use api supported .entries() to iterate...

2017-04-10 Thread scottyaslan

Github user scottyaslan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1659#discussion_r110667414
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-canvas-utils.js
 ---
@@ -412,9 +412,9 @@
 
 // create object whose keys are the parameter name and the 
values are the parameter values
 var paramsObject = {};
-params.forEach(function (v, k) {
-paramsObject[k] = v;
-});
+for(var pair of params.entries()) {
--- End diff --

Changing the "of" to "in" is not correct... I tried it and verified that 
using the "in" breaks deep linking. If you look here (in the Browser 
Compatibility section): 
https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams we see that 
"for...of" is the supported way to iterate over the GET params...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3035) URL to display a particular process group in UI


[ 
https://issues.apache.org/jira/browse/NIFI-3035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962932#comment-15962932
 ] 

ASF GitHub Bot commented on NIFI-3035:
--

Github user scottyaslan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1659#discussion_r110667414
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-canvas-utils.js
 ---
@@ -412,9 +412,9 @@
 
 // create object whose keys are the parameter name and the 
values are the parameter values
 var paramsObject = {};
-params.forEach(function (v, k) {
-paramsObject[k] = v;
-});
+for(var pair of params.entries()) {
--- End diff --

Changing the "of" to "in" is not correct... I tried it and verified that 
using the "in" breaks deep linking. If you look here (in the Browser 
Compatibility section): 
https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams we see that 
"for...of" is the supported way to iterate over the GET params...


> URL to display a particular process group in UI
> ---
>
> Key: NIFI-3035
> URL: https://issues.apache.org/jira/browse/NIFI-3035
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core UI
>Reporter: Christine Draper
>Assignee: Scott Aslan
> Fix For: 1.2.0
>
>
> Our use case has multiple teams of users working on specific process groups. 
> We would like to be able to give them a URL that will launch the UI on the 
> specific group they are working on, rather than them having to navigate to it 
> from the root group.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3414) Implement an EnforceOrder processor

2017-04-10 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/NIFI-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962931#comment-15962931
 ] 

ASF GitHub Bot commented on NIFI-3414:
--

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1496#discussion_r110667002
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EnforceOrder.java
 ---
@@ -0,0 +1,543 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.Stateful;
+import org.apache.nifi.annotation.behavior.TriggerSerially;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.state.Scope;
+import org.apache.nifi.components.state.StateMap;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.TreeMap;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import static org.apache.commons.lang3.StringUtils.isBlank;
+
+@EventDriven
+@Tags({"sort", "order"})
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@TriggerSerially
+@CapabilityDescription("Enforces expected ordering of FlowFiles those 
belong to the same data group. " +
+" Although PriorityAttributePrioritizer can be used on a 
connection to ensure that flow files going through that connection are in 
priority order," +
+" depending on error-handling, branching, and other flow designs, 
it is possible for FlowFiles to get out-of-order." +
+" EnforceOrder can be used to enforce original ordering for those 
FlowFiles." +
+" [IMPORTANT] In order to take effect of EnforceOrder, 
FirstInFirstOutPrioritizer should be used at EVERY downstream relationship" +
+" UNTIL the order of FlowFiles physically get FIXED by operation 
such as MergeContent or being stored to the final destination.")
+@Stateful(scopes = Scope.LOCAL, description = "EnforceOrder uses following 
states per ordering group:" +
+" '.target' is a order number which is being waited to 
arrive next." +
+" When a FlowFile with a matching order arrives, or a FlowFile 
overtakes the FlowFile being waited for because of wait timeout," +
+" target order will be updated to (FlowFile.order + 1)." +
+" '.max is the maximum order number for a group." +
+" '.updatedAt' is a timestamp when the order of a group 
was updated

[GitHub] nifi pull request #1496: NIFI-3414: Added EnforceOrder processor

2017-04-10 Thread ijokarumawak

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1496#discussion_r110667002
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EnforceOrder.java
 ---
@@ -0,0 +1,543 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.Stateful;
+import org.apache.nifi.annotation.behavior.TriggerSerially;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.state.Scope;
+import org.apache.nifi.components.state.StateMap;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Comparator;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.TreeMap;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import static org.apache.commons.lang3.StringUtils.isBlank;
+
+@EventDriven
+@Tags({"sort", "order"})
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@TriggerSerially
+@CapabilityDescription("Enforces expected ordering of FlowFiles those 
belong to the same data group. " +
+" Although PriorityAttributePrioritizer can be used on a 
connection to ensure that flow files going through that connection are in 
priority order," +
+" depending on error-handling, branching, and other flow designs, 
it is possible for FlowFiles to get out-of-order." +
+" EnforceOrder can be used to enforce original ordering for those 
FlowFiles." +
+" [IMPORTANT] In order to take effect of EnforceOrder, 
FirstInFirstOutPrioritizer should be used at EVERY downstream relationship" +
+" UNTIL the order of FlowFiles physically get FIXED by operation 
such as MergeContent or being stored to the final destination.")
+@Stateful(scopes = Scope.LOCAL, description = "EnforceOrder uses following 
states per ordering group:" +
+" '.target' is a order number which is being waited to 
arrive next." +
+" When a FlowFile with a matching order arrives, or a FlowFile 
overtakes the FlowFile being waited for because of wait timeout," +
+" target order will be updated to (FlowFile.order + 1)." +
+" '.max is the maximum order number for a group." +
+" '.updatedAt' is a timestamp when the order of a group 
was updated last time." +
+" These managed states will be removed automatically once a group 
is determined as inactive, see 'Inactive Timeout' for detail.")
+@WritesAttributes({
+@WritesAttribute(attribute =

[jira] [Commented] (NIFI-2952) ReplaceText: Search String Validation Error


[ 
https://issues.apache.org/jira/browse/NIFI-2952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962919#comment-15962919
 ] 

ASF subversion and git services commented on NIFI-2952:
---

Commit 02d38b55bf43e8becf62029c994eba4535cae8f8 in nifi's branch 
refs/heads/master from [~pvillard]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=02d38b5 ]

NIFI-2952 Fixed search value validation in ReplaceText

This closes: #1378

Signed-off-by: Andre F de Miranda 


> ReplaceText: Search String Validation Error
> ---
>
> Key: NIFI-2952
> URL: https://issues.apache.org/jira/browse/NIFI-2952
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Justin M Cavinee
>Assignee: Pierre Villard
>Priority: Minor
>
> The ReplaceText Processor's SearchText Validator always uses a Regex 
> Validator, even when the Literal Replace is chosen as the Replacement 
> Strategy.
> The Replace Works properly, we're just unable to use a search string that 
> fails regex validation.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-2952) ReplaceText: Search String Validation Error


[ 
https://issues.apache.org/jira/browse/NIFI-2952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15962921#comment-15962921
 ] 

ASF GitHub Bot commented on NIFI-2952:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1378


> ReplaceText: Search String Validation Error
> ---
>
> Key: NIFI-2952
> URL: https://issues.apache.org/jira/browse/NIFI-2952
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Reporter: Justin M Cavinee
>Assignee: Pierre Villard
>Priority: Minor
>
> The ReplaceText Processor's SearchText Validator always uses a Regex 
> Validator, even when the Literal Replace is chosen as the Replacement 
> Strategy.
> The Replace Works properly, we're just unable to use a search string that 
> fails regex validation.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi pull request #1378: NIFI-2952 Fixed search value validation in ReplaceT...

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1378


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3679) Add Content Viewer for Avro