[jira] [Commented] (NIFI-4023) WriteAheadProvenanceRepository indexing and query failure under high rate stress testing

[Randy Bovay (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122855#comment-16122855
 ] 

Randy Bovay commented on NIFI-4023:
---

We are experiencing this as well.  Good to see there is already an issue 
tracking this.  
Happy to test with you more to help find a solution to this.

> WriteAheadProvenanceRepository indexing and query failure under high rate 
> stress testing
> 
>
> Key: NIFI-4023
> URL: https://issues.apache.org/jira/browse/NIFI-4023
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.3.0
>Reporter: Joseph Witt
>
> 2017-06-06 00:32:35,995 INFO [pool-10-thread-1] 
> org.wali.MinimalLockingWriteAheadLog 
> org.wali.MinimalLockingWriteAheadLog@5ce7ab6f checkpointed with 5737 Records 
> and 0 Swap Files in 467 milliseconds (Stop-the-world time = 172 milliseconds, 
> Clear Edit Logs time = 137 millis), max Transaction ID 5739
> 2017-06-06 00:32:35,996 INFO [pool-10-thread-1] 
> o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile 
> Repository with 5737 records in 467 milliseconds
> 2017-06-06 00:33:35,418 ERROR [Index Provenance Events-2] 
> o.a.n.p.index.lucene.EventIndexTask Failed to index Provenance Events
> org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: 
> NativeFSLock@/Users/jwitt/build-verify/nifi-1.3.0/nifi-assembly/target/nifi-1.3.0-bin/nifi-1.3.0/provenance_repository/index-1496723454612/write.lock
>   at org.apache.lucene.store.Lock.obtain(Lock.java:89)
>   at org.apache.lucene.index.IndexWriter.(IndexWriter.java:755)
>   at 
> org.apache.nifi.provenance.lucene.SimpleIndexManager.createWriter(SimpleIndexManager.java:198)
>   at 
> org.apache.nifi.provenance.lucene.SimpleIndexManager.borrowIndexWriter(SimpleIndexManager.java:227)
>   at 
> org.apache.nifi.provenance.index.lucene.EventIndexTask.index(EventIndexTask.java:184)
>   at 
> org.apache.nifi.provenance.index.lucene.EventIndexTask.run(EventIndexTask.java:104)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:748)
> 2017-06-06 00:33:36,420 ERROR [Index Provenance Events-1] 
> o.a.n.p.index.lucene.EventIndexTask Failed to index Provenance Events
> org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: 
> NativeFSLock@/Users/jwitt/build-verify/nifi-1.3.0/nifi-assembly/target/nifi-1.3.0-bin/nifi-1.3.0/provenance_repository/index-1496723454612/write.lock
>   at org.apache.lucene.store.Lock.obtain(Lock.java:89)
>   at org.apache.lucene.index.IndexWriter.(IndexWriter.java:755)
>   at 
> org.apache.nifi.provenance.lucene.SimpleIndexManager.createWriter(SimpleIndexManager.java:198)
>   at 
> org.apache.nifi.provenance.lucene.SimpleIndexManager.borrowIndexWriter(SimpleIndexManager.java:227)
>   at 
> org.apache.nifi.provenance.index.lucene.EventIndexTask.index(EventIndexTask.java:184)
>   at 
> org.apache.nifi.provenance.index.lucene.EventIndexTask.run(EventIndexTask.java:104)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:748)
> 2017-06-06 00:33:37,425 ERROR [Index Provenance Events-2] 
> o.a.n.p.index.lucene.EventIndexTask Failed to index Provenance Events
> org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: 
> NativeFSLock@/Users/jwitt/build-verify/nifi-1.3.0/nifi-assembly/target/nifi-1.3.0-bin/nifi-1.3.0/provenance_repository/index-1496723454612/write.lock
>   at org.apache.lucene.store.Lock.obtain(Lock.java:89)
>   at org.apache.lucene.index.IndexWriter.(IndexWriter.java:755)
>   at 
> org.apache.nifi.provenance.lucene.SimpleIndexManager.createWriter(SimpleIndexManager.java:198)
>   at 
> org.apache.nifi.provenance.lucene.SimpleIndexManager.borrowIndexWriter(SimpleIndexManager.java:227)
>   at 
> org.apache.nifi.provenance.index.lucene.EventIndexTask.index(EventIndexTask.java:184)
>   at 
> org.apache.nifi.provenance.index.lucene.EventIndexTask.run(EventIndexTask.java:104)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at 

[GitHub] nifi issue #2020: [NiFi-3973] Add PutKudu Processor for ingesting data to Ku...

[cammachusa]

Github user cammachusa commented on the issue:

https://github.com/apache/nifi/pull/2020
  
The last commit, I switched to onScheduled already. Thanks for your review 
and code @rickysaltzer  There are some code style check errors. I'm on it now.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4257) Allow a custom WHERE clause in AbstractDatabaseFetchProcessor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122783#comment-16122783
 ] 

ASF GitHub Bot commented on NIFI-4257:
--

Github user patricker commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2050#discussion_r132612494
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java
 ---
@@ -259,6 +261,11 @@ public void onTrigger(final ProcessContext context, 
final ProcessSessionFactory
 }
 });
 
+if(customWhereClause != null) {
+// adding the custom WHERE clause (if defined) to the list 
of existing clauses.
+maxValueClauses.add(customWhereClause);
--- End diff --

What do you think about forcing custom where clauses to be surrounded with 
parenthesis?
I worry that just putting it in like this may cause unexpected consequences 
if the where clauses includes OR's.


> Allow a custom WHERE clause in AbstractDatabaseFetchProcessor
> -
>
> Key: NIFI-4257
> URL: https://issues.apache.org/jira/browse/NIFI-4257
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Pierre Villard
>Assignee: Pierre Villard
>
> It could be useful allowing a user to set a custom WHERE clause in 
> AbstractDatabaseFetchProcessor in case not all of the data in the table is 
> required.
> In case the WHERE clause is changed after the processor has already been 
> running, the user will probably have to set the initial maximum values to 
> ensure the expected behaviour.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2050: NIFI-4257 - add custom WHERE clause in database fet...

[patricker]

Github user patricker commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2050#discussion_r132612494
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java
 ---
@@ -259,6 +261,11 @@ public void onTrigger(final ProcessContext context, 
final ProcessSessionFactory
 }
 });
 
+if(customWhereClause != null) {
+// adding the custom WHERE clause (if defined) to the list 
of existing clauses.
+maxValueClauses.add(customWhereClause);
--- End diff --

What do you think about forcing custom where clauses to be surrounded with 
parenthesis?
I worry that just putting it in like this may cause unexpected consequences 
if the where clauses includes OR's.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4279) PutDataBaseRecord steam has already been closed

[Peter Wicks (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122736#comment-16122736
 ] 

Peter Wicks commented on NIFI-4279:
---

Usually when I see this error it's not because of the processor but because of 
the DBCP Connection Pool losing a connection but not recognizing it. It then 
hands a closed connection to the processor and an error like this sometimes 
gets closed (also I've seen 'Closed Pipe').

This may not be your issue, but to check you could try adding a Validation 
Query to your Oracle connection if you don't already have one.

> PutDataBaseRecord steam has already been closed
> ---
>
> Key: NIFI-4279
> URL: https://issues.apache.org/jira/browse/NIFI-4279
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.3.0
> Environment: we have used nifi on openshift cluster and standalone on 
> a linux machine.
> Oracle database with version 11.2.0.2
> linux with rhel 7.0
>Reporter: simon water
>Priority: Minor
>  Labels: database, nifi, patch, processor
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> we have created a table in an Oracle Database with a default sysdate column.
> when i restarted the processor it started throwing the stream has already 
> been closed exception
> i have used a csv file with the headers inside of it
> csv Reader that reads the column names from the csv
> the data base version is 11.2.0 Oracle database



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4279) PutDataBaseRecord steam has already been closed

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-4279:
--
Flags:   (was: Patch)

> PutDataBaseRecord steam has already been closed
> ---
>
> Key: NIFI-4279
> URL: https://issues.apache.org/jira/browse/NIFI-4279
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.3.0
> Environment: we have used nifi on openshift cluster and standalone on 
> a linux machine.
> Oracle database with version 11.2.0.2
> linux with rhel 7.0
>Reporter: simon water
>Priority: Minor
>  Labels: database, nifi, patch, processor
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> we have created a table in an Oracle Database with a default sysdate column.
> when i restarted the processor it started throwing the stream has already 
> been closed exception
> i have used a csv file with the headers inside of it
> csv Reader that reads the column names from the csv
> the data base version is 11.2.0 Oracle database



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4250) Create support for deleting document by id from elasticsearch 5

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122731#comment-16122731
 ] 

Joseph Witt commented on NIFI-4250:
---

Fix version can be set once contrib/review cycle is being merged or very near 
to merge. 

Thanks [~mans2singh] just a matter of finding a reviewer that can help.

> Create support for deleting document by id from elasticsearch 5
> ---
>
> Key: NIFI-4250
> URL: https://issues.apache.org/jira/browse/NIFI-4250
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Mans Singh
>Assignee: Mans Singh
>Priority: Minor
>  Labels: delete, elasticsearch
>
> Create a processor to delete documents from elasticsearch 5 based on document 
> id.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4250) Create support for deleting document by id from elasticsearch 5

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-4250:
--
Fix Version/s: (was: 1.4.0)

> Create support for deleting document by id from elasticsearch 5
> ---
>
> Key: NIFI-4250
> URL: https://issues.apache.org/jira/browse/NIFI-4250
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Mans Singh
>Assignee: Mans Singh
>Priority: Minor
>  Labels: delete, elasticsearch
>
> Create a processor to delete documents from elasticsearch 5 based on document 
> id.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4279) PutDataBaseRecord steam has already been closed

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122732#comment-16122732
 ] 

Joseph Witt commented on NIFI-4279:
---

Fix version can be set once contrib/review cycle is being merged or very near 
to merge. 

> PutDataBaseRecord steam has already been closed
> ---
>
> Key: NIFI-4279
> URL: https://issues.apache.org/jira/browse/NIFI-4279
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.3.0
> Environment: we have used nifi on openshift cluster and standalone on 
> a linux machine.
> Oracle database with version 11.2.0.2
> linux with rhel 7.0
>Reporter: simon water
>Priority: Minor
>  Labels: database, nifi, patch, processor
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> we have created a table in an Oracle Database with a default sysdate column.
> when i restarted the processor it started throwing the stream has already 
> been closed exception
> i have used a csv file with the headers inside of it
> csv Reader that reads the column names from the csv
> the data base version is 11.2.0 Oracle database



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4279) PutDataBaseRecord steam has already been closed

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-4279:
--
Fix Version/s: (was: 1.4.0)

> PutDataBaseRecord steam has already been closed
> ---
>
> Key: NIFI-4279
> URL: https://issues.apache.org/jira/browse/NIFI-4279
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.3.0
> Environment: we have used nifi on openshift cluster and standalone on 
> a linux machine.
> Oracle database with version 11.2.0.2
> linux with rhel 7.0
>Reporter: simon water
>Priority: Minor
>  Labels: database, nifi, patch, processor
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> we have created a table in an Oracle Database with a default sysdate column.
> when i restarted the processor it started throwing the stream has already 
> been closed exception
> i have used a csv file with the headers inside of it
> csv Reader that reads the column names from the csv
> the data base version is 11.2.0 Oracle database



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4237) EncryptionOperationNotPossibleException in nifi-bootstrap.log might suggest underlying cause

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-4237:
--
Fix Version/s: (was: 1.4.0)

> EncryptionOperationNotPossibleException in nifi-bootstrap.log might suggest 
> underlying cause
> 
>
> Key: NIFI-4237
> URL: https://issues.apache.org/jira/browse/NIFI-4237
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Russell Bateman
>Priority: Minor
>
> Our Ansible instructions upgraded NiFi and created a new 
> {{nifi.sensitive.props.key}}. In _nifi.properties_ this property, if extant, 
> is used to encrypt sensitive properties in _flow.xml.gz_. Thus, upon 
> relaunching NiFi, the wrong key was used to decrypt resulting in the reported 
> failure to start, _flow.xml.gz_ is no longer useful.
> We found the problem and fixed it after Mark Payne suggested a possible 
> cause, but if this state of things can be determined, it might save on 
> community support for this situation if the logged message were to suggest 
> what's at the bottom of this problem. The top of the stack trace appears in 
> _logs/nifi-bootstrap.log_ as below:
> 2017-07-25 23:23:31,148 WARN [main] org.apache.nifi.web.server.JettyServer
> Failed to start web server... shutting down.
> org.apache.nifi.encrypt.EncryptionException:
> org.jasypt.exceptions.EncryptionOperationNotPossibleException
> at
> org.apache.nifi.encrypt.StringEncryptor.decrypt(StringEncryptor.java:149)
> ~[nifi-framework-core-1.1.2.jar:1.1.2]
> at
> org.apache.nifi.controller.serialization.FlowFromDOMFactory.decrypt(FlowFromDOMFactory.java:474)
> ~[nifi-framework-core-1.1.2.jar:1.1.2]
> at...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4256) Add support for all AWS S3 Encryption Options

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122727#comment-16122727
 ] 

Joseph Witt commented on NIFI-4256:
---

Fix version can be set once contrib/review cycle is being merged or very near 
to merge. 

> Add support for all AWS S3 Encryption Options
> -
>
> Key: NIFI-4256
> URL: https://issues.apache.org/jira/browse/NIFI-4256
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 1.2.0
>Reporter: Franco
>  Labels: aws, aws-s3, security
>
> NiFi currently only supports SSE-S3 encryption (AES256).
> Support needs to be added for:
> * SSE-S3
> * SSE-KMS
> * SSE-C
> * CSE-KMS CMK
> * CSE-Master Key
> With all of the appropriate configuration options and such that SSE is 
> available only for PutS3Object whilst CSE is available also for FetchS3Object.
> Given that this will add another 20 or so UI properties the intention is to 
> split it into a Client Side Encryption Service and Server Side Encryption 
> Service. This will allow users to reuse "encryption" across different 
> workflows.
> Existing flows using the Server Side Encryption option will still work as is 
> but will be overridden if a service is added.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4237) EncryptionOperationNotPossibleException in nifi-bootstrap.log might suggest underlying cause

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122728#comment-16122728
 ] 

Joseph Witt commented on NIFI-4237:
---

Fix version can be set once contrib/review cycle is being merged or very near 
to merge. 

> EncryptionOperationNotPossibleException in nifi-bootstrap.log might suggest 
> underlying cause
> 
>
> Key: NIFI-4237
> URL: https://issues.apache.org/jira/browse/NIFI-4237
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Russell Bateman
>Priority: Minor
>
> Our Ansible instructions upgraded NiFi and created a new 
> {{nifi.sensitive.props.key}}. In _nifi.properties_ this property, if extant, 
> is used to encrypt sensitive properties in _flow.xml.gz_. Thus, upon 
> relaunching NiFi, the wrong key was used to decrypt resulting in the reported 
> failure to start, _flow.xml.gz_ is no longer useful.
> We found the problem and fixed it after Mark Payne suggested a possible 
> cause, but if this state of things can be determined, it might save on 
> community support for this situation if the logged message were to suggest 
> what's at the bottom of this problem. The top of the stack trace appears in 
> _logs/nifi-bootstrap.log_ as below:
> 2017-07-25 23:23:31,148 WARN [main] org.apache.nifi.web.server.JettyServer
> Failed to start web server... shutting down.
> org.apache.nifi.encrypt.EncryptionException:
> org.jasypt.exceptions.EncryptionOperationNotPossibleException
> at
> org.apache.nifi.encrypt.StringEncryptor.decrypt(StringEncryptor.java:149)
> ~[nifi-framework-core-1.1.2.jar:1.1.2]
> at
> org.apache.nifi.controller.serialization.FlowFromDOMFactory.decrypt(FlowFromDOMFactory.java:474)
> ~[nifi-framework-core-1.1.2.jar:1.1.2]
> at...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4256) Add support for all AWS S3 Encryption Options

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-4256:
--
Fix Version/s: (was: 1.4.0)

> Add support for all AWS S3 Encryption Options
> -
>
> Key: NIFI-4256
> URL: https://issues.apache.org/jira/browse/NIFI-4256
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 1.2.0
>Reporter: Franco
>  Labels: aws, aws-s3, security
>
> NiFi currently only supports SSE-S3 encryption (AES256).
> Support needs to be added for:
> * SSE-S3
> * SSE-KMS
> * SSE-C
> * CSE-KMS CMK
> * CSE-Master Key
> With all of the appropriate configuration options and such that SSE is 
> available only for PutS3Object whilst CSE is available also for FetchS3Object.
> Given that this will add another 20 or so UI properties the intention is to 
> split it into a Client Side Encryption Service and Server Side Encryption 
> Service. This will allow users to reuse "encryption" across different 
> workflows.
> Existing flows using the Server Side Encryption option will still work as is 
> but will be overridden if a service is added.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4177) MergeContent - Tar - Save modification timestamp like Tar does

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-4177:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> MergeContent - Tar - Save modification timestamp like Tar does
> --
>
> Key: NIFI-4177
> URL: https://issues.apache.org/jira/browse/NIFI-4177
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Wayne Steel
>Assignee: Joseph Witt
>Priority: Trivial
> Fix For: 1.4.0
>
>
> Tar by default saves the modification timestamp of entries.
> This mainly affects file based entries so could be done on reading the 
> attribute file.lastModifiedTime, if it exists, which is written to the 
> flowfile by GetFile or ListFile processors.
> Otherwise just leave it out as it does now.
> I propose a property with the default expression ${file.lastModifiedTime} but 
> the value must resolve to a date format of "-MM-dd'T'HH:mm:ssZ". It 
> should only be enabled when MERGE_FORMAT is set to MERGE_FORMAT_TAR



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4177) MergeContent - Tar - Save modification timestamp like Tar does

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122721#comment-16122721
 ] 

ASF subversion and git services commented on NIFI-4177:
---

Commit a6e8f0afe3407bf668e234dcce3c474f1ceb86c8 in nifi's branch 
refs/heads/master from [~makosteel]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=a6e8f0a ]

NIFI-4177 This closes #2002. MergeContent - Tar - Save modification timestamp 
like Tar does

Signed-off-by: joewitt 


> MergeContent - Tar - Save modification timestamp like Tar does
> --
>
> Key: NIFI-4177
> URL: https://issues.apache.org/jira/browse/NIFI-4177
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Wayne Steel
>Assignee: Joseph Witt
>Priority: Trivial
> Fix For: 1.4.0
>
>
> Tar by default saves the modification timestamp of entries.
> This mainly affects file based entries so could be done on reading the 
> attribute file.lastModifiedTime, if it exists, which is written to the 
> flowfile by GetFile or ListFile processors.
> Otherwise just leave it out as it does now.
> I propose a property with the default expression ${file.lastModifiedTime} but 
> the value must resolve to a date format of "-MM-dd'T'HH:mm:ssZ". It 
> should only be enabled when MERGE_FORMAT is set to MERGE_FORMAT_TAR



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4177) MergeContent - Tar - Save modification timestamp like Tar does

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122722#comment-16122722
 ] 

ASF GitHub Bot commented on NIFI-4177:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2002


> MergeContent - Tar - Save modification timestamp like Tar does
> --
>
> Key: NIFI-4177
> URL: https://issues.apache.org/jira/browse/NIFI-4177
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Wayne Steel
>Assignee: Joseph Witt
>Priority: Trivial
> Fix For: 1.4.0
>
>
> Tar by default saves the modification timestamp of entries.
> This mainly affects file based entries so could be done on reading the 
> attribute file.lastModifiedTime, if it exists, which is written to the 
> flowfile by GetFile or ListFile processors.
> Otherwise just leave it out as it does now.
> I propose a property with the default expression ${file.lastModifiedTime} but 
> the value must resolve to a date format of "-MM-dd'T'HH:mm:ssZ". It 
> should only be enabled when MERGE_FORMAT is set to MERGE_FORMAT_TAR



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2002: NIFI-4177 MergeContent - Tar - Save modification ti...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2002


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4250) Create support for deleting document by id from elasticsearch 5

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122716#comment-16122716
 ] 

ASF GitHub Bot commented on NIFI-4250:
--

Github user mans2singh commented on the issue:

https://github.com/apache/nifi/pull/2045
  
Hey Folks:

Let me know if you have any feedback/recommends for this processor.

Thanks

Mans


> Create support for deleting document by id from elasticsearch 5
> ---
>
> Key: NIFI-4250
> URL: https://issues.apache.org/jira/browse/NIFI-4250
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Mans Singh
>Assignee: Mans Singh
>Priority: Minor
>  Labels: delete, elasticsearch
> Fix For: 1.4.0
>
>
> Create a processor to delete documents from elasticsearch 5 based on document 
> id.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2045: NIFI-4250 - Elasticsearch 5 delete processor

[mans2singh]

Github user mans2singh commented on the issue:

https://github.com/apache/nifi/pull/2045
  
Hey Folks:

Let me know if you have any feedback/recommends for this processor.

Thanks

Mans


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Assigned] (NIFI-4177) MergeContent - Tar - Save modification timestamp like Tar does

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt reassigned NIFI-4177:
-

Assignee: Joseph Witt

> MergeContent - Tar - Save modification timestamp like Tar does
> --
>
> Key: NIFI-4177
> URL: https://issues.apache.org/jira/browse/NIFI-4177
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Wayne Steel
>Assignee: Joseph Witt
>Priority: Trivial
> Fix For: 1.4.0
>
>
> Tar by default saves the modification timestamp of entries.
> This mainly affects file based entries so could be done on reading the 
> attribute file.lastModifiedTime, if it exists, which is written to the 
> flowfile by GetFile or ListFile processors.
> Otherwise just leave it out as it does now.
> I propose a property with the default expression ${file.lastModifiedTime} but 
> the value must resolve to a date format of "-MM-dd'T'HH:mm:ssZ". It 
> should only be enabled when MERGE_FORMAT is set to MERGE_FORMAT_TAR



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2057: NIFI-2167 - include disabled state for processors when cop...

[alopresto]

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2057
  
Created two processors and disabled them. Verified that copy/paste and 
creating a template & instantiating it maintain the `disabled` state. 

Ran `contrib-check` and all tests pass. +1, merging. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2167) Disabled state not including in a Processor that was copy/paste

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122667#comment-16122667
 ] 

ASF GitHub Bot commented on NIFI-2167:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2057


> Disabled state not including in a Processor that was copy/paste
> ---
>
> Key: NIFI-2167
> URL: https://issues.apache.org/jira/browse/NIFI-2167
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Matt Gilman
>Assignee: Pierre Villard
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2057: NIFI-2167 - include disabled state for processors w...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2057


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2167) Disabled state not including in a Processor that was copy/paste

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122663#comment-16122663
 ] 

ASF GitHub Bot commented on NIFI-2167:
--

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2057
  
Reviewing...


> Disabled state not including in a Processor that was copy/paste
> ---
>
> Key: NIFI-2167
> URL: https://issues.apache.org/jira/browse/NIFI-2167
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Reporter: Matt Gilman
>Assignee: Pierre Villard
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2015: NIFI-4142: Refactored Record Reader/Writer to allow for re...

[joewitt]

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2015
  
@markap14 can you please rebase and resolve conflict and please go ahead 
and squash.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4142) Implement a ValidateRecord Processor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122660#comment-16122660
 ] 

ASF GitHub Bot commented on NIFI-4142:
--

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2015
  
@markap14 can you please rebase and resolve conflict and please go ahead 
and squash.


> Implement a ValidateRecord Processor
> 
>
> Key: NIFI-4142
> URL: https://issues.apache.org/jira/browse/NIFI-4142
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.4.0
>
>
> We need a processor that is capable of validating that all Records in a 
> FlowFile adhere to the proper schema.
> The Processor should be configured with a Record Reader and should route each 
> record to either 'valid' or 'invalid' based on whether or not the record 
> adheres to the reader's schema. A record would be invalid in any of the 
> following cases:
> - Missing field that is required according to the schema
> - Extra field that is not present in schema (it should be configurable 
> whether or not this is a failure)
> - Field requires coercion and strict type checking enabled (this should also 
> be configurable)
> - Field is invalid, such as the value "hello" when it should be an integer



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3335) GenerateTableFetch should allow you to specify an initial Max Value

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122657#comment-16122657
 ] 

Joseph Witt commented on NIFI-3335:
---

Ok it appears the db for JIRA for this ticket is wacky.  Can be fixed if done 
by an admin.  Rather than bothering ASF admins just cloned it and ported the 
history/log there.  Will still be searchable by those comments.  Deleting this 
JIRA.

> GenerateTableFetch should allow you to specify an initial Max Value
> ---
>
> Key: NIFI-3335
> URL: https://issues.apache.org/jira/browse/NIFI-3335
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Matt Burgess
>Assignee: Joseph Witt
>
> NIFI-2583 added the ability (via dynamic properties) to specify initial Max 
> Values for columns, to enable the user to "pick up where they left off" if 
> something happened with a flow, a NiFi instance, etc. where the state was 
> stored but the processing did not complete successfully.
> This feature would also be helpful in GenerateTableFetch, which also supports 
> max-value columns.
> Since NIFI-2881 adds incoming flow file support, it's more useful if Initial 
> max values can be specified via flow file attributes. Because if a table name 
> is dynamically passed via flow file attribute and Expression Language, user 
> won't be able to configure dynamic processor attribute in advance for each 
> possible table.
> Add dynamic properties ('initial.maxvalue.' same as 
> QueryDatabaseTable) to specify initial max values statically, and also use 
> incoming flow file attributes named 'initial.maxvalue.' if 
> any. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Deleted] (NIFI-3335) GenerateTableFetch should allow you to specify an initial Max Value

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt deleted NIFI-3335:
--


> GenerateTableFetch should allow you to specify an initial Max Value
> ---
>
> Key: NIFI-3335
> URL: https://issues.apache.org/jira/browse/NIFI-3335
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Matt Burgess
>Assignee: Joseph Witt
>
> NIFI-2583 added the ability (via dynamic properties) to specify initial Max 
> Values for columns, to enable the user to "pick up where they left off" if 
> something happened with a flow, a NiFi instance, etc. where the state was 
> stored but the processing did not complete successfully.
> This feature would also be helpful in GenerateTableFetch, which also supports 
> max-value columns.
> Since NIFI-2881 adds incoming flow file support, it's more useful if Initial 
> max values can be specified via flow file attributes. Because if a table name 
> is dynamically passed via flow file attribute and Expression Language, user 
> won't be able to configure dynamic processor attribute in advance for each 
> possible table.
> Add dynamic properties ('initial.maxvalue.' same as 
> QueryDatabaseTable) to specify initial max values statically, and also use 
> incoming flow file attributes named 'initial.maxvalue.' if 
> any. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4283) GenerateTableFetch should allow you to specify an initial Max Value (clone of defunct NIFI-3335)

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122656#comment-16122656
 ] 

Joseph Witt commented on NIFI-4283:
---

NIFI-3335 got into a funky state that probably would have been solved by asking 
for integrity checker run against NIFI.  Instead just cloning and deleting old 
ticket.  History from NIFI-3335 is here

Back to previous view
[NIFI-3335] GenerateTableFetch should allow you to specify an initial Max Value 
Created: 12/Jan/17  Updated: 11/Aug/17
Status: Open
Project:Apache NiFi
Component/s:Extensions
Affects Version/s:  None
Fix Version/s:  1.4.0

Type:   Improvement Priority:   Major
Reporter:   Matt BurgessAssignee:   Joseph Witt
Resolution: Unresolved  Votes:  0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate:  Not Specified


 Description 
NIFI-2583 added the ability (via dynamic properties) to specify initial Max 
Values for columns, to enable the user to "pick up where they left off" if 
something happened with a flow, a NiFi instance, etc. where the state was 
stored but the processing did not complete successfully.
This feature would also be helpful in GenerateTableFetch, which also supports 
max-value columns.
Since NIFI-2881 adds incoming flow file support, it's more useful if Initial 
max values can be specified via flow file attributes. Because if a table name 
is dynamically passed via flow file attribute and Expression Language, user 
won't be able to configure dynamic processor attribute in advance for each 
possible table.
Add dynamic properties ('initial.maxvalue.' same as 
QueryDatabaseTable) to specify initial max values statically, and also use 
incoming flow file attributes named 'initial.maxvalue.' if 
any.


 Comments
Comment by Matt Burgess [ 17/Jul/17 ]
Peter Wicks Are you still working on this? If not, do you mind if I unassign 
it? Thanks!
Comment by Peter Wicks [ 19/Jul/17 ]
Matt Burgess I'm not working on this. I've unassigned myself.
Comment by ASF GitHub Bot [ 25/Jul/17 ]
GitHub user mattyb149 opened a pull request:
https://github.com/apache/nifi/pull/2039
NIFI-3335: Add initial.maxvalue support to GenerateTableFetch
Moved the common code back to the abstract base class
For all changes:
[x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
[x] Does your PR title start with NIFI- where  is the JIRA number you 
are trying to resolve? Pay particular attention to the hyphen "-" character.
[x] Has your PR been rebased against the latest commit within the target branch 
(typically master)?
[x] Is your initial contribution a single, squashed commit?
For code changes:
[x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
[x] Have you written or updated unit tests to verify your changes?
[ ] If adding new dependencies to the code, are these dependencies licensed in 
a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)?
[ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE 
file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name 
(programmatic access) for each of the new properties?
For documentation related changes:
[x] Have you ensured that format looks appropriate for the output in which it 
is rendered?
Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mattyb149/nifi NIFI-3335
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/2039.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2039
commit 8eea477c02fdb2ca19ec77da9168ceec5e8eae7c
Author: Matt Burgess 
Date: 2017-07-25T20:07:52Z
NIFI-3335: Add initial.maxvalue support to GenerateTableFetch
Comment by ASF GitHub Bot [ 26/Jul/17 ]
Github user pvillard31 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2039#discussion_r129494131
— Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java
 —
@@ -198,6 +201,25 @@ public void onTrigger(final ProcessContext context, final 
ProcessSessionFactory
// set as the current state map (after the session has been committed)
final Map statePropertyMap = new HashMap<>(stateMap.toMap());
+ // If an initial max value for column(s) has 

[jira] [Created] (NIFI-4283) GenerateTableFetch should allow you to specify an initial Max Value (clone of defunct NIFI-3335)

[Joseph Witt (JIRA)]

Joseph Witt created NIFI-4283:
-

 Summary: GenerateTableFetch should allow you to specify an initial 
Max Value (clone of defunct NIFI-3335)
 Key: NIFI-4283
 URL: https://issues.apache.org/jira/browse/NIFI-4283
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Extensions
Reporter: Matt Burgess
Assignee: Joseph Witt
 Fix For: 1.4.0


NIFI-2583 added the ability (via dynamic properties) to specify initial Max 
Values for columns, to enable the user to "pick up where they left off" if 
something happened with a flow, a NiFi instance, etc. where the state was 
stored but the processing did not complete successfully.

This feature would also be helpful in GenerateTableFetch, which also supports 
max-value columns.

Since NIFI-2881 adds incoming flow file support, it's more useful if Initial 
max values can be specified via flow file attributes. Because if a table name 
is dynamically passed via flow file attribute and Expression Language, user 
won't be able to configure dynamic processor attribute in advance for each 
possible table.

Add dynamic properties ('initial.maxvalue.' same as 
QueryDatabaseTable) to specify initial max values statically, and also use 
incoming flow file attributes named 'initial.maxvalue.' if 
any. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-3335) GenerateTableFetch should allow you to specify an initial Max Value

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-3335:
--
Component/s: (was: Extensions)

> GenerateTableFetch should allow you to specify an initial Max Value
> ---
>
> Key: NIFI-3335
> URL: https://issues.apache.org/jira/browse/NIFI-3335
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Matt Burgess
>Assignee: Joseph Witt
> Fix For: 1.4.0
>
>
> NIFI-2583 added the ability (via dynamic properties) to specify initial Max 
> Values for columns, to enable the user to "pick up where they left off" if 
> something happened with a flow, a NiFi instance, etc. where the state was 
> stored but the processing did not complete successfully.
> This feature would also be helpful in GenerateTableFetch, which also supports 
> max-value columns.
> Since NIFI-2881 adds incoming flow file support, it's more useful if Initial 
> max values can be specified via flow file attributes. Because if a table name 
> is dynamically passed via flow file attribute and Expression Language, user 
> won't be able to configure dynamic processor attribute in advance for each 
> possible table.
> Add dynamic properties ('initial.maxvalue.' same as 
> QueryDatabaseTable) to specify initial max values statically, and also use 
> incoming flow file attributes named 'initial.maxvalue.' if 
> any. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (NIFI-4282) GenerateTableFetch should allow you to specify an initial Max Value (cloned)

[Joseph Witt (JIRA)]

Joseph Witt created NIFI-4282:
-

 Summary: GenerateTableFetch should allow you to specify an initial 
Max Value (cloned)
 Key: NIFI-4282
 URL: https://issues.apache.org/jira/browse/NIFI-4282
 Project: Apache NiFi
  Issue Type: Task
  Components: Extensions
Reporter: Matt Burgess
Assignee: Matt Burgess
 Fix For: 1.4.0


NIFI-2583 added the ability (via dynamic properties) to specify initial Max 
Values for columns, to enable the user to "pick up where they left off" if 
something happened with a flow, a NiFi instance, etc. where the state was 
stored but the processing did not complete successfully.

This feature would also be helpful in GenerateTableFetch, which also supports 
max-value columns.

Since NIFI-2881 adds incoming flow file support, it's more useful if Initial 
max values can be specified via flow file attributes. Because if a table name 
is dynamically passed via flow file attribute and Expression Language, user 
won't be able to configure dynamic processor attribute in advance for each 
possible table.

Add dynamic properties ('initial.maxvalue.' same as 
QueryDatabaseTable) to specify initial max values statically, and also use 
incoming flow file attributes named 'initial.maxvalue.' if 
any. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Deleted] (NIFI-4282) GenerateTableFetch should allow you to specify an initial Max Value (cloned)

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt deleted NIFI-4282:
--


> GenerateTableFetch should allow you to specify an initial Max Value (cloned)
> 
>
> Key: NIFI-4282
> URL: https://issues.apache.org/jira/browse/NIFI-4282
> Project: Apache NiFi
>  Issue Type: Task
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>
> NIFI-2583 added the ability (via dynamic properties) to specify initial Max 
> Values for columns, to enable the user to "pick up where they left off" if 
> something happened with a flow, a NiFi instance, etc. where the state was 
> stored but the processing did not complete successfully.
> This feature would also be helpful in GenerateTableFetch, which also supports 
> max-value columns.
> Since NIFI-2881 adds incoming flow file support, it's more useful if Initial 
> max values can be specified via flow file attributes. Because if a table name 
> is dynamically passed via flow file attribute and Expression Language, user 
> won't be able to configure dynamic processor attribute in advance for each 
> possible table.
> Add dynamic properties ('initial.maxvalue.' same as 
> QueryDatabaseTable) to specify initial max values statically, and also use 
> incoming flow file attributes named 'initial.maxvalue.' if 
> any. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-3335) GenerateTableFetch should allow you to specify an initial Max Value

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-3335:
--
Issue Type: Task  (was: Improvement)

> GenerateTableFetch should allow you to specify an initial Max Value
> ---
>
> Key: NIFI-3335
> URL: https://issues.apache.org/jira/browse/NIFI-3335
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
> Fix For: 1.4.0
>
>
> NIFI-2583 added the ability (via dynamic properties) to specify initial Max 
> Values for columns, to enable the user to "pick up where they left off" if 
> something happened with a flow, a NiFi instance, etc. where the state was 
> stored but the processing did not complete successfully.
> This feature would also be helpful in GenerateTableFetch, which also supports 
> max-value columns.
> Since NIFI-2881 adds incoming flow file support, it's more useful if Initial 
> max values can be specified via flow file attributes. Because if a table name 
> is dynamically passed via flow file attribute and Expression Language, user 
> won't be able to configure dynamic processor attribute in advance for each 
> possible table.
> Add dynamic properties ('initial.maxvalue.' same as 
> QueryDatabaseTable) to specify initial max values statically, and also use 
> incoming flow file attributes named 'initial.maxvalue.' if 
> any. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-3329) MergeContent throws FlowFileHandlingException: not the most recent version of this FlowFile within this session

[Joseph Witt (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Witt updated NIFI-3329:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

+1 merged to master.  Forgot to do signoff commit entry.

> MergeContent throws FlowFileHandlingException: not the most recent version of 
> this FlowFile within this session
> ---
>
> Key: NIFI-3329
> URL: https://issues.apache.org/jira/browse/NIFI-3329
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.1.1
>Reporter: Mark Payne
>Assignee: Mark Payne
> Fix For: 1.4.0
>
> Attachments: simple_flow.xml
>
>
> The following error was reported on the dev mailing list:
> {code}
> This error occurred in NiFi 1.1.1 (single node) right after an upgrade from
> 1.1.0, so I thought it was due to the upgrade, but now I see that it happens
> if I restart as well and there is data in the flow. 
> This MergeContent processor is sorting incoming FlowFiles into bins based on
> a correlation attribute and dumping out the accumulated uber-FlowFile after
> one hour or 500K messages are accumulated.
> ...
> 2017-01-11 10:53:39,561 WARN [Timer-Driven Process Thread-9]
> o.a.n.processors.standard.MergeContent
> MergeContent[id=b9898788-1c11-45d9-8646-44b1dceb92d5] Processor
> Administratively Yielded for 1 sec due to processing failure
> 2017-01-11 10:53:39,561 WARN [Timer-Driven Process Thread-9]
> o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding
> MergeContent[id=b9898788-1c11-45d9-8646-44b1dceb92d5] due to uncaught
> Exception: org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=07f3d75a-3347-45da-8dd4-fecf775d1dcf,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1484151577298-1971,
> container=default, section=947], offset=566526,
> length=64],offset=0,name=8551433629757608.tsv,size=64] is not the most
> recent version of this FlowFile within this session
> (StandardProcessSession[id=133767])
> 2017-01-11 10:53:39,563 WARN [Timer-Driven Process Thread-9]
> o.a.n.c.t.ContinuallyRunProcessorTask
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=07f3d75a-3347-45da-8dd4-fecf775d1dcf,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1484151577298-1971,
> container=default, section=947], offset=566526,
> length=64],offset=0,name=8551433629757608.tsv,size=64] is not the most
> recent version of this FlowFile within this session
> (StandardProcessSession[id=133767])
>at
> org.apache.nifi.controller.repository.StandardProcessSession.migrate(StandardProcessSession.java:1121)
> ~[nifi-framework-core-1.1.0.jar:1.1.0]
>at
> org.apache.nifi.controller.repository.StandardProcessSession.migrate(StandardProcessSession.java:1102)
> ~[nifi-framework-core-1.1.0.jar:1.1.0]
>at org.apache.nifi.processor.util.bin.Bin.offer(Bin.java:142)
> ~[na:na]
>at
> org.apache.nifi.processor.util.bin.BinManager.offer(BinManager.java:194)
> ~[na:na]
>at
> org.apache.nifi.processor.util.bin.BinFiles.binFlowFiles(BinFiles.java:279)
> ~[na:na]
>at
> org.apache.nifi.processor.util.bin.BinFiles.onTrigger(BinFiles.java:178)
> ~[na:na]
>at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099)
> ~[nifi-framework-core-1.1.0.jar:1.1.0]
>at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
> [nifi-framework-core-1.1.0.jar:1.1.0]
>at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> [nifi-framework-core-1.1.0.jar:1.1.0]
>at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
> [nifi-framework-core-1.1.0.jar:1.1.0]
>at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_65]
>at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_65]
>at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_65]
>at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_65]
>at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_65]
>at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_65]
>at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2053: NIFI-3329: Removed check for latest version of flow...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2053


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4208) Node failed to join cluster due to NullPointerException

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122601#comment-16122601
 ] 

Joseph Witt commented on NIFI-4208:
---

[~markap14] Any more thoughts/plans on this?  I see it is tagged to 1.4.0 and 
said to be critical but no update since then.  

> Node failed to join cluster due to NullPointerException
> ---
>
> Key: NIFI-4208
> URL: https://issues.apache.org/jira/browse/NIFI-4208
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Critical
> Fix For: 1.4.0
>
>
> A clustered node ran out of disk space. Upon restart, I came across the 
> following error:
> 2017-07-20 09:03:00,988 ERROR [main] o.a.nifi.controller.StandardFlowService 
> Failed to load flow from cluster due to: 
> org.apache.nifi.cluster.ConnectionException: Failed to connect node to 
> cluster due to: java.lang.NullPointerException
> org.apache.nifi.cluster.ConnectionException: Failed to connect node to 
> cluster due to: java.lang.NullPointerException
>   at 
> org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:945)
>   at 
> org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:515)
>   at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:800)
>   at org.apache.nifi.NiFi.(NiFi.java:160)
>   at org.apache.nifi.NiFi.main(NiFi.java:267)
> Caused by: java.lang.NullPointerException: null
>   at 
> org.apache.nifi.controller.repository.RepositoryRecordSerde.getRecordIdentifier(RepositoryRecordSerde.java:43)
>   at 
> org.apache.nifi.controller.repository.RepositoryRecordSerde.getRecordIdentifier(RepositoryRecordSerde.java:26)
>   at 
> org.wali.MinimalLockingWriteAheadLog$Partition.recoverNextTransaction(MinimalLockingWriteAheadLog.java:1132)
>   at 
> org.wali.MinimalLockingWriteAheadLog.recoverFromEdits(MinimalLockingWriteAheadLog.java:459)
>   at 
> org.wali.MinimalLockingWriteAheadLog.recoverRecords(MinimalLockingWriteAheadLog.java:301)
>   at 
> org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.loadFlowFiles(WriteAheadFlowFileRepository.java:381)
>   at 
> org.apache.nifi.controller.FlowController.initializeFlow(FlowController.java:713)
>   at 
> org.apache.nifi.controller.StandardFlowService.initializeController(StandardFlowService.java:955)
>   at 
> org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:927)
>   ... 4 common frames omitted



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4206) Include proxy instructions in admin guide

[Andy LoPresto (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy LoPresto updated NIFI-4206:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Include proxy instructions in admin guide
> -
>
> Key: NIFI-4206
> URL: https://issues.apache.org/jira/browse/NIFI-4206
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Documentation & Website
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Minor
> Fix For: 1.4.0
>
>
> Update the admin guide with instructions when running behind a proxy.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4206) Include proxy instructions in admin guide

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122575#comment-16122575
 ] 

ASF GitHub Bot commented on NIFI-4206:
--

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2023
  
I fixed a few formatting issues (see 
[e091b8f](https://github.com/alopresto/nifi/commit/e091b8fff5e8a6090b07cd8084bf4f7f76bc9424))
 but other than that it looks good to me. Ran `contrib-check` and all tests 
pass. +1, merging. 


> Include proxy instructions in admin guide
> -
>
> Key: NIFI-4206
> URL: https://issues.apache.org/jira/browse/NIFI-4206
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Documentation & Website
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Minor
> Fix For: 1.4.0
>
>
> Update the admin guide with instructions when running behind a proxy.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2023: NIFI-4206: Proxy instructions in Admin Guide

[alopresto]

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2023
  
I fixed a few formatting issues (see 
[e091b8f](https://github.com/alopresto/nifi/commit/e091b8fff5e8a6090b07cd8084bf4f7f76bc9424))
 but other than that it looks good to me. Ran `contrib-check` and all tests 
pass. +1, merging. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4206) Include proxy instructions in admin guide

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122572#comment-16122572
 ] 

ASF subversion and git services commented on NIFI-4206:
---

Commit ec0c8278b42916637c7ce76b05aff3f35a77ece8 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=ec0c827 ]

NIFI-4206:
- Updating admin guide to include instructions for running NiFi behind a proxy.
- Including a brief example proxy configuration for NiFi specific properties.

This closes #2023.

Signed-off-by: Andy LoPresto 


> Include proxy instructions in admin guide
> -
>
> Key: NIFI-4206
> URL: https://issues.apache.org/jira/browse/NIFI-4206
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Documentation & Website
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Minor
> Fix For: 1.4.0
>
>
> Update the admin guide with instructions when running behind a proxy.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4206) Include proxy instructions in admin guide

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122573#comment-16122573
 ] 

ASF GitHub Bot commented on NIFI-4206:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2023


> Include proxy instructions in admin guide
> -
>
> Key: NIFI-4206
> URL: https://issues.apache.org/jira/browse/NIFI-4206
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Documentation & Website
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Minor
> Fix For: 1.4.0
>
>
> Update the admin guide with instructions when running behind a proxy.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2023: NIFI-4206: Proxy instructions in Admin Guide

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2023


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-4187) If NiFi process is killed, bootstrap auto-restart hangs on missing sensitive key file

[Andy LoPresto (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy LoPresto updated NIFI-4187:

Assignee: Andy LoPresto
  Status: Patch Available  (was: Open)

> If NiFi process is killed, bootstrap auto-restart hangs on missing sensitive 
> key file
> -
>
> Key: NIFI-4187
> URL: https://issues.apache.org/jira/browse/NIFI-4187
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Critical
>  Labels: bootstrap, encryption
>
> If the NiFi process is killed and {{autoRestart}} is enabled in the bootstrap 
> process ({{RunNiFi.java}}) (enabled by default except when in {{run}} mode) 
> and the instance is using encrypted configuration files, the 
> {{sensitive.key}} file which contains the master encryption key is no longer 
> available when restart occurs. This results in the application exiting. 
> To resolve this, the logic to prepare and generate the sensitive key file 
> should be extracted to a method and invoked during initial start *and* during 
> the auto-restart loop at 
> https://github.com/apache/nifi/blob/master/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java#L1159
>  . 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4187) If NiFi process is killed, bootstrap auto-restart hangs on missing sensitive key file

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122529#comment-16122529
 ] 

ASF GitHub Bot commented on NIFI-4187:
--

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2073
  
To verify this fix: 

1. modify the `conf/bootstrap.conf` file to provide a *master encryption 
key* (it doesn't matter if any `conf/nifi.properties` values are encrypted with 
it, the mere presence will trigger the relevant code)
1. Start NiFi as normal (i.e. `$ ./bin/nifi.sh start`) 
1. Verify that NiFi starts (UI, `nifi-bootstrap.log`, `nifi-app.log`, etc.)
1. Identify the NiFi process (*not* the bootstrap process) and kill it 
(imitating an `OOME` or external process killing NiFi)
  * `ps -ef | grep nifi` to get the PID
  * `kill -9 `
1. Verify that NiFi successfully restarts

The `logs/nifi-bootstrap.log` will look like this:

```
2017-08-10 16:08:49,675 INFO [main] o.a.n.b.NotificationServiceManager 
Successfully loaded the following 0 services: []
2017-08-10 16:08:49,679 INFO [main] org.apache.nifi.bootstrap.RunNiFi 
Registered no Notification Services for Notification Type NIFI_STARTED
2017-08-10 16:08:49,679 INFO [main] org.apache.nifi.bootstrap.RunNiFi 
Registered no Notification Services for Notification Type NIFI_STOPPED
2017-08-10 16:08:49,679 INFO [main] org.apache.nifi.bootstrap.RunNiFi 
Registered no Notification Services for Notification Type NIFI_DIED
2017-08-10 16:08:49,736 INFO [main] org.apache.nifi.bootstrap.Command 
Starting Apache NiFi...
2017-08-10 16:08:49,736 INFO [main] org.apache.nifi.bootstrap.Command 
Working Directory: 
/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT
2017-08-10 16:08:49,736 INFO [main] org.apache.nifi.bootstrap.Command 
Command: /Users/alopresto/.jenv/versions/1.8/bin/java -classpath 
/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./conf:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/javax.servlet-api-3.1.0.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/jcl-over-slf4j-1.7.25.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/jetty-schemas-3.1.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/jul-to-slf4j-1.7.25.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/log4j-over-slf4j-1.7.25.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/logback-classic-1.2.3.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/logback-core-1.2.3.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-api-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-framework-api-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-nar-utils-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-properties-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-runtime-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/slf4j-api-1.7.25.jar
 -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m 
-Djava.security.egd=file:/dev/urandom 
-Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true 
-Djava.awt.headless=true -XX:+UseG1GC 
-Djava.protocol.handler.pkgs=sun.net.www.protocol 
-Dnifi.properties.file.path=/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./conf/nifi.properties
 -Dnifi.bootstrap.listen.port=52192 -Dapp=NiFi 
-Dorg.apache.nifi.bootstrap.config.log.dir=/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/logs
 org.apache.nifi.NiFi -K 
/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./conf/sensitive.key
2017-08-10 16:08:49,749 INFO [main] org.apache.nifi.bootstrap.Command 
Launched Apache NiFi with Process ID 90937
2017-08-10 16:08:50,294 INFO [NiFi Bootstrap Command Listener] 
org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for 
Bootstrap requests on port 52193
-External command kills NiFi process-
2017-08-10 16:11:13,185 WARN [main] 

[GitHub] nifi issue #2073: NIFI-4187 Resolved issue where restart failed due to missi...

[alopresto]

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2073
  
To verify this fix: 

1. modify the `conf/bootstrap.conf` file to provide a *master encryption 
key* (it doesn't matter if any `conf/nifi.properties` values are encrypted with 
it, the mere presence will trigger the relevant code)
1. Start NiFi as normal (i.e. `$ ./bin/nifi.sh start`) 
1. Verify that NiFi starts (UI, `nifi-bootstrap.log`, `nifi-app.log`, etc.)
1. Identify the NiFi process (*not* the bootstrap process) and kill it 
(imitating an `OOME` or external process killing NiFi)
  * `ps -ef | grep nifi` to get the PID
  * `kill -9 `
1. Verify that NiFi successfully restarts

The `logs/nifi-bootstrap.log` will look like this:

```
2017-08-10 16:08:49,675 INFO [main] o.a.n.b.NotificationServiceManager 
Successfully loaded the following 0 services: []
2017-08-10 16:08:49,679 INFO [main] org.apache.nifi.bootstrap.RunNiFi 
Registered no Notification Services for Notification Type NIFI_STARTED
2017-08-10 16:08:49,679 INFO [main] org.apache.nifi.bootstrap.RunNiFi 
Registered no Notification Services for Notification Type NIFI_STOPPED
2017-08-10 16:08:49,679 INFO [main] org.apache.nifi.bootstrap.RunNiFi 
Registered no Notification Services for Notification Type NIFI_DIED
2017-08-10 16:08:49,736 INFO [main] org.apache.nifi.bootstrap.Command 
Starting Apache NiFi...
2017-08-10 16:08:49,736 INFO [main] org.apache.nifi.bootstrap.Command 
Working Directory: 
/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT
2017-08-10 16:08:49,736 INFO [main] org.apache.nifi.bootstrap.Command 
Command: /Users/alopresto/.jenv/versions/1.8/bin/java -classpath 
/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./conf:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/javax.servlet-api-3.1.0.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/jcl-over-slf4j-1.7.25.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/jetty-schemas-3.1.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/jul-to-slf4j-1.7.25.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/log4j-over-slf4j-1.7.25.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/logback-c
 
lassic-1.2.3.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/logback-core-1.2.3.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-api-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-framework-api-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-nar-utils-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-properties-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-runtime-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/slf4j-api-1.7.25.jar
 -Dorg.apache.jasper.compiler.disablejsr199=true 
 -Xmx512m -Xms512m -Djava.security.egd=file:/dev/urandom 
-Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true 
-Djava.awt.headless=true -XX:+UseG1GC 
-Djava.protocol.handler.pkgs=sun.net.www.protocol 
-Dnifi.properties.file.path=/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./conf/nifi.properties
 -Dnifi.bootstrap.listen.port=52192 -Dapp=NiFi 
-Dorg.apache.nifi.bootstrap.config.log.dir=/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/logs
 org.apache.nifi.NiFi -K 
/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./conf/sensitive.key
2017-08-10 16:08:49,749 INFO [main] org.apache.nifi.bootstrap.Command 
Launched Apache NiFi with Process ID 90937
2017-08-10 16:08:50,294 INFO [NiFi Bootstrap Command Listener] 
org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for 
Bootstrap requests on port 52193
-External command kills NiFi process-
2017-08-10 16:11:13,185 WARN [main] org.apache.nifi.bootstrap.RunNiFi 
Apache NiFi appears to have died. Restarting...
2017-08-10 16:11:13,195 INFO [main] org.apache.nifi.bootstrap.Command 
Launched Apache NiFi with Process ID 91211
2017-08-10 16:11:13,196 INFO [main] 

[jira] [Commented] (NIFI-4187) If NiFi process is killed, bootstrap auto-restart hangs on missing sensitive key file

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122517#comment-16122517
 ] 

ASF GitHub Bot commented on NIFI-4187:
--

GitHub user alopresto opened a pull request:

https://github.com/apache/nifi/pull/2073

NIFI-4187 Resolved issue where restart failed due to missing sensitiv…

…e key file.

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/alopresto/nifi NIFI-4187

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2073.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2073


commit 7dc0bd2084820a9f924a0593f2e938efb1a0ceed
Author: Andy LoPresto 
Date:   2017-08-10T23:16:29Z

NIFI-4187 Resolved issue where restart failed due to missing sensitive key 
file.




> If NiFi process is killed, bootstrap auto-restart hangs on missing sensitive 
> key file
> -
>
> Key: NIFI-4187
> URL: https://issues.apache.org/jira/browse/NIFI-4187
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Priority: Critical
>  Labels: bootstrap, encryption
>
> If the NiFi process is killed and {{autoRestart}} is enabled in the bootstrap 
> process ({{RunNiFi.java}}) (enabled by default except when in {{run}} mode) 
> and the instance is using encrypted configuration files, the 
> {{sensitive.key}} file which contains the master encryption key is no longer 
> available when restart occurs. This results in the application exiting. 
> To resolve this, the logic to prepare and generate the sensitive key file 
> should be extracted to a method and invoked during initial start *and* during 
> the auto-restart loop at 
> https://github.com/apache/nifi/blob/master/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java#L1159
>  . 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2073: NIFI-4187 Resolved issue where restart failed due t...

[alopresto]

GitHub user alopresto opened a pull request:

https://github.com/apache/nifi/pull/2073

NIFI-4187 Resolved issue where restart failed due to missing sensitiv…

…e key file.

Thank you for submitting a contribution to Apache NiFi.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/alopresto/nifi NIFI-4187

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2073.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2073


commit 7dc0bd2084820a9f924a0593f2e938efb1a0ceed
Author: Andy LoPresto 
Date:   2017-08-10T23:16:29Z

NIFI-4187 Resolved issue where restart failed due to missing sensitive key 
file.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (NIFI-4281) ExecuteScript with ruby throws IndexOutOfBoundsException on all scripts

[Jeremy Lautman (JIRA)]

Jeremy Lautman created NIFI-4281:


 Summary: ExecuteScript with ruby throws IndexOutOfBoundsException 
on all scripts
 Key: NIFI-4281
 URL: https://issues.apache.org/jira/browse/NIFI-4281
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Affects Versions: 1.3.0
Reporter: Jeremy Lautman


In running ExecuteScript with the following ruby script, ExecuteScript throws 
IndexOutOfBoundsExceptions.

{quote}session.transfer(session.get(), REL_SUCCESS){quote}

The same script with python works appropriately.

It also does the same when using the session::get(int) variant, and when 
running larger scripts.

NiFi version 1.3.0
Java openjdk version 1.8.0_121



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2020: [NiFi-3973] Add PutKudu Processor for ingesting data to Ku...

[rickysaltzer]

Github user rickysaltzer commented on the issue:

https://github.com/apache/nifi/pull/2020
  
@cammachusa seems I can't update this PR with my own code. My changes are 
below so you can merge them in and then push them again.


https://github.com/rickysaltzer/nifi/commit/c17bbaff08685f74eef037f5e34dd6339b3aac68

I was able to get a sample pipeline working locally which is great. I want 
to do a little bit more testing before we merge it in.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4276) Add Provenance Migration section to User Guide

[Andrew Lim (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122329#comment-16122329
 ] 

Andrew Lim commented on NIFI-4276:
--

Adding to the Data Provenance section of the User Guide makes more sense than 
the Admin Guide.  There is also a section in the User Guide regarding the 
Encrypted Provenance Repository which relates to this migration.

> Add Provenance Migration section to User Guide
> --
>
> Key: NIFI-4276
> URL: https://issues.apache.org/jira/browse/NIFI-4276
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Minor
>
> WriteAheadProvenanceRepository configuration was introduced in version 1.2.0 
> (NIFI-3356).  Should add a section to the documentation that covers how to 
> migrate from default configuration of PersistentProvenantRepository and 
> recommend any system property value changes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4276) Add Provenance Migration section to User Guide

[Andrew Lim (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Lim updated NIFI-4276:
-
Summary: Add Provenance Migration section to User Guide  (was: Add 
Provenance Migration section to Admin Guide)

> Add Provenance Migration section to User Guide
> --
>
> Key: NIFI-4276
> URL: https://issues.apache.org/jira/browse/NIFI-4276
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Minor
>
> WriteAheadProvenanceRepository configuration was introduced in version 1.2.0 
> (NIFI-3356).  Should add a section to the documentation that covers how to 
> migrate from default configuration of PersistentProvenantRepository and 
> recommend any system property value changes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-3232) Extend UI menus to allow cascading (menu item > subitem)

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan updated NIFI-3232:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Extend UI menus to allow cascading (menu item > subitem)
> 
>
> Key: NIFI-3232
> URL: https://issues.apache.org/jira/browse/NIFI-3232
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Reporter: Rob Moran
>Assignee: Matt Gilman
>
> With expanding functionality UI menu options are growing, making the size of 
> some quite large. Cascading menus will allow better information hierarchy to 
> improve the presentation of available user actions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3232) Extend UI menus to allow cascading (menu item > subitem)

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122327#comment-16122327
 ] 

ASF GitHub Bot commented on NIFI-3232:
--

Github user scottyaslan commented on the issue:

https://github.com/apache/nifi/pull/2072
  
Thanks @mcgilman this has been merged to master!


> Extend UI menus to allow cascading (menu item > subitem)
> 
>
> Key: NIFI-3232
> URL: https://issues.apache.org/jira/browse/NIFI-3232
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Reporter: Rob Moran
>Assignee: Matt Gilman
>
> With expanding functionality UI menu options are growing, making the size of 
> some quite large. Cascading menus will allow better information hierarchy to 
> improve the presentation of available user actions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2072: NIFI-3232: Adding support for cascading sub context menus

[scottyaslan]

Github user scottyaslan commented on the issue:

https://github.com/apache/nifi/pull/2072
  
Thanks @mcgilman this has been merged to master!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3232) Extend UI menus to allow cascading (menu item > subitem)

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122323#comment-16122323
 ] 

ASF GitHub Bot commented on NIFI-3232:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2072


> Extend UI menus to allow cascading (menu item > subitem)
> 
>
> Key: NIFI-3232
> URL: https://issues.apache.org/jira/browse/NIFI-3232
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Reporter: Rob Moran
>Assignee: Matt Gilman
>
> With expanding functionality UI menu options are growing, making the size of 
> some quite large. Cascading menus will allow better information hierarchy to 
> improve the presentation of available user actions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3232) Extend UI menus to allow cascading (menu item > subitem)

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122321#comment-16122321
 ] 

ASF subversion and git services commented on NIFI-3232:
---

Commit 20d6596df097e4fd3716b25ff9927e93f7bff3a9 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=20d6596 ]

NIFI-3232:
- Adding support for cascading sub context menus.

Signed-off-by: Scott Aslan 

This closes #2072


> Extend UI menus to allow cascading (menu item > subitem)
> 
>
> Key: NIFI-3232
> URL: https://issues.apache.org/jira/browse/NIFI-3232
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Reporter: Rob Moran
>Assignee: Matt Gilman
>
> With expanding functionality UI menu options are growing, making the size of 
> some quite large. Cascading menus will allow better information hierarchy to 
> improve the presentation of available user actions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2020: [NiFi-3973] Add PutKudu Processor for ingesting data to Ku...

[rickysaltzer]

Github user rickysaltzer commented on the issue:

https://github.com/apache/nifi/pull/2020
  
I've made some changes that allows the tests to run successfully. I'm going 
to test out this in an actual NiFi installation and then provide you my 
changes. Unfortunately this may not happen today due to my schedule. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2072: NIFI-3232: Adding support for cascading sub context...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2072


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122297#comment-16122297
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user markap14 commented on the issue:

https://github.com/apache/nifi/pull/2051
  
@mcgilman thanks for the thorough review! I have pushed a new commit, which 
I believe addresses all of the feedback above.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2051: NIFI-4224: Initial implementation of Process Group level V...

[markap14]

Github user markap14 commented on the issue:

https://github.com/apache/nifi/pull/2051
  
@mcgilman thanks for the thorough review! I have pushed a new commit, which 
I believe addresses all of the feedback above.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (NIFI-4280) UI - Support configuring variables at a Process Group level

[Matt Gilman (JIRA)]

Matt Gilman created NIFI-4280:
-

 Summary: UI - Support configuring variables at a Process Group 
level
 Key: NIFI-4280
 URL: https://issues.apache.org/jira/browse/NIFI-4280
 Project: Apache NiFi
  Issue Type: New Feature
  Components: Core UI
Reporter: Matt Gilman
Assignee: Matt Gilman


Add support for configuring variables at a Process Group level. Variables, like 
Controller Services, are scoped by the group they are defined in and are 
available to any Processor defined there and below (ie and descendant 
Processors). Variables in a descendant group can override the value in a parent 
group. Variable permissions will be based solely on the corresponding Process 
Group.

Because variables are specified in EL, there is no controlling access to them 
like Controller Services which can be restricted using the combo selector. 
Because of this, variables should not be used for sensitive values.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-4032) Create Managed Ranger Authorizer

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-4032:
--
Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Create Managed Ranger Authorizer
> 
>
> Key: NIFI-4032
> URL: https://issues.apache.org/jira/browse/NIFI-4032
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Extensions
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.4.0
>
>
> Update the RangerAuthorizer to implement the ManagedAuthorizer interface. 
> This will allow the Ranger policies to be visualized in NiFi UI. May even be 
> able to extend the RangerAuthorizer to maintain compatibility with existing 
> configurations.
> Additionally, update the RangerAuthorizer's authorize(...) method to consider 
> the user's groups.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (NIFI-968) Add Primary Node Scheduling to User and Dev Guide

[Andy LoPresto (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy LoPresto updated NIFI-968:
---
   Resolution: Fixed
Fix Version/s: 1.4.0
   Status: Resolved  (was: Patch Available)

> Add Primary Node Scheduling to User and Dev Guide
> -
>
> Key: NIFI-968
> URL: https://issues.apache.org/jira/browse/NIFI-968
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Reporter: Joseph Percivall
>Assignee: Pierre Villard
>Priority: Minor
> Fix For: 1.4.0
>
>
> The User Guide talks about every other scheduling type except Primary Node 
> Only. This should be noted as an option in cluster environments.
> In the Dev Guide, every other annotation is talked about except for 
> OnPrimaryNodeStateChange. It should be added and potentially some information 
> about cluster environment considerations when coding could be added too 
> (currently nothing regarding cluster environments in dev guide).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-968) Add Primary Node Scheduling to User and Dev Guide

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122160#comment-16122160
 ] 

ASF GitHub Bot commented on NIFI-968:
-

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2027


> Add Primary Node Scheduling to User and Dev Guide
> -
>
> Key: NIFI-968
> URL: https://issues.apache.org/jira/browse/NIFI-968
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Reporter: Joseph Percivall
>Assignee: Pierre Villard
>Priority: Minor
>
> The User Guide talks about every other scheduling type except Primary Node 
> Only. This should be noted as an option in cluster environments.
> In the Dev Guide, every other annotation is talked about except for 
> OnPrimaryNodeStateChange. It should be added and potentially some information 
> about cluster environment considerations when coding could be added too 
> (currently nothing regarding cluster environments in dev guide).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-968) Add Primary Node Scheduling to User and Dev Guide

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122158#comment-16122158
 ] 

ASF subversion and git services commented on NIFI-968:
--

Commit c03f4e731bbc9ac3a74d1e1ca800942b5b73dc23 in nifi's branch 
refs/heads/master from [~pvillard]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=c03f4e7 ]

NIFI-968 - add @OnPrimaryNodeStateChange to dev guide

This closes #2027.

Signed-off-by: Andy LoPresto 


> Add Primary Node Scheduling to User and Dev Guide
> -
>
> Key: NIFI-968
> URL: https://issues.apache.org/jira/browse/NIFI-968
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Reporter: Joseph Percivall
>Assignee: Pierre Villard
>Priority: Minor
>
> The User Guide talks about every other scheduling type except Primary Node 
> Only. This should be noted as an option in cluster environments.
> In the Dev Guide, every other annotation is talked about except for 
> OnPrimaryNodeStateChange. It should be added and potentially some information 
> about cluster environment considerations when coding could be added too 
> (currently nothing regarding cluster environments in dev guide).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-968) Add Primary Node Scheduling to User and Dev Guide

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122157#comment-16122157
 ] 

ASF GitHub Bot commented on NIFI-968:
-

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2027
  
Fixed a couple typos. Ran `contrib-check` and all tests pass. +1, merging. 


> Add Primary Node Scheduling to User and Dev Guide
> -
>
> Key: NIFI-968
> URL: https://issues.apache.org/jira/browse/NIFI-968
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Reporter: Joseph Percivall
>Assignee: Pierre Villard
>Priority: Minor
>
> The User Guide talks about every other scheduling type except Primary Node 
> Only. This should be noted as an option in cluster environments.
> In the Dev Guide, every other annotation is talked about except for 
> OnPrimaryNodeStateChange. It should be added and potentially some information 
> about cluster environment considerations when coding could be added too 
> (currently nothing regarding cluster environments in dev guide).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2020: [NiFi-3973] Add PutKudu Processor for ingesting data to Ku...

[rickysaltzer]

Github user rickysaltzer commented on the issue:

https://github.com/apache/nifi/pull/2020
  
@cammachusa thanks for the reminder :) - I've been a bit busy due to 
business travel. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #2027: NIFI-968 - add @OnPrimaryNodeStateChange to dev gui...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2027


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #2027: NIFI-968 - add @OnPrimaryNodeStateChange to dev guide

[alopresto]

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2027
  
Fixed a couple typos. Ran `contrib-check` and all tests pass. +1, merging. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3232) Extend UI menus to allow cascading (menu item > subitem)

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122148#comment-16122148
 ] 

ASF GitHub Bot commented on NIFI-3232:
--

Github user scottyaslan commented on the issue:

https://github.com/apache/nifi/pull/2072
  
Reviewing...


> Extend UI menus to allow cascading (menu item > subitem)
> 
>
> Key: NIFI-3232
> URL: https://issues.apache.org/jira/browse/NIFI-3232
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Reporter: Rob Moran
>Assignee: Matt Gilman
>
> With expanding functionality UI menu options are growing, making the size of 
> some quite large. Cascading menus will allow better information hierarchy to 
> improve the presentation of available user actions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2072: NIFI-3232: Adding support for cascading sub context menus

[scottyaslan]

Github user scottyaslan commented on the issue:

https://github.com/apache/nifi/pull/2072
  
Reviewing...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-3232) Extend UI menus to allow cascading (menu item > subitem)

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-3232:
--
Status: Patch Available  (was: In Progress)

> Extend UI menus to allow cascading (menu item > subitem)
> 
>
> Key: NIFI-3232
> URL: https://issues.apache.org/jira/browse/NIFI-3232
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Reporter: Rob Moran
>Assignee: Matt Gilman
>
> With expanding functionality UI menu options are growing, making the size of 
> some quite large. Cascading menus will allow better information hierarchy to 
> improve the presentation of available user actions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-3232) Extend UI menus to allow cascading (menu item > subitem)

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122142#comment-16122142
 ] 

ASF GitHub Bot commented on NIFI-3232:
--

GitHub user mcgilman opened a pull request:

https://github.com/apache/nifi/pull/2072

NIFI-3232: Adding support for cascading sub context menus

NIFI-3232:
- Adding support for cascading sub context menus.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mcgilman/nifi NIFI-3232

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2072.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2072


commit bfb3f33e547e60e8d1a8100cf1dfd5727c3f3d37
Author: Matt Gilman 
Date:   2017-08-10T19:09:58Z

NIFI-3232:
- Adding support for cascading sub context menus.




> Extend UI menus to allow cascading (menu item > subitem)
> 
>
> Key: NIFI-3232
> URL: https://issues.apache.org/jira/browse/NIFI-3232
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Reporter: Rob Moran
>Assignee: Matt Gilman
>
> With expanding functionality UI menu options are growing, making the size of 
> some quite large. Cascading menus will allow better information hierarchy to 
> improve the presentation of available user actions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2072: NIFI-3232: Adding support for cascading sub context...

[mcgilman]

GitHub user mcgilman opened a pull request:

https://github.com/apache/nifi/pull/2072

NIFI-3232: Adding support for cascading sub context menus

NIFI-3232:
- Adding support for cascading sub context menus.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mcgilman/nifi NIFI-3232

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/2072.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2072


commit bfb3f33e547e60e8d1a8100cf1dfd5727c3f3d37
Author: Matt Gilman 
Date:   2017-08-10T19:09:58Z

NIFI-3232:
- Adding support for cascading sub context menus.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi issue #2027: NIFI-968 - add @OnPrimaryNodeStateChange to dev guide

[alopresto]

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2027
  
Reviewing...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4277) StandardLogRepository does not log exceptions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122104#comment-16122104
 ] 

ASF GitHub Bot commented on NIFI-4277:
--

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2068
  
Ran `contrib-check` and all tests pass. +1, merging. 


> StandardLogRepository does not log exceptions
> -
>
> Key: NIFI-4277
> URL: https://issues.apache.org/jira/browse/NIFI-4277
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.3.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
> Attachments: Screen Shot 2017-08-08 at 2.48.33 PM.png
>
>
> When logging a message, it is logged with the SLF4J logger and also stored in 
> the standard log repository (for the bulletins). However if the array of 
> objects contains the exception (and not the message of the exception), this 
> exception won't be displayed in the bulletin message.
> That's because of:
> {code:title=StandardLogRepository.java|borderStyle=solid}
> @Override
> public void addLogMessage(final LogLevel level, final String format, 
> final Object[] params) {
> final String formattedMessage = MessageFormatter.arrayFormat(format, 
> params).getMessage();
> addLogMessage(level, formattedMessage);
> }
> {code}
> If the params object contains a Throwable object, it'll be removed from the 
> array in the {{MessageFormatter}}:
> {code:title=MessageFormatter.java|borderStyle=solid}
> final public static FormattingTuple arrayFormat(final String 
> messagePattern, final Object[] argArray) {
> Throwable throwableCandidate = getThrowableCandidate(argArray);
> Object[] args = argArray;
> if (throwableCandidate != null) {
> args = trimmedCopy(argArray);
> }
> return arrayFormat(messagePattern, args, throwableCandidate);
> }
> {code}
> Easy solution would be to change:
> {noformat}
> logger.debug("Failed to validate {} against schema due to {}", new 
> Object[]{flowFile, e});
> {noformat}
> into:
> {noformat}
> logger.debug("Failed to validate {} against schema due to {}", new 
> Object[]{flowFile, e.getLocalizedMessage()});
> {noformat}
> However this pattern can be found in quite a large number of places... And 
> it'd be certainly better to provide a permanent solution supporting the 
> existing pattern. Suggestion is to modify the method in 
> {{StandardLogRepository}} to go through all the items of the array and for 
> each Throwable object, replace it by the localized message.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4277) StandardLogRepository does not log exceptions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122097#comment-16122097
 ] 

ASF GitHub Bot commented on NIFI-4277:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2068


> StandardLogRepository does not log exceptions
> -
>
> Key: NIFI-4277
> URL: https://issues.apache.org/jira/browse/NIFI-4277
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.3.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
> Attachments: Screen Shot 2017-08-08 at 2.48.33 PM.png
>
>
> When logging a message, it is logged with the SLF4J logger and also stored in 
> the standard log repository (for the bulletins). However if the array of 
> objects contains the exception (and not the message of the exception), this 
> exception won't be displayed in the bulletin message.
> That's because of:
> {code:title=StandardLogRepository.java|borderStyle=solid}
> @Override
> public void addLogMessage(final LogLevel level, final String format, 
> final Object[] params) {
> final String formattedMessage = MessageFormatter.arrayFormat(format, 
> params).getMessage();
> addLogMessage(level, formattedMessage);
> }
> {code}
> If the params object contains a Throwable object, it'll be removed from the 
> array in the {{MessageFormatter}}:
> {code:title=MessageFormatter.java|borderStyle=solid}
> final public static FormattingTuple arrayFormat(final String 
> messagePattern, final Object[] argArray) {
> Throwable throwableCandidate = getThrowableCandidate(argArray);
> Object[] args = argArray;
> if (throwableCandidate != null) {
> args = trimmedCopy(argArray);
> }
> return arrayFormat(messagePattern, args, throwableCandidate);
> }
> {code}
> Easy solution would be to change:
> {noformat}
> logger.debug("Failed to validate {} against schema due to {}", new 
> Object[]{flowFile, e});
> {noformat}
> into:
> {noformat}
> logger.debug("Failed to validate {} against schema due to {}", new 
> Object[]{flowFile, e.getLocalizedMessage()});
> {noformat}
> However this pattern can be found in quite a large number of places... And 
> it'd be certainly better to provide a permanent solution supporting the 
> existing pattern. Suggestion is to modify the method in 
> {{StandardLogRepository}} to go through all the items of the array and for 
> each Throwable object, replace it by the localized message.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4277) StandardLogRepository does not log exceptions

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122095#comment-16122095
 ] 

ASF subversion and git services commented on NIFI-4277:
---

Commit 18c82eb6af6648460365f8db286729629258e95c in nifi's branch 
refs/heads/master from [~pvillard]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=18c82eb ]

NIFI-4277 Fixed exception logging in StandardLogRepository

This closes #2068.

Signed-off-by: Andy LoPresto 


> StandardLogRepository does not log exceptions
> -
>
> Key: NIFI-4277
> URL: https://issues.apache.org/jira/browse/NIFI-4277
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.3.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
> Attachments: Screen Shot 2017-08-08 at 2.48.33 PM.png
>
>
> When logging a message, it is logged with the SLF4J logger and also stored in 
> the standard log repository (for the bulletins). However if the array of 
> objects contains the exception (and not the message of the exception), this 
> exception won't be displayed in the bulletin message.
> That's because of:
> {code:title=StandardLogRepository.java|borderStyle=solid}
> @Override
> public void addLogMessage(final LogLevel level, final String format, 
> final Object[] params) {
> final String formattedMessage = MessageFormatter.arrayFormat(format, 
> params).getMessage();
> addLogMessage(level, formattedMessage);
> }
> {code}
> If the params object contains a Throwable object, it'll be removed from the 
> array in the {{MessageFormatter}}:
> {code:title=MessageFormatter.java|borderStyle=solid}
> final public static FormattingTuple arrayFormat(final String 
> messagePattern, final Object[] argArray) {
> Throwable throwableCandidate = getThrowableCandidate(argArray);
> Object[] args = argArray;
> if (throwableCandidate != null) {
> args = trimmedCopy(argArray);
> }
> return arrayFormat(messagePattern, args, throwableCandidate);
> }
> {code}
> Easy solution would be to change:
> {noformat}
> logger.debug("Failed to validate {} against schema due to {}", new 
> Object[]{flowFile, e});
> {noformat}
> into:
> {noformat}
> logger.debug("Failed to validate {} against schema due to {}", new 
> Object[]{flowFile, e.getLocalizedMessage()});
> {noformat}
> However this pattern can be found in quite a large number of places... And 
> it'd be certainly better to provide a permanent solution supporting the 
> existing pattern. Suggestion is to modify the method in 
> {{StandardLogRepository}} to go through all the items of the array and for 
> each Throwable object, replace it by the localized message.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2068: NIFI-4277 Fixed exception logging in StandardLogRep...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2068


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4277) StandardLogRepository does not log exceptions

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122037#comment-16122037
 ] 

ASF GitHub Bot commented on NIFI-4277:
--

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2068
  
Reviewing...


> StandardLogRepository does not log exceptions
> -
>
> Key: NIFI-4277
> URL: https://issues.apache.org/jira/browse/NIFI-4277
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.3.0
>Reporter: Pierre Villard
>Assignee: Pierre Villard
> Attachments: Screen Shot 2017-08-08 at 2.48.33 PM.png
>
>
> When logging a message, it is logged with the SLF4J logger and also stored in 
> the standard log repository (for the bulletins). However if the array of 
> objects contains the exception (and not the message of the exception), this 
> exception won't be displayed in the bulletin message.
> That's because of:
> {code:title=StandardLogRepository.java|borderStyle=solid}
> @Override
> public void addLogMessage(final LogLevel level, final String format, 
> final Object[] params) {
> final String formattedMessage = MessageFormatter.arrayFormat(format, 
> params).getMessage();
> addLogMessage(level, formattedMessage);
> }
> {code}
> If the params object contains a Throwable object, it'll be removed from the 
> array in the {{MessageFormatter}}:
> {code:title=MessageFormatter.java|borderStyle=solid}
> final public static FormattingTuple arrayFormat(final String 
> messagePattern, final Object[] argArray) {
> Throwable throwableCandidate = getThrowableCandidate(argArray);
> Object[] args = argArray;
> if (throwableCandidate != null) {
> args = trimmedCopy(argArray);
> }
> return arrayFormat(messagePattern, args, throwableCandidate);
> }
> {code}
> Easy solution would be to change:
> {noformat}
> logger.debug("Failed to validate {} against schema due to {}", new 
> Object[]{flowFile, e});
> {noformat}
> into:
> {noformat}
> logger.debug("Failed to validate {} against schema due to {}", new 
> Object[]{flowFile, e.getLocalizedMessage()});
> {noformat}
> However this pattern can be found in quite a large number of places... And 
> it'd be certainly better to provide a permanent solution supporting the 
> existing pattern. Suggestion is to modify the method in 
> {{StandardLogRepository}} to go through all the items of the array and for 
> each Throwable object, replace it by the localized message.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2068: NIFI-4277 Fixed exception logging in StandardLogRepository

[alopresto]

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2068
  
Reviewing...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-4210) Add OpenId Connect support for authenticating users

[Andy LoPresto (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy LoPresto updated NIFI-4210:

   Resolution: Fixed
Fix Version/s: 1.4.0
   Status: Resolved  (was: Patch Available)

> Add OpenId Connect support for authenticating users
> ---
>
> Key: NIFI-4210
> URL: https://issues.apache.org/jira/browse/NIFI-4210
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework, Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.4.0
>
>
> Add support for authenticating users with the OpenId Connection 
> specification. Evaluate whether a new extension point is necessary to allow 
> for a given provider to supply custom code for instance to implement custom 
> token validation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4210) Add OpenId Connect support for authenticating users

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122022#comment-16122022
 ] 

ASF GitHub Bot commented on NIFI-4210:
--

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2047
  
Ran `contrib-check` and all tests pass. +1, merging. 


> Add OpenId Connect support for authenticating users
> ---
>
> Key: NIFI-4210
> URL: https://issues.apache.org/jira/browse/NIFI-4210
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework, Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>
> Add support for authenticating users with the OpenId Connection 
> specification. Evaluate whether a new extension point is necessary to allow 
> for a given provider to supply custom code for instance to implement custom 
> token validation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (NIFI-4210) Add OpenId Connect support for authenticating users

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122024#comment-16122024
 ] 

ASF GitHub Bot commented on NIFI-4210:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2047


> Add OpenId Connect support for authenticating users
> ---
>
> Key: NIFI-4210
> URL: https://issues.apache.org/jira/browse/NIFI-4210
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework, Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>
> Add support for authenticating users with the OpenId Connection 
> specification. Evaluate whether a new extension point is necessary to allow 
> for a given provider to supply custom code for instance to implement custom 
> token validation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2047: NIFI-4210: Add support for OpenId Connect

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2047


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4210) Add OpenId Connect support for authenticating users

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122021#comment-16122021
 ] 

ASF subversion and git services commented on NIFI-4210:
---

Commit 528b82634f528468970eaa655c0085b1c9592b71 in nifi's branch 
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=528b826 ]

NIFI-4210:
- Introducing support for OpenId Connect.
- Updating REST API and UI to support the authorization code flow.
- Adding/fixing documentation.
- Implementing time constant equality checks where appropriate.
- Corrected error handling during startup and throughout the OIDC login 
sequence.
- Redacting the token values from the user log.
- Defaulting to RS256 when not preferred algorithm is specified.
- Marking the OIDC endpoints as non-guaranteed in to allow for minor 
adjustments if/when additional SSO techniques are introduced.

This closes #2047.

Signed-off-by: Andy LoPresto 


> Add OpenId Connect support for authenticating users
> ---
>
> Key: NIFI-4210
> URL: https://issues.apache.org/jira/browse/NIFI-4210
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework, Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>
> Add support for authenticating users with the OpenId Connection 
> specification. Evaluate whether a new extension point is necessary to allow 
> for a given provider to supply custom code for instance to implement custom 
> token validation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2047: NIFI-4210: Add support for OpenId Connect

[alopresto]

Github user alopresto commented on the issue:

https://github.com/apache/nifi/pull/2047
  
Ran `contrib-check` and all tests pass. +1, merging. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3927) Extract HL7 Attributes throwing NULLpointerException

[Raj karan (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121958#comment-16121958
 ] 

Raj karan commented on NIFI-3927:
-

[~dmoore247] Have you tried the message containing Z segments; as Z segments 
were causing the problem.

> Extract HL7 Attributes throwing NULLpointerException
> 
>
> Key: NIFI-3927
> URL: https://issues.apache.org/jira/browse/NIFI-3927
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Extensions
>Affects Versions: 1.0.0
> Environment: NiFi with HortonWorks
>Reporter: Raj karan
>Assignee: Joey Frazee
> Attachments: null pointer.png, pipe_ended_charcter_encoded_ascii.png, 
> resultWithDefault.txt, source.txt, when not ended with pipe.png
>
>
> I have an HL7 file which I want to put in HBase, So I am parsing this file 
> through ExtractHL7Attributes processor. With the default value for every 
> property processor works with no error but resultant attributes file only 
> have one segment. When I sets `Use Segment Names` property true it throws 
> NULLPointerException.
> Stack trace:
> 2017-05-17 11:11:58,390 INFO [Heartbeat Monitor Thread-1] 
> o.a.n.c.c.h.AbstractHeartbeatMonitor Finished processing 1 heartbeats in 4756 
> nanos
> 2017-05-17 11:11:58,847 ERROR [Timer-Driven Process Thread-2] 
> o.a.n.p.hl7.ExtractHL7Attributes 
> ExtractHL7Attributes[id=bea89fef-86db-1094--81c2e524] Failed to 
> extract attributes from 
> StandardFlowFileRecord[uuid=73a649fe-261c-40d2-bad7-b0bc595c0158,claim=StandardContentClaim
>  [resourceClaim=StandardResourceClaim[id=1495030753601-25550, 
> container=default, section=974], offset=912561, 
> length=288],offset=0,name=source.txt,size=288] due to 
> ca.uhn.hl7v2.HL7Exception: The HL7 version 2.3
> EVN is not recognized: ca.uhn.hl7v2.HL7Exception: The HL7 version 2.3
> EVN is not recognized
> 2017-05-17 11:11:58,848 ERROR [Timer-Driven Process Thread-2] 
> o.a.n.p.hl7.ExtractHL7Attributes 
> ca.uhn.hl7v2.HL7Exception: The HL7 version 2.3
> EVN is not recognized
>   at ca.uhn.hl7v2.parser.Parser.assertVersionExists(Parser.java:527) 
> ~[hapi-base-2.2.jar:na]
>   at ca.uhn.hl7v2.parser.Parser.parse(Parser.java:208) 
> ~[hapi-base-2.2.jar:na]
>   at ca.uhn.hl7v2.parser.PipeParser.parse(PipeParser.java:1018) 
> ~[hapi-base-2.2.jar:na]
>   at 
> org.apache.nifi.processors.hl7.ExtractHL7Attributes.onTrigger(ExtractHL7Attributes.java:195)
>  ~[nifi-hl7-processors-1.0.0.2.0.2.0-17.jar:1.0.0.2.0.2.0-17]
>   at 
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>  [nifi-api-1.0.0.2.0.2.0-17.jar:1.0.0.2.0.2.0-17]
>   at 
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1064)
>  [nifi-framework-core-1.0.0.2.0.2.0-17.jar:1.0.0.2.0.2.0-17]
>   at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
>  [nifi-framework-core-1.0.0.2.0.2.0-17.jar:1.0.0.2.0.2.0-17]
>   at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
>  [nifi-framework-core-1.0.0.2.0.2.0-17.jar:1.0.0.2.0.2.0-17]
>   at 
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
>  [nifi-framework-core-1.0.0.2.0.2.0-17.jar:1.0.0.2.0.2.0-17]
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_77]
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_77]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_77]
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [na:1.8.0_77]
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_77]
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_77]
>   at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
> 2017-05-17 11:11:58,852 ERROR [Timer-Driven Process Thread-1] 
> o.a.n.p.hl7.ExtractHL7Attributes 
> ExtractHL7Attributes[id=bea89fef-86db-1094--81c2e524] 
> ExtractHL7Attributes[id=bea89fef-86db-1094--81c2e524] failed to 
> process due to java.lang.NullPointerException; rolling back session: 
> java.lang.NullPointerException
> 2017-05-17 11:11:58,852 ERROR [Timer-Driven Process Thread-1] 
> o.a.n.p.hl7.ExtractHL7Attributes 
> java.lang.NullPointerException: null
> 2017-05-17 11:11:58,852 ERROR [Timer-Driven Process Thread-1] 
> o.a.n.p.hl7.ExtractHL7Attributes 
> ExtractHL7Attributes[id=bea89fef-86db-1094--81c2e524] 
> 

[jira] [Commented] (NIFI-3973) Create a new Kudu Processor to ingest data

[Cam Mach (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121918#comment-16121918
 ] 

Cam Mach commented on NIFI-3973:


[~sanysand...@gmail.com] You're welcome to review the PR. It has been out there 
for months, but it's active. Thanks

> Create a new Kudu Processor to ingest data
> --
>
> Key: NIFI-3973
> URL: https://issues.apache.org/jira/browse/NIFI-3973
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Cam Mach
>Assignee: Cam Quoc Mach
>   Original Estimate: 336h
>  Remaining Estimate: 336h
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi issue #2020: [NiFi-3973] Add PutKudu Processor for ingesting data to Ku...

[cammachusa]

Github user cammachusa commented on the issue:

https://github.com/apache/nifi/pull/2020
  
@rickysaltzer , remind :-)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121861#comment-16121861
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132502384
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The 

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132502384
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
 

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121762#comment-16121762
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132477480
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The 

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132477480
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
 

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132476153
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -325,6 +441,859 @@ public Response updateProcessGroup(
 );
 }
 
+
+@GET
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Gets a process group's variable registry", 
response = VariableRegistryUpdateRequestEntity.class, authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = "")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response getVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
+if (groupId == null || updateId == null) {
+throw new IllegalArgumentException("Group ID and Update ID 
must both be specified.");
+}
+
+if (isReplicateRequest()) {
+return replicate(HttpMethod.GET);
+}
+
+// authorize access
+serviceFacade.authorizeAccess(lookup -> {
+final Authorizable processGroup = 
lookup.getProcessGroup(groupId).getAuthorizable();
+processGroup.authorize(authorizer, RequestAction.READ, 
NiFiUserUtils.getNiFiUser());
+});
+
+final VariableRegistryUpdateRequest request = 
varRegistryUpdateRequests.get(updateId);
+if (request == null) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId);
+}
+
+if (!groupId.equals(request.getProcessGroupId())) {
+throw new ResourceNotFoundException("Could not find a Variable 
Registry Update Request with identifier " + updateId + " for Process Group with 
identifier " + groupId);
+}
+
+final VariableRegistryUpdateRequestEntity entity = new 
VariableRegistryUpdateRequestEntity();
+entity.setId(request.getRequestId());
+
entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request));
+entity.setUri(generateResourceUri("process-groups", groupId, 
"variable-registry", updateId));
+return generateOkResponse(entity).build();
+}
+
+
+@DELETE
+@Consumes(MediaType.WILDCARD)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("{groupId}/variable-registry/update-requests/{updateId}")
+@ApiOperation(value = "Deletes an update request for a process group's 
variable registry. If the request is not yet complete, it will automatically be 
cancelled.",
+response = VariableRegistryUpdateRequestEntity.class, 
authorizations = {
+@Authorization(value = "Read - /process-groups/{uuid}", type = 
"")
+})
+@ApiResponses(value = {
+@ApiResponse(code = 400, message = "NiFi was unable to complete 
the request because it was invalid. The request should not be retried without 
modification."),
+@ApiResponse(code = 401, message = "Client could not be 
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized to 
make this request."),
+@ApiResponse(code = 404, message = "The specified resource could 
not be found."),
+@ApiResponse(code = 409, message = "The request was valid but NiFi 
was not in the appropriate state to process it. Retrying the same request later 
may be successful.")
+})
+public Response deleteVariableRegistryUpdateRequest(
+@ApiParam(value = "The process group id.", required = true) 
@PathParam("groupId") final String groupId,
+@ApiParam(value = "The ID of the Variable Registry Update 
Request", required = true) @PathParam("updateId") final String updateId) {
+
 

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121741#comment-16121741
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132472005
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -314,7 +430,7 @@ public Response updateProcessGroup(
 Authorizable authorizable = 
lookup.getProcessGroup(id).getAuthorizable();
 authorizable.authorize(authorizer, 
RequestAction.WRITE, NiFiUserUtils.getNiFiUser());
 },
-null,
+() -> 
serviceFacade.verifyUpdateProcessGroup(requestProcessGroupDTO),
--- End diff --

Good call. Originally, I had it updating the variables here but then 
refactored quite a bit. I think it's best to go ahead and leave in the addition 
of the verifyUpdateProcessGroup method, but make the verification a NOP. We do 
this in a few other places, as well, so I feel it's best to leave the 
'plumbing' there, since it's already been built. It will make it easier to 
update later.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...

[markap14]

Github user markap14 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2051#discussion_r132472005
  
--- Diff: 
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
 ---
@@ -314,7 +430,7 @@ public Response updateProcessGroup(
 Authorizable authorizable = 
lookup.getProcessGroup(id).getAuthorizable();
 authorizable.authorize(authorizer, 
RequestAction.WRITE, NiFiUserUtils.getNiFiUser());
 },
-null,
+() -> 
serviceFacade.verifyUpdateProcessGroup(requestProcessGroupDTO),
--- End diff --

Good call. Originally, I had it updating the variables here but then 
refactored quite a bit. I think it's best to go ahead and leave in the addition 
of the verifyUpdateProcessGroup method, but make the verification a NOP. We do 
this in a few other places, as well, so I feel it's best to leave the 
'plumbing' there, since it's already been built. It will make it easier to 
update later.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121651#comment-16121651
 ] 

ASF GitHub Bot commented on NIFI-4224:
--

Github user markap14 commented on the issue:

https://github.com/apache/nifi/pull/2051
  
@mcgilman thanks for the review. Will update and push a new commit.


> Add Variable Registry at Process Group level
> 
>
> Key: NIFI-4224
> URL: https://issues.apache.org/jira/browse/NIFI-4224
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>
> Currently, NiFi exposes a variable registry that is configurable by adding 
> the name of a properties file to nifi.properties and then treating the 
> referenced properties file as key/value pairs for the variable registry. 
> This, however, is very limiting, as it provides a global scope for all 
> variables, and it requires a restart of NiFi in order to pick up any updates 
> to the file. We should expose a Process Group-level Variable Registry.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)