[jira] [Commented] (NIFI-4023) WriteAheadProvenanceRepository indexing and query failure under high rate stress testing
[ https://issues.apache.org/jira/browse/NIFI-4023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122855#comment-16122855 ] Randy Bovay commented on NIFI-4023: --- We are experiencing this as well. Good to see there is already an issue tracking this. Happy to test with you more to help find a solution to this. > WriteAheadProvenanceRepository indexing and query failure under high rate > stress testing > > > Key: NIFI-4023 > URL: https://issues.apache.org/jira/browse/NIFI-4023 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.3.0 >Reporter: Joseph Witt > > 2017-06-06 00:32:35,995 INFO [pool-10-thread-1] > org.wali.MinimalLockingWriteAheadLog > org.wali.MinimalLockingWriteAheadLog@5ce7ab6f checkpointed with 5737 Records > and 0 Swap Files in 467 milliseconds (Stop-the-world time = 172 milliseconds, > Clear Edit Logs time = 137 millis), max Transaction ID 5739 > 2017-06-06 00:32:35,996 INFO [pool-10-thread-1] > o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile > Repository with 5737 records in 467 milliseconds > 2017-06-06 00:33:35,418 ERROR [Index Provenance Events-2] > o.a.n.p.index.lucene.EventIndexTask Failed to index Provenance Events > org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: > NativeFSLock@/Users/jwitt/build-verify/nifi-1.3.0/nifi-assembly/target/nifi-1.3.0-bin/nifi-1.3.0/provenance_repository/index-1496723454612/write.lock > at org.apache.lucene.store.Lock.obtain(Lock.java:89) > at org.apache.lucene.index.IndexWriter.(IndexWriter.java:755) > at > org.apache.nifi.provenance.lucene.SimpleIndexManager.createWriter(SimpleIndexManager.java:198) > at > org.apache.nifi.provenance.lucene.SimpleIndexManager.borrowIndexWriter(SimpleIndexManager.java:227) > at > org.apache.nifi.provenance.index.lucene.EventIndexTask.index(EventIndexTask.java:184) > at > org.apache.nifi.provenance.index.lucene.EventIndexTask.run(EventIndexTask.java:104) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:748) > 2017-06-06 00:33:36,420 ERROR [Index Provenance Events-1] > o.a.n.p.index.lucene.EventIndexTask Failed to index Provenance Events > org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: > NativeFSLock@/Users/jwitt/build-verify/nifi-1.3.0/nifi-assembly/target/nifi-1.3.0-bin/nifi-1.3.0/provenance_repository/index-1496723454612/write.lock > at org.apache.lucene.store.Lock.obtain(Lock.java:89) > at org.apache.lucene.index.IndexWriter.(IndexWriter.java:755) > at > org.apache.nifi.provenance.lucene.SimpleIndexManager.createWriter(SimpleIndexManager.java:198) > at > org.apache.nifi.provenance.lucene.SimpleIndexManager.borrowIndexWriter(SimpleIndexManager.java:227) > at > org.apache.nifi.provenance.index.lucene.EventIndexTask.index(EventIndexTask.java:184) > at > org.apache.nifi.provenance.index.lucene.EventIndexTask.run(EventIndexTask.java:104) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:748) > 2017-06-06 00:33:37,425 ERROR [Index Provenance Events-2] > o.a.n.p.index.lucene.EventIndexTask Failed to index Provenance Events > org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: > NativeFSLock@/Users/jwitt/build-verify/nifi-1.3.0/nifi-assembly/target/nifi-1.3.0-bin/nifi-1.3.0/provenance_repository/index-1496723454612/write.lock > at org.apache.lucene.store.Lock.obtain(Lock.java:89) > at org.apache.lucene.index.IndexWriter.(IndexWriter.java:755) > at > org.apache.nifi.provenance.lucene.SimpleIndexManager.createWriter(SimpleIndexManager.java:198) > at > org.apache.nifi.provenance.lucene.SimpleIndexManager.borrowIndexWriter(SimpleIndexManager.java:227) > at > org.apache.nifi.provenance.index.lucene.EventIndexTask.index(EventIndexTask.java:184) > at > org.apache.nifi.provenance.index.lucene.EventIndexTask.run(EventIndexTask.java:104) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at
[GitHub] nifi issue #2020: [NiFi-3973] Add PutKudu Processor for ingesting data to Ku...
Github user cammachusa commented on the issue: https://github.com/apache/nifi/pull/2020 The last commit, I switched to onScheduled already. Thanks for your review and code @rickysaltzer There are some code style check errors. I'm on it now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4257) Allow a custom WHERE clause in AbstractDatabaseFetchProcessor
[ https://issues.apache.org/jira/browse/NIFI-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122783#comment-16122783 ] ASF GitHub Bot commented on NIFI-4257: -- Github user patricker commented on a diff in the pull request: https://github.com/apache/nifi/pull/2050#discussion_r132612494 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java --- @@ -259,6 +261,11 @@ public void onTrigger(final ProcessContext context, final ProcessSessionFactory } }); +if(customWhereClause != null) { +// adding the custom WHERE clause (if defined) to the list of existing clauses. +maxValueClauses.add(customWhereClause); --- End diff -- What do you think about forcing custom where clauses to be surrounded with parenthesis? I worry that just putting it in like this may cause unexpected consequences if the where clauses includes OR's. > Allow a custom WHERE clause in AbstractDatabaseFetchProcessor > - > > Key: NIFI-4257 > URL: https://issues.apache.org/jira/browse/NIFI-4257 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard > > It could be useful allowing a user to set a custom WHERE clause in > AbstractDatabaseFetchProcessor in case not all of the data in the table is > required. > In case the WHERE clause is changed after the processor has already been > running, the user will probably have to set the initial maximum values to > ensure the expected behaviour. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2050: NIFI-4257 - add custom WHERE clause in database fet...
Github user patricker commented on a diff in the pull request: https://github.com/apache/nifi/pull/2050#discussion_r132612494 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java --- @@ -259,6 +261,11 @@ public void onTrigger(final ProcessContext context, final ProcessSessionFactory } }); +if(customWhereClause != null) { +// adding the custom WHERE clause (if defined) to the list of existing clauses. +maxValueClauses.add(customWhereClause); --- End diff -- What do you think about forcing custom where clauses to be surrounded with parenthesis? I worry that just putting it in like this may cause unexpected consequences if the where clauses includes OR's. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4279) PutDataBaseRecord steam has already been closed
[ https://issues.apache.org/jira/browse/NIFI-4279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122736#comment-16122736 ] Peter Wicks commented on NIFI-4279: --- Usually when I see this error it's not because of the processor but because of the DBCP Connection Pool losing a connection but not recognizing it. It then hands a closed connection to the processor and an error like this sometimes gets closed (also I've seen 'Closed Pipe'). This may not be your issue, but to check you could try adding a Validation Query to your Oracle connection if you don't already have one. > PutDataBaseRecord steam has already been closed > --- > > Key: NIFI-4279 > URL: https://issues.apache.org/jira/browse/NIFI-4279 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.3.0 > Environment: we have used nifi on openshift cluster and standalone on > a linux machine. > Oracle database with version 11.2.0.2 > linux with rhel 7.0 >Reporter: simon water >Priority: Minor > Labels: database, nifi, patch, processor > Original Estimate: 24h > Remaining Estimate: 24h > > we have created a table in an Oracle Database with a default sysdate column. > when i restarted the processor it started throwing the stream has already > been closed exception > i have used a csv file with the headers inside of it > csv Reader that reads the column names from the csv > the data base version is 11.2.0 Oracle database -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4279) PutDataBaseRecord steam has already been closed
[ https://issues.apache.org/jira/browse/NIFI-4279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-4279: -- Flags: (was: Patch) > PutDataBaseRecord steam has already been closed > --- > > Key: NIFI-4279 > URL: https://issues.apache.org/jira/browse/NIFI-4279 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.3.0 > Environment: we have used nifi on openshift cluster and standalone on > a linux machine. > Oracle database with version 11.2.0.2 > linux with rhel 7.0 >Reporter: simon water >Priority: Minor > Labels: database, nifi, patch, processor > Original Estimate: 24h > Remaining Estimate: 24h > > we have created a table in an Oracle Database with a default sysdate column. > when i restarted the processor it started throwing the stream has already > been closed exception > i have used a csv file with the headers inside of it > csv Reader that reads the column names from the csv > the data base version is 11.2.0 Oracle database -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4250) Create support for deleting document by id from elasticsearch 5
[ https://issues.apache.org/jira/browse/NIFI-4250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122731#comment-16122731 ] Joseph Witt commented on NIFI-4250: --- Fix version can be set once contrib/review cycle is being merged or very near to merge. Thanks [~mans2singh] just a matter of finding a reviewer that can help. > Create support for deleting document by id from elasticsearch 5 > --- > > Key: NIFI-4250 > URL: https://issues.apache.org/jira/browse/NIFI-4250 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Mans Singh >Assignee: Mans Singh >Priority: Minor > Labels: delete, elasticsearch > > Create a processor to delete documents from elasticsearch 5 based on document > id. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4250) Create support for deleting document by id from elasticsearch 5
[ https://issues.apache.org/jira/browse/NIFI-4250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-4250: -- Fix Version/s: (was: 1.4.0) > Create support for deleting document by id from elasticsearch 5 > --- > > Key: NIFI-4250 > URL: https://issues.apache.org/jira/browse/NIFI-4250 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Mans Singh >Assignee: Mans Singh >Priority: Minor > Labels: delete, elasticsearch > > Create a processor to delete documents from elasticsearch 5 based on document > id. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4279) PutDataBaseRecord steam has already been closed
[ https://issues.apache.org/jira/browse/NIFI-4279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122732#comment-16122732 ] Joseph Witt commented on NIFI-4279: --- Fix version can be set once contrib/review cycle is being merged or very near to merge. > PutDataBaseRecord steam has already been closed > --- > > Key: NIFI-4279 > URL: https://issues.apache.org/jira/browse/NIFI-4279 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.3.0 > Environment: we have used nifi on openshift cluster and standalone on > a linux machine. > Oracle database with version 11.2.0.2 > linux with rhel 7.0 >Reporter: simon water >Priority: Minor > Labels: database, nifi, patch, processor > Original Estimate: 24h > Remaining Estimate: 24h > > we have created a table in an Oracle Database with a default sysdate column. > when i restarted the processor it started throwing the stream has already > been closed exception > i have used a csv file with the headers inside of it > csv Reader that reads the column names from the csv > the data base version is 11.2.0 Oracle database -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4279) PutDataBaseRecord steam has already been closed
[ https://issues.apache.org/jira/browse/NIFI-4279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-4279: -- Fix Version/s: (was: 1.4.0) > PutDataBaseRecord steam has already been closed > --- > > Key: NIFI-4279 > URL: https://issues.apache.org/jira/browse/NIFI-4279 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.3.0 > Environment: we have used nifi on openshift cluster and standalone on > a linux machine. > Oracle database with version 11.2.0.2 > linux with rhel 7.0 >Reporter: simon water >Priority: Minor > Labels: database, nifi, patch, processor > Original Estimate: 24h > Remaining Estimate: 24h > > we have created a table in an Oracle Database with a default sysdate column. > when i restarted the processor it started throwing the stream has already > been closed exception > i have used a csv file with the headers inside of it > csv Reader that reads the column names from the csv > the data base version is 11.2.0 Oracle database -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4237) EncryptionOperationNotPossibleException in nifi-bootstrap.log might suggest underlying cause
[ https://issues.apache.org/jira/browse/NIFI-4237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-4237: -- Fix Version/s: (was: 1.4.0) > EncryptionOperationNotPossibleException in nifi-bootstrap.log might suggest > underlying cause > > > Key: NIFI-4237 > URL: https://issues.apache.org/jira/browse/NIFI-4237 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Russell Bateman >Priority: Minor > > Our Ansible instructions upgraded NiFi and created a new > {{nifi.sensitive.props.key}}. In _nifi.properties_ this property, if extant, > is used to encrypt sensitive properties in _flow.xml.gz_. Thus, upon > relaunching NiFi, the wrong key was used to decrypt resulting in the reported > failure to start, _flow.xml.gz_ is no longer useful. > We found the problem and fixed it after Mark Payne suggested a possible > cause, but if this state of things can be determined, it might save on > community support for this situation if the logged message were to suggest > what's at the bottom of this problem. The top of the stack trace appears in > _logs/nifi-bootstrap.log_ as below: > 2017-07-25 23:23:31,148 WARN [main] org.apache.nifi.web.server.JettyServer > Failed to start web server... shutting down. > org.apache.nifi.encrypt.EncryptionException: > org.jasypt.exceptions.EncryptionOperationNotPossibleException > at > org.apache.nifi.encrypt.StringEncryptor.decrypt(StringEncryptor.java:149) > ~[nifi-framework-core-1.1.2.jar:1.1.2] > at > org.apache.nifi.controller.serialization.FlowFromDOMFactory.decrypt(FlowFromDOMFactory.java:474) > ~[nifi-framework-core-1.1.2.jar:1.1.2] > at... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4256) Add support for all AWS S3 Encryption Options
[ https://issues.apache.org/jira/browse/NIFI-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122727#comment-16122727 ] Joseph Witt commented on NIFI-4256: --- Fix version can be set once contrib/review cycle is being merged or very near to merge. > Add support for all AWS S3 Encryption Options > - > > Key: NIFI-4256 > URL: https://issues.apache.org/jira/browse/NIFI-4256 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Franco > Labels: aws, aws-s3, security > > NiFi currently only supportsĀ SSE-S3 encryption (AES256). > Support needs to be added for: > * SSE-S3 > * SSE-KMS > * SSE-C > * CSE-KMS CMK > * CSE-Master Key > With all of the appropriate configuration options and such that SSE is > available only for PutS3Object whilst CSE is available also for FetchS3Object. > Given that this will add another 20 or so UI properties the intention is to > split it into a Client Side Encryption Service and Server Side Encryption > Service. This will allow users to reuse "encryption" across different > workflows. > Existing flows using the Server Side Encryption option will still work as is > but will be overridden if a service is added. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4237) EncryptionOperationNotPossibleException in nifi-bootstrap.log might suggest underlying cause
[ https://issues.apache.org/jira/browse/NIFI-4237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122728#comment-16122728 ] Joseph Witt commented on NIFI-4237: --- Fix version can be set once contrib/review cycle is being merged or very near to merge. > EncryptionOperationNotPossibleException in nifi-bootstrap.log might suggest > underlying cause > > > Key: NIFI-4237 > URL: https://issues.apache.org/jira/browse/NIFI-4237 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Russell Bateman >Priority: Minor > > Our Ansible instructions upgraded NiFi and created a new > {{nifi.sensitive.props.key}}. In _nifi.properties_ this property, if extant, > is used to encrypt sensitive properties in _flow.xml.gz_. Thus, upon > relaunching NiFi, the wrong key was used to decrypt resulting in the reported > failure to start, _flow.xml.gz_ is no longer useful. > We found the problem and fixed it after Mark Payne suggested a possible > cause, but if this state of things can be determined, it might save on > community support for this situation if the logged message were to suggest > what's at the bottom of this problem. The top of the stack trace appears in > _logs/nifi-bootstrap.log_ as below: > 2017-07-25 23:23:31,148 WARN [main] org.apache.nifi.web.server.JettyServer > Failed to start web server... shutting down. > org.apache.nifi.encrypt.EncryptionException: > org.jasypt.exceptions.EncryptionOperationNotPossibleException > at > org.apache.nifi.encrypt.StringEncryptor.decrypt(StringEncryptor.java:149) > ~[nifi-framework-core-1.1.2.jar:1.1.2] > at > org.apache.nifi.controller.serialization.FlowFromDOMFactory.decrypt(FlowFromDOMFactory.java:474) > ~[nifi-framework-core-1.1.2.jar:1.1.2] > at... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4256) Add support for all AWS S3 Encryption Options
[ https://issues.apache.org/jira/browse/NIFI-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-4256: -- Fix Version/s: (was: 1.4.0) > Add support for all AWS S3 Encryption Options > - > > Key: NIFI-4256 > URL: https://issues.apache.org/jira/browse/NIFI-4256 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Franco > Labels: aws, aws-s3, security > > NiFi currently only supportsĀ SSE-S3 encryption (AES256). > Support needs to be added for: > * SSE-S3 > * SSE-KMS > * SSE-C > * CSE-KMS CMK > * CSE-Master Key > With all of the appropriate configuration options and such that SSE is > available only for PutS3Object whilst CSE is available also for FetchS3Object. > Given that this will add another 20 or so UI properties the intention is to > split it into a Client Side Encryption Service and Server Side Encryption > Service. This will allow users to reuse "encryption" across different > workflows. > Existing flows using the Server Side Encryption option will still work as is > but will be overridden if a service is added. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4177) MergeContent - Tar - Save modification timestamp like Tar does
[ https://issues.apache.org/jira/browse/NIFI-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-4177: -- Resolution: Fixed Status: Resolved (was: Patch Available) > MergeContent - Tar - Save modification timestamp like Tar does > -- > > Key: NIFI-4177 > URL: https://issues.apache.org/jira/browse/NIFI-4177 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Wayne Steel >Assignee: Joseph Witt >Priority: Trivial > Fix For: 1.4.0 > > > Tar by default saves the modification timestamp of entries. > This mainly affects file based entries so could be done on reading the > attribute file.lastModifiedTime, if it exists, which is written to the > flowfile by GetFile or ListFile processors. > Otherwise just leave it out as it does now. > I propose a property with the default expression ${file.lastModifiedTime} but > the value must resolve to a date format of "-MM-dd'T'HH:mm:ssZ". It > should only be enabled when MERGE_FORMAT is set to MERGE_FORMAT_TAR -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4177) MergeContent - Tar - Save modification timestamp like Tar does
[ https://issues.apache.org/jira/browse/NIFI-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122721#comment-16122721 ] ASF subversion and git services commented on NIFI-4177: --- Commit a6e8f0afe3407bf668e234dcce3c474f1ceb86c8 in nifi's branch refs/heads/master from [~makosteel] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=a6e8f0a ] NIFI-4177 This closes #2002. MergeContent - Tar - Save modification timestamp like Tar does Signed-off-by: joewitt> MergeContent - Tar - Save modification timestamp like Tar does > -- > > Key: NIFI-4177 > URL: https://issues.apache.org/jira/browse/NIFI-4177 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Wayne Steel >Assignee: Joseph Witt >Priority: Trivial > Fix For: 1.4.0 > > > Tar by default saves the modification timestamp of entries. > This mainly affects file based entries so could be done on reading the > attribute file.lastModifiedTime, if it exists, which is written to the > flowfile by GetFile or ListFile processors. > Otherwise just leave it out as it does now. > I propose a property with the default expression ${file.lastModifiedTime} but > the value must resolve to a date format of "-MM-dd'T'HH:mm:ssZ". It > should only be enabled when MERGE_FORMAT is set to MERGE_FORMAT_TAR -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4177) MergeContent - Tar - Save modification timestamp like Tar does
[ https://issues.apache.org/jira/browse/NIFI-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122722#comment-16122722 ] ASF GitHub Bot commented on NIFI-4177: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2002 > MergeContent - Tar - Save modification timestamp like Tar does > -- > > Key: NIFI-4177 > URL: https://issues.apache.org/jira/browse/NIFI-4177 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Wayne Steel >Assignee: Joseph Witt >Priority: Trivial > Fix For: 1.4.0 > > > Tar by default saves the modification timestamp of entries. > This mainly affects file based entries so could be done on reading the > attribute file.lastModifiedTime, if it exists, which is written to the > flowfile by GetFile or ListFile processors. > Otherwise just leave it out as it does now. > I propose a property with the default expression ${file.lastModifiedTime} but > the value must resolve to a date format of "-MM-dd'T'HH:mm:ssZ". It > should only be enabled when MERGE_FORMAT is set to MERGE_FORMAT_TAR -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2002: NIFI-4177 MergeContent - Tar - Save modification ti...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2002 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4250) Create support for deleting document by id from elasticsearch 5
[ https://issues.apache.org/jira/browse/NIFI-4250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122716#comment-16122716 ] ASF GitHub Bot commented on NIFI-4250: -- Github user mans2singh commented on the issue: https://github.com/apache/nifi/pull/2045 Hey Folks: Let me know if you have any feedback/recommends for this processor. Thanks Mans > Create support for deleting document by id from elasticsearch 5 > --- > > Key: NIFI-4250 > URL: https://issues.apache.org/jira/browse/NIFI-4250 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Mans Singh >Assignee: Mans Singh >Priority: Minor > Labels: delete, elasticsearch > Fix For: 1.4.0 > > > Create a processor to delete documents from elasticsearch 5 based on document > id. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2045: NIFI-4250 - Elasticsearch 5 delete processor
Github user mans2singh commented on the issue: https://github.com/apache/nifi/pull/2045 Hey Folks: Let me know if you have any feedback/recommends for this processor. Thanks Mans --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Assigned] (NIFI-4177) MergeContent - Tar - Save modification timestamp like Tar does
[ https://issues.apache.org/jira/browse/NIFI-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt reassigned NIFI-4177: - Assignee: Joseph Witt > MergeContent - Tar - Save modification timestamp like Tar does > -- > > Key: NIFI-4177 > URL: https://issues.apache.org/jira/browse/NIFI-4177 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Wayne Steel >Assignee: Joseph Witt >Priority: Trivial > Fix For: 1.4.0 > > > Tar by default saves the modification timestamp of entries. > This mainly affects file based entries so could be done on reading the > attribute file.lastModifiedTime, if it exists, which is written to the > flowfile by GetFile or ListFile processors. > Otherwise just leave it out as it does now. > I propose a property with the default expression ${file.lastModifiedTime} but > the value must resolve to a date format of "-MM-dd'T'HH:mm:ssZ". It > should only be enabled when MERGE_FORMAT is set to MERGE_FORMAT_TAR -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2057: NIFI-2167 - include disabled state for processors when cop...
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2057 Created two processors and disabled them. Verified that copy/paste and creating a template & instantiating it maintain the `disabled` state. Ran `contrib-check` and all tests pass. +1, merging. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2167) Disabled state not including in a Processor that was copy/paste
[ https://issues.apache.org/jira/browse/NIFI-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122667#comment-16122667 ] ASF GitHub Bot commented on NIFI-2167: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2057 > Disabled state not including in a Processor that was copy/paste > --- > > Key: NIFI-2167 > URL: https://issues.apache.org/jira/browse/NIFI-2167 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Matt Gilman >Assignee: Pierre Villard > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2057: NIFI-2167 - include disabled state for processors w...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2057 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2167) Disabled state not including in a Processor that was copy/paste
[ https://issues.apache.org/jira/browse/NIFI-2167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122663#comment-16122663 ] ASF GitHub Bot commented on NIFI-2167: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2057 Reviewing... > Disabled state not including in a Processor that was copy/paste > --- > > Key: NIFI-2167 > URL: https://issues.apache.org/jira/browse/NIFI-2167 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Matt Gilman >Assignee: Pierre Villard > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2015: NIFI-4142: Refactored Record Reader/Writer to allow for re...
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/2015 @markap14 can you please rebase and resolve conflict and please go ahead and squash. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4142) Implement a ValidateRecord Processor
[ https://issues.apache.org/jira/browse/NIFI-4142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122660#comment-16122660 ] ASF GitHub Bot commented on NIFI-4142: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/2015 @markap14 can you please rebase and resolve conflict and please go ahead and squash. > Implement a ValidateRecord Processor > > > Key: NIFI-4142 > URL: https://issues.apache.org/jira/browse/NIFI-4142 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.4.0 > > > We need a processor that is capable of validating that all Records in a > FlowFile adhere to the proper schema. > The Processor should be configured with a Record Reader and should route each > record to either 'valid' or 'invalid' based on whether or not the record > adheres to the reader's schema. A record would be invalid in any of the > following cases: > - Missing field that is required according to the schema > - Extra field that is not present in schema (it should be configurable > whether or not this is a failure) > - Field requires coercion and strict type checking enabled (this should also > be configurable) > - Field is invalid, such as the value "hello" when it should be an integer -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-3335) GenerateTableFetch should allow you to specify an initial Max Value
[ https://issues.apache.org/jira/browse/NIFI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122657#comment-16122657 ] Joseph Witt commented on NIFI-3335: --- Ok it appears the db for JIRA for this ticket is wacky. Can be fixed if done by an admin. Rather than bothering ASF admins just cloned it and ported the history/log there. Will still be searchable by those comments. Deleting this JIRA. > GenerateTableFetch should allow you to specify an initial Max Value > --- > > Key: NIFI-3335 > URL: https://issues.apache.org/jira/browse/NIFI-3335 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Matt Burgess >Assignee: Joseph Witt > > NIFI-2583 added the ability (via dynamic properties) to specify initial Max > Values for columns, to enable the user to "pick up where they left off" if > something happened with a flow, a NiFi instance, etc. where the state was > stored but the processing did not complete successfully. > This feature would also be helpful in GenerateTableFetch, which also supports > max-value columns. > Since NIFI-2881 adds incoming flow file support, it's more useful if Initial > max values can be specified via flow file attributes. Because if a table name > is dynamically passed via flow file attribute and Expression Language, user > won't be able to configure dynamic processor attribute in advance for each > possible table. > Add dynamic properties ('initial.maxvalue.' same as > QueryDatabaseTable) to specify initial max values statically, and also use > incoming flow file attributes named 'initial.maxvalue.' if > any. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Deleted] (NIFI-3335) GenerateTableFetch should allow you to specify an initial Max Value
[ https://issues.apache.org/jira/browse/NIFI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt deleted NIFI-3335: -- > GenerateTableFetch should allow you to specify an initial Max Value > --- > > Key: NIFI-3335 > URL: https://issues.apache.org/jira/browse/NIFI-3335 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Matt Burgess >Assignee: Joseph Witt > > NIFI-2583 added the ability (via dynamic properties) to specify initial Max > Values for columns, to enable the user to "pick up where they left off" if > something happened with a flow, a NiFi instance, etc. where the state was > stored but the processing did not complete successfully. > This feature would also be helpful in GenerateTableFetch, which also supports > max-value columns. > Since NIFI-2881 adds incoming flow file support, it's more useful if Initial > max values can be specified via flow file attributes. Because if a table name > is dynamically passed via flow file attribute and Expression Language, user > won't be able to configure dynamic processor attribute in advance for each > possible table. > Add dynamic properties ('initial.maxvalue.' same as > QueryDatabaseTable) to specify initial max values statically, and also use > incoming flow file attributes named 'initial.maxvalue.' if > any. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4283) GenerateTableFetch should allow you to specify an initial Max Value (clone of defunct NIFI-3335)
[ https://issues.apache.org/jira/browse/NIFI-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122656#comment-16122656 ] Joseph Witt commented on NIFI-4283: --- NIFI-3335 got into a funky state that probably would have been solved by asking for integrity checker run against NIFI. Instead just cloning and deleting old ticket. History from NIFI-3335 is here Back to previous view [NIFI-3335] GenerateTableFetch should allow you to specify an initial Max Value Created: 12/Jan/17 Updated: 11/Aug/17 Status: Open Project:Apache NiFi Component/s:Extensions Affects Version/s: None Fix Version/s: 1.4.0 Type: Improvement Priority: Major Reporter: Matt BurgessAssignee: Joseph Witt Resolution: Unresolved Votes: 0 Labels: None Remaining Estimate: Not Specified Time Spent: Not Specified Original Estimate: Not Specified Description NIFI-2583 added the ability (via dynamic properties) to specify initial Max Values for columns, to enable the user to "pick up where they left off" if something happened with a flow, a NiFi instance, etc. where the state was stored but the processing did not complete successfully. This feature would also be helpful in GenerateTableFetch, which also supports max-value columns. Since NIFI-2881 adds incoming flow file support, it's more useful if Initial max values can be specified via flow file attributes. Because if a table name is dynamically passed via flow file attribute and Expression Language, user won't be able to configure dynamic processor attribute in advance for each possible table. Add dynamic properties ('initial.maxvalue.' same as QueryDatabaseTable) to specify initial max values statically, and also use incoming flow file attributes named 'initial.maxvalue.' if any. Comments Comment by Matt Burgess [ 17/Jul/17 ] Peter Wicks Are you still working on this? If not, do you mind if I unassign it? Thanks! Comment by Peter Wicks [ 19/Jul/17 ] Matt Burgess I'm not working on this. I've unassigned myself. Comment by ASF GitHub Bot [ 25/Jul/17 ] GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/2039 NIFI-3335: Add initial.maxvalue support to GenerateTableFetch Moved the common code back to the abstract base class For all changes: [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. [x] Has your PR been rebased against the latest commit within the target branch (typically master)? [x] Is your initial contribution a single, squashed commit? For code changes: [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? [x] Have you written or updated unit tests to verify your changes? [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? For documentation related changes: [x] Have you ensured that format looks appropriate for the output in which it is rendered? Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-3335 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2039.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2039 commit 8eea477c02fdb2ca19ec77da9168ceec5e8eae7c Author: Matt BurgessDate: 2017-07-25T20:07:52Z NIFI-3335: Add initial.maxvalue support to GenerateTableFetch Comment by ASF GitHub Bot [ 26/Jul/17 ] Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2039#discussion_r129494131 ā Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java ā @@ -198,6 +201,25 @@ public void onTrigger(final ProcessContext context, final ProcessSessionFactory // set as the current state map (after the session has been committed) final Map statePropertyMap = new HashMap<>(stateMap.toMap()); + // If an initial max value for column(s) has
[jira] [Created] (NIFI-4283) GenerateTableFetch should allow you to specify an initial Max Value (clone of defunct NIFI-3335)
Joseph Witt created NIFI-4283: - Summary: GenerateTableFetch should allow you to specify an initial Max Value (clone of defunct NIFI-3335) Key: NIFI-4283 URL: https://issues.apache.org/jira/browse/NIFI-4283 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Matt Burgess Assignee: Joseph Witt Fix For: 1.4.0 NIFI-2583 added the ability (via dynamic properties) to specify initial Max Values for columns, to enable the user to "pick up where they left off" if something happened with a flow, a NiFi instance, etc. where the state was stored but the processing did not complete successfully. This feature would also be helpful in GenerateTableFetch, which also supports max-value columns. Since NIFI-2881 adds incoming flow file support, it's more useful if Initial max values can be specified via flow file attributes. Because if a table name is dynamically passed via flow file attribute and Expression Language, user won't be able to configure dynamic processor attribute in advance for each possible table. Add dynamic properties ('initial.maxvalue.' same as QueryDatabaseTable) to specify initial max values statically, and also use incoming flow file attributes named 'initial.maxvalue.' if any. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-3335) GenerateTableFetch should allow you to specify an initial Max Value
[ https://issues.apache.org/jira/browse/NIFI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-3335: -- Component/s: (was: Extensions) > GenerateTableFetch should allow you to specify an initial Max Value > --- > > Key: NIFI-3335 > URL: https://issues.apache.org/jira/browse/NIFI-3335 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Matt Burgess >Assignee: Joseph Witt > Fix For: 1.4.0 > > > NIFI-2583 added the ability (via dynamic properties) to specify initial Max > Values for columns, to enable the user to "pick up where they left off" if > something happened with a flow, a NiFi instance, etc. where the state was > stored but the processing did not complete successfully. > This feature would also be helpful in GenerateTableFetch, which also supports > max-value columns. > Since NIFI-2881 adds incoming flow file support, it's more useful if Initial > max values can be specified via flow file attributes. Because if a table name > is dynamically passed via flow file attribute and Expression Language, user > won't be able to configure dynamic processor attribute in advance for each > possible table. > Add dynamic properties ('initial.maxvalue.' same as > QueryDatabaseTable) to specify initial max values statically, and also use > incoming flow file attributes named 'initial.maxvalue.' if > any. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (NIFI-4282) GenerateTableFetch should allow you to specify an initial Max Value (cloned)
Joseph Witt created NIFI-4282: - Summary: GenerateTableFetch should allow you to specify an initial Max Value (cloned) Key: NIFI-4282 URL: https://issues.apache.org/jira/browse/NIFI-4282 Project: Apache NiFi Issue Type: Task Components: Extensions Reporter: Matt Burgess Assignee: Matt Burgess Fix For: 1.4.0 NIFI-2583 added the ability (via dynamic properties) to specify initial Max Values for columns, to enable the user to "pick up where they left off" if something happened with a flow, a NiFi instance, etc. where the state was stored but the processing did not complete successfully. This feature would also be helpful in GenerateTableFetch, which also supports max-value columns. Since NIFI-2881 adds incoming flow file support, it's more useful if Initial max values can be specified via flow file attributes. Because if a table name is dynamically passed via flow file attribute and Expression Language, user won't be able to configure dynamic processor attribute in advance for each possible table. Add dynamic properties ('initial.maxvalue.' same as QueryDatabaseTable) to specify initial max values statically, and also use incoming flow file attributes named 'initial.maxvalue.' if any. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Deleted] (NIFI-4282) GenerateTableFetch should allow you to specify an initial Max Value (cloned)
[ https://issues.apache.org/jira/browse/NIFI-4282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt deleted NIFI-4282: -- > GenerateTableFetch should allow you to specify an initial Max Value (cloned) > > > Key: NIFI-4282 > URL: https://issues.apache.org/jira/browse/NIFI-4282 > Project: Apache NiFi > Issue Type: Task >Reporter: Matt Burgess >Assignee: Matt Burgess > > NIFI-2583 added the ability (via dynamic properties) to specify initial Max > Values for columns, to enable the user to "pick up where they left off" if > something happened with a flow, a NiFi instance, etc. where the state was > stored but the processing did not complete successfully. > This feature would also be helpful in GenerateTableFetch, which also supports > max-value columns. > Since NIFI-2881 adds incoming flow file support, it's more useful if Initial > max values can be specified via flow file attributes. Because if a table name > is dynamically passed via flow file attribute and Expression Language, user > won't be able to configure dynamic processor attribute in advance for each > possible table. > Add dynamic properties ('initial.maxvalue.' same as > QueryDatabaseTable) to specify initial max values statically, and also use > incoming flow file attributes named 'initial.maxvalue.' if > any. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-3335) GenerateTableFetch should allow you to specify an initial Max Value
[ https://issues.apache.org/jira/browse/NIFI-3335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-3335: -- Issue Type: Task (was: Improvement) > GenerateTableFetch should allow you to specify an initial Max Value > --- > > Key: NIFI-3335 > URL: https://issues.apache.org/jira/browse/NIFI-3335 > Project: Apache NiFi > Issue Type: Task > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.4.0 > > > NIFI-2583 added the ability (via dynamic properties) to specify initial Max > Values for columns, to enable the user to "pick up where they left off" if > something happened with a flow, a NiFi instance, etc. where the state was > stored but the processing did not complete successfully. > This feature would also be helpful in GenerateTableFetch, which also supports > max-value columns. > Since NIFI-2881 adds incoming flow file support, it's more useful if Initial > max values can be specified via flow file attributes. Because if a table name > is dynamically passed via flow file attribute and Expression Language, user > won't be able to configure dynamic processor attribute in advance for each > possible table. > Add dynamic properties ('initial.maxvalue.' same as > QueryDatabaseTable) to specify initial max values statically, and also use > incoming flow file attributes named 'initial.maxvalue.' if > any. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-3329) MergeContent throws FlowFileHandlingException: not the most recent version of this FlowFile within this session
[ https://issues.apache.org/jira/browse/NIFI-3329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-3329: -- Resolution: Fixed Status: Resolved (was: Patch Available) +1 merged to master. Forgot to do signoff commit entry. > MergeContent throws FlowFileHandlingException: not the most recent version of > this FlowFile within this session > --- > > Key: NIFI-3329 > URL: https://issues.apache.org/jira/browse/NIFI-3329 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.1.1 >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.4.0 > > Attachments: simple_flow.xml > > > The following error was reported on the dev mailing list: > {code} > This error occurred in NiFi 1.1.1 (single node) right after an upgrade from > 1.1.0, so I thought it was due to the upgrade, but now I see that it happens > if I restart as well and there is data in the flow. > This MergeContent processor is sorting incoming FlowFiles into bins based on > a correlation attribute and dumping out the accumulated uber-FlowFile after > one hour or 500K messages are accumulated. > ... > 2017-01-11 10:53:39,561 WARN [Timer-Driven Process Thread-9] > o.a.n.processors.standard.MergeContent > MergeContent[id=b9898788-1c11-45d9-8646-44b1dceb92d5] Processor > Administratively Yielded for 1 sec due to processing failure > 2017-01-11 10:53:39,561 WARN [Timer-Driven Process Thread-9] > o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding > MergeContent[id=b9898788-1c11-45d9-8646-44b1dceb92d5] due to uncaught > Exception: org.apache.nifi.processor.exception.FlowFileHandlingException: > StandardFlowFileRecord[uuid=07f3d75a-3347-45da-8dd4-fecf775d1dcf,claim=StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1484151577298-1971, > container=default, section=947], offset=566526, > length=64],offset=0,name=8551433629757608.tsv,size=64] is not the most > recent version of this FlowFile within this session > (StandardProcessSession[id=133767]) > 2017-01-11 10:53:39,563 WARN [Timer-Driven Process Thread-9] > o.a.n.c.t.ContinuallyRunProcessorTask > org.apache.nifi.processor.exception.FlowFileHandlingException: > StandardFlowFileRecord[uuid=07f3d75a-3347-45da-8dd4-fecf775d1dcf,claim=StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1484151577298-1971, > container=default, section=947], offset=566526, > length=64],offset=0,name=8551433629757608.tsv,size=64] is not the most > recent version of this FlowFile within this session > (StandardProcessSession[id=133767]) >at > org.apache.nifi.controller.repository.StandardProcessSession.migrate(StandardProcessSession.java:1121) > ~[nifi-framework-core-1.1.0.jar:1.1.0] >at > org.apache.nifi.controller.repository.StandardProcessSession.migrate(StandardProcessSession.java:1102) > ~[nifi-framework-core-1.1.0.jar:1.1.0] >at org.apache.nifi.processor.util.bin.Bin.offer(Bin.java:142) > ~[na:na] >at > org.apache.nifi.processor.util.bin.BinManager.offer(BinManager.java:194) > ~[na:na] >at > org.apache.nifi.processor.util.bin.BinFiles.binFlowFiles(BinFiles.java:279) > ~[na:na] >at > org.apache.nifi.processor.util.bin.BinFiles.onTrigger(BinFiles.java:178) > ~[na:na] >at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099) > ~[nifi-framework-core-1.1.0.jar:1.1.0] >at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) > [nifi-framework-core-1.1.0.jar:1.1.0] >at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) > [nifi-framework-core-1.1.0.jar:1.1.0] >at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) > [nifi-framework-core-1.1.0.jar:1.1.0] >at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_65] >at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > [na:1.8.0_65] >at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > [na:1.8.0_65] >at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > [na:1.8.0_65] >at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_65] >at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_65] >at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65] > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2053: NIFI-3329: Removed check for latest version of flow...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2053 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4208) Node failed to join cluster due to NullPointerException
[ https://issues.apache.org/jira/browse/NIFI-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122601#comment-16122601 ] Joseph Witt commented on NIFI-4208: --- [~markap14] Any more thoughts/plans on this? I see it is tagged to 1.4.0 and said to be critical but no update since then. > Node failed to join cluster due to NullPointerException > --- > > Key: NIFI-4208 > URL: https://issues.apache.org/jira/browse/NIFI-4208 > Project: Apache NiFi > Issue Type: Bug >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Critical > Fix For: 1.4.0 > > > A clustered node ran out of disk space. Upon restart, I came across the > following error: > 2017-07-20 09:03:00,988 ERROR [main] o.a.nifi.controller.StandardFlowService > Failed to load flow from cluster due to: > org.apache.nifi.cluster.ConnectionException: Failed to connect node to > cluster due to: java.lang.NullPointerException > org.apache.nifi.cluster.ConnectionException: Failed to connect node to > cluster due to: java.lang.NullPointerException > at > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:945) > at > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:515) > at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:800) > at org.apache.nifi.NiFi.(NiFi.java:160) > at org.apache.nifi.NiFi.main(NiFi.java:267) > Caused by: java.lang.NullPointerException: null > at > org.apache.nifi.controller.repository.RepositoryRecordSerde.getRecordIdentifier(RepositoryRecordSerde.java:43) > at > org.apache.nifi.controller.repository.RepositoryRecordSerde.getRecordIdentifier(RepositoryRecordSerde.java:26) > at > org.wali.MinimalLockingWriteAheadLog$Partition.recoverNextTransaction(MinimalLockingWriteAheadLog.java:1132) > at > org.wali.MinimalLockingWriteAheadLog.recoverFromEdits(MinimalLockingWriteAheadLog.java:459) > at > org.wali.MinimalLockingWriteAheadLog.recoverRecords(MinimalLockingWriteAheadLog.java:301) > at > org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.loadFlowFiles(WriteAheadFlowFileRepository.java:381) > at > org.apache.nifi.controller.FlowController.initializeFlow(FlowController.java:713) > at > org.apache.nifi.controller.StandardFlowService.initializeController(StandardFlowService.java:955) > at > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:927) > ... 4 common frames omitted -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4206) Include proxy instructions in admin guide
[ https://issues.apache.org/jira/browse/NIFI-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto updated NIFI-4206: Resolution: Fixed Status: Resolved (was: Patch Available) > Include proxy instructions in admin guide > - > > Key: NIFI-4206 > URL: https://issues.apache.org/jira/browse/NIFI-4206 > Project: Apache NiFi > Issue Type: Task > Components: Documentation & Website >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Minor > Fix For: 1.4.0 > > > Update the admin guide with instructions when running behind a proxy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4206) Include proxy instructions in admin guide
[ https://issues.apache.org/jira/browse/NIFI-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122575#comment-16122575 ] ASF GitHub Bot commented on NIFI-4206: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2023 I fixed a few formatting issues (see [e091b8f](https://github.com/alopresto/nifi/commit/e091b8fff5e8a6090b07cd8084bf4f7f76bc9424)) but other than that it looks good to me. Ran `contrib-check` and all tests pass. +1, merging. > Include proxy instructions in admin guide > - > > Key: NIFI-4206 > URL: https://issues.apache.org/jira/browse/NIFI-4206 > Project: Apache NiFi > Issue Type: Task > Components: Documentation & Website >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Minor > Fix For: 1.4.0 > > > Update the admin guide with instructions when running behind a proxy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2023: NIFI-4206: Proxy instructions in Admin Guide
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2023 I fixed a few formatting issues (see [e091b8f](https://github.com/alopresto/nifi/commit/e091b8fff5e8a6090b07cd8084bf4f7f76bc9424)) but other than that it looks good to me. Ran `contrib-check` and all tests pass. +1, merging. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4206) Include proxy instructions in admin guide
[ https://issues.apache.org/jira/browse/NIFI-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122572#comment-16122572 ] ASF subversion and git services commented on NIFI-4206: --- Commit ec0c8278b42916637c7ce76b05aff3f35a77ece8 in nifi's branch refs/heads/master from [~mcgilman] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=ec0c827 ] NIFI-4206: - Updating admin guide to include instructions for running NiFi behind a proxy. - Including a brief example proxy configuration for NiFi specific properties. This closes #2023. Signed-off-by: Andy LoPresto> Include proxy instructions in admin guide > - > > Key: NIFI-4206 > URL: https://issues.apache.org/jira/browse/NIFI-4206 > Project: Apache NiFi > Issue Type: Task > Components: Documentation & Website >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Minor > Fix For: 1.4.0 > > > Update the admin guide with instructions when running behind a proxy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4206) Include proxy instructions in admin guide
[ https://issues.apache.org/jira/browse/NIFI-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122573#comment-16122573 ] ASF GitHub Bot commented on NIFI-4206: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2023 > Include proxy instructions in admin guide > - > > Key: NIFI-4206 > URL: https://issues.apache.org/jira/browse/NIFI-4206 > Project: Apache NiFi > Issue Type: Task > Components: Documentation & Website >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Minor > Fix For: 1.4.0 > > > Update the admin guide with instructions when running behind a proxy. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2023: NIFI-4206: Proxy instructions in Admin Guide
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2023 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-4187) If NiFi process is killed, bootstrap auto-restart hangs on missing sensitive key file
[ https://issues.apache.org/jira/browse/NIFI-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto updated NIFI-4187: Assignee: Andy LoPresto Status: Patch Available (was: Open) > If NiFi process is killed, bootstrap auto-restart hangs on missing sensitive > key file > - > > Key: NIFI-4187 > URL: https://issues.apache.org/jira/browse/NIFI-4187 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Critical > Labels: bootstrap, encryption > > If the NiFi process is killed and {{autoRestart}} is enabled in the bootstrap > process ({{RunNiFi.java}}) (enabled by default except when in {{run}} mode) > and the instance is using encrypted configuration files, the > {{sensitive.key}} file which contains the master encryption key is no longer > available when restart occurs. This results in the application exiting. > To resolve this, the logic to prepare and generate the sensitive key file > should be extracted to a method and invoked during initial start *and* during > the auto-restart loop at > https://github.com/apache/nifi/blob/master/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java#L1159 > . -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4187) If NiFi process is killed, bootstrap auto-restart hangs on missing sensitive key file
[ https://issues.apache.org/jira/browse/NIFI-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122529#comment-16122529 ] ASF GitHub Bot commented on NIFI-4187: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2073 To verify this fix: 1. modify the `conf/bootstrap.conf` file to provide a *master encryption key* (it doesn't matter if any `conf/nifi.properties` values are encrypted with it, the mere presence will trigger the relevant code) 1. Start NiFi as normal (i.e. `$ ./bin/nifi.sh start`) 1. Verify that NiFi starts (UI, `nifi-bootstrap.log`, `nifi-app.log`, etc.) 1. Identify the NiFi process (*not* the bootstrap process) and kill it (imitating an `OOME` or external process killing NiFi) * `ps -ef | grep nifi` to get the PID * `kill -9 ` 1. Verify that NiFi successfully restarts The `logs/nifi-bootstrap.log` will look like this: ``` 2017-08-10 16:08:49,675 INFO [main] o.a.n.b.NotificationServiceManager Successfully loaded the following 0 services: [] 2017-08-10 16:08:49,679 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_STARTED 2017-08-10 16:08:49,679 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_STOPPED 2017-08-10 16:08:49,679 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_DIED 2017-08-10 16:08:49,736 INFO [main] org.apache.nifi.bootstrap.Command Starting Apache NiFi... 2017-08-10 16:08:49,736 INFO [main] org.apache.nifi.bootstrap.Command Working Directory: /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT 2017-08-10 16:08:49,736 INFO [main] org.apache.nifi.bootstrap.Command Command: /Users/alopresto/.jenv/versions/1.8/bin/java -classpath /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./conf:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/javax.servlet-api-3.1.0.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/jcl-over-slf4j-1.7.25.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/jetty-schemas-3.1.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/jul-to-slf4j-1.7.25.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/log4j-over-slf4j-1.7.25.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/logback-classic-1.2.3.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/logback-core-1.2.3.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-api-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-framework-api-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-nar-utils-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-properties-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-runtime-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/slf4j-api-1.7.25.jar -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m -Djava.security.egd=file:/dev/urandom -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -XX:+UseG1GC -Djava.protocol.handler.pkgs=sun.net.www.protocol -Dnifi.properties.file.path=/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./conf/nifi.properties -Dnifi.bootstrap.listen.port=52192 -Dapp=NiFi -Dorg.apache.nifi.bootstrap.config.log.dir=/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/logs org.apache.nifi.NiFi -K /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./conf/sensitive.key 2017-08-10 16:08:49,749 INFO [main] org.apache.nifi.bootstrap.Command Launched Apache NiFi with Process ID 90937 2017-08-10 16:08:50,294 INFO [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for Bootstrap requests on port 52193 -External command kills NiFi process- 2017-08-10 16:11:13,185 WARN [main]
[GitHub] nifi issue #2073: NIFI-4187 Resolved issue where restart failed due to missi...
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2073 To verify this fix: 1. modify the `conf/bootstrap.conf` file to provide a *master encryption key* (it doesn't matter if any `conf/nifi.properties` values are encrypted with it, the mere presence will trigger the relevant code) 1. Start NiFi as normal (i.e. `$ ./bin/nifi.sh start`) 1. Verify that NiFi starts (UI, `nifi-bootstrap.log`, `nifi-app.log`, etc.) 1. Identify the NiFi process (*not* the bootstrap process) and kill it (imitating an `OOME` or external process killing NiFi) * `ps -ef | grep nifi` to get the PID * `kill -9 ` 1. Verify that NiFi successfully restarts The `logs/nifi-bootstrap.log` will look like this: ``` 2017-08-10 16:08:49,675 INFO [main] o.a.n.b.NotificationServiceManager Successfully loaded the following 0 services: [] 2017-08-10 16:08:49,679 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_STARTED 2017-08-10 16:08:49,679 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_STOPPED 2017-08-10 16:08:49,679 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_DIED 2017-08-10 16:08:49,736 INFO [main] org.apache.nifi.bootstrap.Command Starting Apache NiFi... 2017-08-10 16:08:49,736 INFO [main] org.apache.nifi.bootstrap.Command Working Directory: /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT 2017-08-10 16:08:49,736 INFO [main] org.apache.nifi.bootstrap.Command Command: /Users/alopresto/.jenv/versions/1.8/bin/java -classpath /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./conf:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/javax.servlet-api-3.1.0.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/jcl-over-slf4j-1.7.25.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/jetty-schemas-3.1.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/jul-to-slf4j-1.7.25.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/log4j-over-slf4j-1.7.25.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/logback-c lassic-1.2.3.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/logback-core-1.2.3.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-api-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-framework-api-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-nar-utils-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-properties-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-runtime-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/slf4j-api-1.7.25.jar -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m -Djava.security.egd=file:/dev/urandom -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -XX:+UseG1GC -Djava.protocol.handler.pkgs=sun.net.www.protocol -Dnifi.properties.file.path=/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./conf/nifi.properties -Dnifi.bootstrap.listen.port=52192 -Dapp=NiFi -Dorg.apache.nifi.bootstrap.config.log.dir=/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/logs org.apache.nifi.NiFi -K /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./conf/sensitive.key 2017-08-10 16:08:49,749 INFO [main] org.apache.nifi.bootstrap.Command Launched Apache NiFi with Process ID 90937 2017-08-10 16:08:50,294 INFO [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for Bootstrap requests on port 52193 -External command kills NiFi process- 2017-08-10 16:11:13,185 WARN [main] org.apache.nifi.bootstrap.RunNiFi Apache NiFi appears to have died. Restarting... 2017-08-10 16:11:13,195 INFO [main] org.apache.nifi.bootstrap.Command Launched Apache NiFi with Process ID 91211 2017-08-10 16:11:13,196 INFO [main]
[jira] [Commented] (NIFI-4187) If NiFi process is killed, bootstrap auto-restart hangs on missing sensitive key file
[ https://issues.apache.org/jira/browse/NIFI-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122517#comment-16122517 ] ASF GitHub Bot commented on NIFI-4187: -- GitHub user alopresto opened a pull request: https://github.com/apache/nifi/pull/2073 NIFI-4187 Resolved issue where restart failed due to missing sensitivā¦ ā¦e key file. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/alopresto/nifi NIFI-4187 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2073.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2073 commit 7dc0bd2084820a9f924a0593f2e938efb1a0ceed Author: Andy LoPrestoDate: 2017-08-10T23:16:29Z NIFI-4187 Resolved issue where restart failed due to missing sensitive key file. > If NiFi process is killed, bootstrap auto-restart hangs on missing sensitive > key file > - > > Key: NIFI-4187 > URL: https://issues.apache.org/jira/browse/NIFI-4187 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Priority: Critical > Labels: bootstrap, encryption > > If the NiFi process is killed and {{autoRestart}} is enabled in the bootstrap > process ({{RunNiFi.java}}) (enabled by default except when in {{run}} mode) > and the instance is using encrypted configuration files, the > {{sensitive.key}} file which contains the master encryption key is no longer > available when restart occurs. This results in the application exiting. > To resolve this, the logic to prepare and generate the sensitive key file > should be extracted to a method and invoked during initial start *and* during > the auto-restart loop at > https://github.com/apache/nifi/blob/master/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java#L1159 > . -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2073: NIFI-4187 Resolved issue where restart failed due t...
GitHub user alopresto opened a pull request: https://github.com/apache/nifi/pull/2073 NIFI-4187 Resolved issue where restart failed due to missing sensitivĆ¢ĀĀ¦ Ć¢ĀĀ¦e key file. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/alopresto/nifi NIFI-4187 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2073.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2073 commit 7dc0bd2084820a9f924a0593f2e938efb1a0ceed Author: Andy LoPrestoDate: 2017-08-10T23:16:29Z NIFI-4187 Resolved issue where restart failed due to missing sensitive key file. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (NIFI-4281) ExecuteScript with ruby throws IndexOutOfBoundsException on all scripts
Jeremy Lautman created NIFI-4281: Summary: ExecuteScript with ruby throws IndexOutOfBoundsException on all scripts Key: NIFI-4281 URL: https://issues.apache.org/jira/browse/NIFI-4281 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.3.0 Reporter: Jeremy Lautman In running ExecuteScript with the following ruby script, ExecuteScript throws IndexOutOfBoundsExceptions. {quote}session.transfer(session.get(), REL_SUCCESS){quote} The same script with python works appropriately. It also does the same when using the session::get(int) variant, and when running larger scripts. NiFi version 1.3.0 Java openjdk version 1.8.0_121 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2020: [NiFi-3973] Add PutKudu Processor for ingesting data to Ku...
Github user rickysaltzer commented on the issue: https://github.com/apache/nifi/pull/2020 @cammachusa seems I can't update this PR with my own code. My changes are below so you can merge them in and then push them again. https://github.com/rickysaltzer/nifi/commit/c17bbaff08685f74eef037f5e34dd6339b3aac68 I was able to get a sample pipeline working locally which is great. I want to do a little bit more testing before we merge it in. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4276) Add Provenance Migration section to User Guide
[ https://issues.apache.org/jira/browse/NIFI-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122329#comment-16122329 ] Andrew Lim commented on NIFI-4276: -- Adding to the Data Provenance section of the User Guide makes more sense than the Admin Guide. There is also a section in the User Guide regarding the Encrypted Provenance Repository which relates to this migration. > Add Provenance Migration section to User Guide > -- > > Key: NIFI-4276 > URL: https://issues.apache.org/jira/browse/NIFI-4276 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Minor > > WriteAheadProvenanceRepository configuration was introduced in version 1.2.0 > (NIFI-3356). Should add a section to the documentation that covers how to > migrate from default configuration of PersistentProvenantRepository and > recommend any system property value changes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4276) Add Provenance Migration section to User Guide
[ https://issues.apache.org/jira/browse/NIFI-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Lim updated NIFI-4276: - Summary: Add Provenance Migration section to User Guide (was: Add Provenance Migration section to Admin Guide) > Add Provenance Migration section to User Guide > -- > > Key: NIFI-4276 > URL: https://issues.apache.org/jira/browse/NIFI-4276 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Minor > > WriteAheadProvenanceRepository configuration was introduced in version 1.2.0 > (NIFI-3356). Should add a section to the documentation that covers how to > migrate from default configuration of PersistentProvenantRepository and > recommend any system property value changes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-3232) Extend UI menus to allow cascading (menu item > subitem)
[ https://issues.apache.org/jira/browse/NIFI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan updated NIFI-3232: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Extend UI menus to allow cascading (menu item > subitem) > > > Key: NIFI-3232 > URL: https://issues.apache.org/jira/browse/NIFI-3232 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Rob Moran >Assignee: Matt Gilman > > With expanding functionality UI menu options are growing, making the size of > some quite large. Cascading menus will allow better information hierarchy to > improve the presentation of available user actions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-3232) Extend UI menus to allow cascading (menu item > subitem)
[ https://issues.apache.org/jira/browse/NIFI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122327#comment-16122327 ] ASF GitHub Bot commented on NIFI-3232: -- Github user scottyaslan commented on the issue: https://github.com/apache/nifi/pull/2072 Thanks @mcgilman this has been merged to master! > Extend UI menus to allow cascading (menu item > subitem) > > > Key: NIFI-3232 > URL: https://issues.apache.org/jira/browse/NIFI-3232 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Rob Moran >Assignee: Matt Gilman > > With expanding functionality UI menu options are growing, making the size of > some quite large. Cascading menus will allow better information hierarchy to > improve the presentation of available user actions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2072: NIFI-3232: Adding support for cascading sub context menus
Github user scottyaslan commented on the issue: https://github.com/apache/nifi/pull/2072 Thanks @mcgilman this has been merged to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3232) Extend UI menus to allow cascading (menu item > subitem)
[ https://issues.apache.org/jira/browse/NIFI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122323#comment-16122323 ] ASF GitHub Bot commented on NIFI-3232: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2072 > Extend UI menus to allow cascading (menu item > subitem) > > > Key: NIFI-3232 > URL: https://issues.apache.org/jira/browse/NIFI-3232 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Rob Moran >Assignee: Matt Gilman > > With expanding functionality UI menu options are growing, making the size of > some quite large. Cascading menus will allow better information hierarchy to > improve the presentation of available user actions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-3232) Extend UI menus to allow cascading (menu item > subitem)
[ https://issues.apache.org/jira/browse/NIFI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122321#comment-16122321 ] ASF subversion and git services commented on NIFI-3232: --- Commit 20d6596df097e4fd3716b25ff9927e93f7bff3a9 in nifi's branch refs/heads/master from [~mcgilman] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=20d6596 ] NIFI-3232: - Adding support for cascading sub context menus. Signed-off-by: Scott AslanThis closes #2072 > Extend UI menus to allow cascading (menu item > subitem) > > > Key: NIFI-3232 > URL: https://issues.apache.org/jira/browse/NIFI-3232 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Rob Moran >Assignee: Matt Gilman > > With expanding functionality UI menu options are growing, making the size of > some quite large. Cascading menus will allow better information hierarchy to > improve the presentation of available user actions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2020: [NiFi-3973] Add PutKudu Processor for ingesting data to Ku...
Github user rickysaltzer commented on the issue: https://github.com/apache/nifi/pull/2020 I've made some changes that allows the tests to run successfully. I'm going to test out this in an actual NiFi installation and then provide you my changes. Unfortunately this may not happen today due to my schedule. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #2072: NIFI-3232: Adding support for cascading sub context...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2072 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level
[ https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122297#comment-16122297 ] ASF GitHub Bot commented on NIFI-4224: -- Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/2051 @mcgilman thanks for the thorough review! I have pushed a new commit, which I believe addresses all of the feedback above. > Add Variable Registry at Process Group level > > > Key: NIFI-4224 > URL: https://issues.apache.org/jira/browse/NIFI-4224 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne > > Currently, NiFi exposes a variable registry that is configurable by adding > the name of a properties file to nifi.properties and then treating the > referenced properties file as key/value pairs for the variable registry. > This, however, is very limiting, as it provides a global scope for all > variables, and it requires a restart of NiFi in order to pick up any updates > to the file. We should expose a Process Group-level Variable Registry. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2051: NIFI-4224: Initial implementation of Process Group level V...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/2051 @mcgilman thanks for the thorough review! I have pushed a new commit, which I believe addresses all of the feedback above. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (NIFI-4280) UI - Support configuring variables at a Process Group level
Matt Gilman created NIFI-4280: - Summary: UI - Support configuring variables at a Process Group level Key: NIFI-4280 URL: https://issues.apache.org/jira/browse/NIFI-4280 Project: Apache NiFi Issue Type: New Feature Components: Core UI Reporter: Matt Gilman Assignee: Matt Gilman Add support for configuring variables at a Process Group level. Variables, like Controller Services, are scoped by the group they are defined in and are available to any Processor defined there and below (ie and descendant Processors). Variables in a descendant group can override the value in a parent group. Variable permissions will be based solely on the corresponding Process Group. Because variables are specified in EL, there is no controlling access to them like Controller Services which can be restricted using the combo selector. Because of this, variables should not be used for sensitive values. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4032) Create Managed Ranger Authorizer
[ https://issues.apache.org/jira/browse/NIFI-4032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-4032: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Create Managed Ranger Authorizer > > > Key: NIFI-4032 > URL: https://issues.apache.org/jira/browse/NIFI-4032 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.4.0 > > > Update the RangerAuthorizer to implement the ManagedAuthorizer interface. > This will allow the Ranger policies to be visualized in NiFi UI. May even be > able to extend the RangerAuthorizer to maintain compatibility with existing > configurations. > Additionally, update the RangerAuthorizer's authorize(...) method to consider > the user's groups. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-968) Add Primary Node Scheduling to User and Dev Guide
[ https://issues.apache.org/jira/browse/NIFI-968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto updated NIFI-968: --- Resolution: Fixed Fix Version/s: 1.4.0 Status: Resolved (was: Patch Available) > Add Primary Node Scheduling to User and Dev Guide > - > > Key: NIFI-968 > URL: https://issues.apache.org/jira/browse/NIFI-968 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Joseph Percivall >Assignee: Pierre Villard >Priority: Minor > Fix For: 1.4.0 > > > The User Guide talks about every other scheduling type except Primary Node > Only. This should be noted as an option in cluster environments. > In the Dev Guide, every other annotation is talked about except for > OnPrimaryNodeStateChange. It should be added and potentially some information > about cluster environment considerations when coding could be added too > (currently nothing regarding cluster environments in dev guide). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-968) Add Primary Node Scheduling to User and Dev Guide
[ https://issues.apache.org/jira/browse/NIFI-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122160#comment-16122160 ] ASF GitHub Bot commented on NIFI-968: - Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2027 > Add Primary Node Scheduling to User and Dev Guide > - > > Key: NIFI-968 > URL: https://issues.apache.org/jira/browse/NIFI-968 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Joseph Percivall >Assignee: Pierre Villard >Priority: Minor > > The User Guide talks about every other scheduling type except Primary Node > Only. This should be noted as an option in cluster environments. > In the Dev Guide, every other annotation is talked about except for > OnPrimaryNodeStateChange. It should be added and potentially some information > about cluster environment considerations when coding could be added too > (currently nothing regarding cluster environments in dev guide). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-968) Add Primary Node Scheduling to User and Dev Guide
[ https://issues.apache.org/jira/browse/NIFI-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122158#comment-16122158 ] ASF subversion and git services commented on NIFI-968: -- Commit c03f4e731bbc9ac3a74d1e1ca800942b5b73dc23 in nifi's branch refs/heads/master from [~pvillard] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=c03f4e7 ] NIFI-968 - add @OnPrimaryNodeStateChange to dev guide This closes #2027. Signed-off-by: Andy LoPresto> Add Primary Node Scheduling to User and Dev Guide > - > > Key: NIFI-968 > URL: https://issues.apache.org/jira/browse/NIFI-968 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Joseph Percivall >Assignee: Pierre Villard >Priority: Minor > > The User Guide talks about every other scheduling type except Primary Node > Only. This should be noted as an option in cluster environments. > In the Dev Guide, every other annotation is talked about except for > OnPrimaryNodeStateChange. It should be added and potentially some information > about cluster environment considerations when coding could be added too > (currently nothing regarding cluster environments in dev guide). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-968) Add Primary Node Scheduling to User and Dev Guide
[ https://issues.apache.org/jira/browse/NIFI-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122157#comment-16122157 ] ASF GitHub Bot commented on NIFI-968: - Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2027 Fixed a couple typos. Ran `contrib-check` and all tests pass. +1, merging. > Add Primary Node Scheduling to User and Dev Guide > - > > Key: NIFI-968 > URL: https://issues.apache.org/jira/browse/NIFI-968 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Joseph Percivall >Assignee: Pierre Villard >Priority: Minor > > The User Guide talks about every other scheduling type except Primary Node > Only. This should be noted as an option in cluster environments. > In the Dev Guide, every other annotation is talked about except for > OnPrimaryNodeStateChange. It should be added and potentially some information > about cluster environment considerations when coding could be added too > (currently nothing regarding cluster environments in dev guide). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2020: [NiFi-3973] Add PutKudu Processor for ingesting data to Ku...
Github user rickysaltzer commented on the issue: https://github.com/apache/nifi/pull/2020 @cammachusa thanks for the reminder :) - I've been a bit busy due to business travel. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #2027: NIFI-968 - add @OnPrimaryNodeStateChange to dev gui...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2027 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #2027: NIFI-968 - add @OnPrimaryNodeStateChange to dev guide
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2027 Fixed a couple typos. Ran `contrib-check` and all tests pass. +1, merging. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3232) Extend UI menus to allow cascading (menu item > subitem)
[ https://issues.apache.org/jira/browse/NIFI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122148#comment-16122148 ] ASF GitHub Bot commented on NIFI-3232: -- Github user scottyaslan commented on the issue: https://github.com/apache/nifi/pull/2072 Reviewing... > Extend UI menus to allow cascading (menu item > subitem) > > > Key: NIFI-3232 > URL: https://issues.apache.org/jira/browse/NIFI-3232 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Rob Moran >Assignee: Matt Gilman > > With expanding functionality UI menu options are growing, making the size of > some quite large. Cascading menus will allow better information hierarchy to > improve the presentation of available user actions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2072: NIFI-3232: Adding support for cascading sub context menus
Github user scottyaslan commented on the issue: https://github.com/apache/nifi/pull/2072 Reviewing... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-3232) Extend UI menus to allow cascading (menu item > subitem)
[ https://issues.apache.org/jira/browse/NIFI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-3232: -- Status: Patch Available (was: In Progress) > Extend UI menus to allow cascading (menu item > subitem) > > > Key: NIFI-3232 > URL: https://issues.apache.org/jira/browse/NIFI-3232 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Rob Moran >Assignee: Matt Gilman > > With expanding functionality UI menu options are growing, making the size of > some quite large. Cascading menus will allow better information hierarchy to > improve the presentation of available user actions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-3232) Extend UI menus to allow cascading (menu item > subitem)
[ https://issues.apache.org/jira/browse/NIFI-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122142#comment-16122142 ] ASF GitHub Bot commented on NIFI-3232: -- GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/2072 NIFI-3232: Adding support for cascading sub context menus NIFI-3232: - Adding support for cascading sub context menus. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-3232 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2072.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2072 commit bfb3f33e547e60e8d1a8100cf1dfd5727c3f3d37 Author: Matt GilmanDate: 2017-08-10T19:09:58Z NIFI-3232: - Adding support for cascading sub context menus. > Extend UI menus to allow cascading (menu item > subitem) > > > Key: NIFI-3232 > URL: https://issues.apache.org/jira/browse/NIFI-3232 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Rob Moran >Assignee: Matt Gilman > > With expanding functionality UI menu options are growing, making the size of > some quite large. Cascading menus will allow better information hierarchy to > improve the presentation of available user actions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2072: NIFI-3232: Adding support for cascading sub context...
GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/2072 NIFI-3232: Adding support for cascading sub context menus NIFI-3232: - Adding support for cascading sub context menus. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-3232 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2072.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2072 commit bfb3f33e547e60e8d1a8100cf1dfd5727c3f3d37 Author: Matt GilmanDate: 2017-08-10T19:09:58Z NIFI-3232: - Adding support for cascading sub context menus. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #2027: NIFI-968 - add @OnPrimaryNodeStateChange to dev guide
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2027 Reviewing... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4277) StandardLogRepository does not log exceptions
[ https://issues.apache.org/jira/browse/NIFI-4277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122104#comment-16122104 ] ASF GitHub Bot commented on NIFI-4277: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2068 Ran `contrib-check` and all tests pass. +1, merging. > StandardLogRepository does not log exceptions > - > > Key: NIFI-4277 > URL: https://issues.apache.org/jira/browse/NIFI-4277 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.3.0 >Reporter: Pierre Villard >Assignee: Pierre Villard > Attachments: Screen Shot 2017-08-08 at 2.48.33 PM.png > > > When logging a message, it is logged with the SLF4J logger and also stored in > the standard log repository (for the bulletins). However if the array of > objects contains the exception (and not the message of the exception), this > exception won't be displayed in the bulletin message. > That's because of: > {code:title=StandardLogRepository.java|borderStyle=solid} > @Override > public void addLogMessage(final LogLevel level, final String format, > final Object[] params) { > final String formattedMessage = MessageFormatter.arrayFormat(format, > params).getMessage(); > addLogMessage(level, formattedMessage); > } > {code} > If the params object contains a Throwable object, it'll be removed from the > array in the {{MessageFormatter}}: > {code:title=MessageFormatter.java|borderStyle=solid} > final public static FormattingTuple arrayFormat(final String > messagePattern, final Object[] argArray) { > Throwable throwableCandidate = getThrowableCandidate(argArray); > Object[] args = argArray; > if (throwableCandidate != null) { > args = trimmedCopy(argArray); > } > return arrayFormat(messagePattern, args, throwableCandidate); > } > {code} > Easy solution would be to change: > {noformat} > logger.debug("Failed to validate {} against schema due to {}", new > Object[]{flowFile, e}); > {noformat} > into: > {noformat} > logger.debug("Failed to validate {} against schema due to {}", new > Object[]{flowFile, e.getLocalizedMessage()}); > {noformat} > However this pattern can be found in quite a large number of places... And > it'd be certainly better to provide a permanent solution supporting the > existing pattern. Suggestion is to modify the method in > {{StandardLogRepository}} to go through all the items of the array and for > each Throwable object, replace it by the localized message. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4277) StandardLogRepository does not log exceptions
[ https://issues.apache.org/jira/browse/NIFI-4277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122097#comment-16122097 ] ASF GitHub Bot commented on NIFI-4277: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2068 > StandardLogRepository does not log exceptions > - > > Key: NIFI-4277 > URL: https://issues.apache.org/jira/browse/NIFI-4277 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.3.0 >Reporter: Pierre Villard >Assignee: Pierre Villard > Attachments: Screen Shot 2017-08-08 at 2.48.33 PM.png > > > When logging a message, it is logged with the SLF4J logger and also stored in > the standard log repository (for the bulletins). However if the array of > objects contains the exception (and not the message of the exception), this > exception won't be displayed in the bulletin message. > That's because of: > {code:title=StandardLogRepository.java|borderStyle=solid} > @Override > public void addLogMessage(final LogLevel level, final String format, > final Object[] params) { > final String formattedMessage = MessageFormatter.arrayFormat(format, > params).getMessage(); > addLogMessage(level, formattedMessage); > } > {code} > If the params object contains a Throwable object, it'll be removed from the > array in the {{MessageFormatter}}: > {code:title=MessageFormatter.java|borderStyle=solid} > final public static FormattingTuple arrayFormat(final String > messagePattern, final Object[] argArray) { > Throwable throwableCandidate = getThrowableCandidate(argArray); > Object[] args = argArray; > if (throwableCandidate != null) { > args = trimmedCopy(argArray); > } > return arrayFormat(messagePattern, args, throwableCandidate); > } > {code} > Easy solution would be to change: > {noformat} > logger.debug("Failed to validate {} against schema due to {}", new > Object[]{flowFile, e}); > {noformat} > into: > {noformat} > logger.debug("Failed to validate {} against schema due to {}", new > Object[]{flowFile, e.getLocalizedMessage()}); > {noformat} > However this pattern can be found in quite a large number of places... And > it'd be certainly better to provide a permanent solution supporting the > existing pattern. Suggestion is to modify the method in > {{StandardLogRepository}} to go through all the items of the array and for > each Throwable object, replace it by the localized message. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4277) StandardLogRepository does not log exceptions
[ https://issues.apache.org/jira/browse/NIFI-4277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122095#comment-16122095 ] ASF subversion and git services commented on NIFI-4277: --- Commit 18c82eb6af6648460365f8db286729629258e95c in nifi's branch refs/heads/master from [~pvillard] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=18c82eb ] NIFI-4277 Fixed exception logging in StandardLogRepository This closes #2068. Signed-off-by: Andy LoPresto> StandardLogRepository does not log exceptions > - > > Key: NIFI-4277 > URL: https://issues.apache.org/jira/browse/NIFI-4277 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.3.0 >Reporter: Pierre Villard >Assignee: Pierre Villard > Attachments: Screen Shot 2017-08-08 at 2.48.33 PM.png > > > When logging a message, it is logged with the SLF4J logger and also stored in > the standard log repository (for the bulletins). However if the array of > objects contains the exception (and not the message of the exception), this > exception won't be displayed in the bulletin message. > That's because of: > {code:title=StandardLogRepository.java|borderStyle=solid} > @Override > public void addLogMessage(final LogLevel level, final String format, > final Object[] params) { > final String formattedMessage = MessageFormatter.arrayFormat(format, > params).getMessage(); > addLogMessage(level, formattedMessage); > } > {code} > If the params object contains a Throwable object, it'll be removed from the > array in the {{MessageFormatter}}: > {code:title=MessageFormatter.java|borderStyle=solid} > final public static FormattingTuple arrayFormat(final String > messagePattern, final Object[] argArray) { > Throwable throwableCandidate = getThrowableCandidate(argArray); > Object[] args = argArray; > if (throwableCandidate != null) { > args = trimmedCopy(argArray); > } > return arrayFormat(messagePattern, args, throwableCandidate); > } > {code} > Easy solution would be to change: > {noformat} > logger.debug("Failed to validate {} against schema due to {}", new > Object[]{flowFile, e}); > {noformat} > into: > {noformat} > logger.debug("Failed to validate {} against schema due to {}", new > Object[]{flowFile, e.getLocalizedMessage()}); > {noformat} > However this pattern can be found in quite a large number of places... And > it'd be certainly better to provide a permanent solution supporting the > existing pattern. Suggestion is to modify the method in > {{StandardLogRepository}} to go through all the items of the array and for > each Throwable object, replace it by the localized message. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2068: NIFI-4277 Fixed exception logging in StandardLogRep...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2068 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4277) StandardLogRepository does not log exceptions
[ https://issues.apache.org/jira/browse/NIFI-4277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122037#comment-16122037 ] ASF GitHub Bot commented on NIFI-4277: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2068 Reviewing... > StandardLogRepository does not log exceptions > - > > Key: NIFI-4277 > URL: https://issues.apache.org/jira/browse/NIFI-4277 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.3.0 >Reporter: Pierre Villard >Assignee: Pierre Villard > Attachments: Screen Shot 2017-08-08 at 2.48.33 PM.png > > > When logging a message, it is logged with the SLF4J logger and also stored in > the standard log repository (for the bulletins). However if the array of > objects contains the exception (and not the message of the exception), this > exception won't be displayed in the bulletin message. > That's because of: > {code:title=StandardLogRepository.java|borderStyle=solid} > @Override > public void addLogMessage(final LogLevel level, final String format, > final Object[] params) { > final String formattedMessage = MessageFormatter.arrayFormat(format, > params).getMessage(); > addLogMessage(level, formattedMessage); > } > {code} > If the params object contains a Throwable object, it'll be removed from the > array in the {{MessageFormatter}}: > {code:title=MessageFormatter.java|borderStyle=solid} > final public static FormattingTuple arrayFormat(final String > messagePattern, final Object[] argArray) { > Throwable throwableCandidate = getThrowableCandidate(argArray); > Object[] args = argArray; > if (throwableCandidate != null) { > args = trimmedCopy(argArray); > } > return arrayFormat(messagePattern, args, throwableCandidate); > } > {code} > Easy solution would be to change: > {noformat} > logger.debug("Failed to validate {} against schema due to {}", new > Object[]{flowFile, e}); > {noformat} > into: > {noformat} > logger.debug("Failed to validate {} against schema due to {}", new > Object[]{flowFile, e.getLocalizedMessage()}); > {noformat} > However this pattern can be found in quite a large number of places... And > it'd be certainly better to provide a permanent solution supporting the > existing pattern. Suggestion is to modify the method in > {{StandardLogRepository}} to go through all the items of the array and for > each Throwable object, replace it by the localized message. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2068: NIFI-4277 Fixed exception logging in StandardLogRepository
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2068 Reviewing... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-4210) Add OpenId Connect support for authenticating users
[ https://issues.apache.org/jira/browse/NIFI-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto updated NIFI-4210: Resolution: Fixed Fix Version/s: 1.4.0 Status: Resolved (was: Patch Available) > Add OpenId Connect support for authenticating users > --- > > Key: NIFI-4210 > URL: https://issues.apache.org/jira/browse/NIFI-4210 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.4.0 > > > Add support for authenticating users with the OpenId Connection > specification. Evaluate whether a new extension point is necessary to allow > for a given provider to supply custom code for instance to implement custom > token validation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4210) Add OpenId Connect support for authenticating users
[ https://issues.apache.org/jira/browse/NIFI-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122022#comment-16122022 ] ASF GitHub Bot commented on NIFI-4210: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2047 Ran `contrib-check` and all tests pass. +1, merging. > Add OpenId Connect support for authenticating users > --- > > Key: NIFI-4210 > URL: https://issues.apache.org/jira/browse/NIFI-4210 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > > Add support for authenticating users with the OpenId Connection > specification. Evaluate whether a new extension point is necessary to allow > for a given provider to supply custom code for instance to implement custom > token validation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4210) Add OpenId Connect support for authenticating users
[ https://issues.apache.org/jira/browse/NIFI-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122024#comment-16122024 ] ASF GitHub Bot commented on NIFI-4210: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2047 > Add OpenId Connect support for authenticating users > --- > > Key: NIFI-4210 > URL: https://issues.apache.org/jira/browse/NIFI-4210 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > > Add support for authenticating users with the OpenId Connection > specification. Evaluate whether a new extension point is necessary to allow > for a given provider to supply custom code for instance to implement custom > token validation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2047: NIFI-4210: Add support for OpenId Connect
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2047 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4210) Add OpenId Connect support for authenticating users
[ https://issues.apache.org/jira/browse/NIFI-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122021#comment-16122021 ] ASF subversion and git services commented on NIFI-4210: --- Commit 528b82634f528468970eaa655c0085b1c9592b71 in nifi's branch refs/heads/master from [~mcgilman] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=528b826 ] NIFI-4210: - Introducing support for OpenId Connect. - Updating REST API and UI to support the authorization code flow. - Adding/fixing documentation. - Implementing time constant equality checks where appropriate. - Corrected error handling during startup and throughout the OIDC login sequence. - Redacting the token values from the user log. - Defaulting to RS256 when not preferred algorithm is specified. - Marking the OIDC endpoints as non-guaranteed in to allow for minor adjustments if/when additional SSO techniques are introduced. This closes #2047. Signed-off-by: Andy LoPresto> Add OpenId Connect support for authenticating users > --- > > Key: NIFI-4210 > URL: https://issues.apache.org/jira/browse/NIFI-4210 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > > Add support for authenticating users with the OpenId Connection > specification. Evaluate whether a new extension point is necessary to allow > for a given provider to supply custom code for instance to implement custom > token validation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2047: NIFI-4210: Add support for OpenId Connect
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2047 Ran `contrib-check` and all tests pass. +1, merging. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3927) Extract HL7 Attributes throwing NULLpointerException
[ https://issues.apache.org/jira/browse/NIFI-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121958#comment-16121958 ] Raj karan commented on NIFI-3927: - [~dmoore247] Have you tried the message containing Z segments; as Z segments were causing the problem. > Extract HL7 Attributes throwing NULLpointerException > > > Key: NIFI-3927 > URL: https://issues.apache.org/jira/browse/NIFI-3927 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: NiFi with HortonWorks >Reporter: Raj karan >Assignee: Joey Frazee > Attachments: null pointer.png, pipe_ended_charcter_encoded_ascii.png, > resultWithDefault.txt, source.txt, when not ended with pipe.png > > > I have an HL7 file which I want to put in HBase, So I am parsing this file > through ExtractHL7Attributes processor. With the default value for every > property processor works with no error but resultant attributes file only > have one segment. When I sets `Use Segment Names` property true it throws > NULLPointerException. > Stack trace: > 2017-05-17 11:11:58,390 INFO [Heartbeat Monitor Thread-1] > o.a.n.c.c.h.AbstractHeartbeatMonitor Finished processing 1 heartbeats in 4756 > nanos > 2017-05-17 11:11:58,847 ERROR [Timer-Driven Process Thread-2] > o.a.n.p.hl7.ExtractHL7Attributes > ExtractHL7Attributes[id=bea89fef-86db-1094--81c2e524] Failed to > extract attributes from > StandardFlowFileRecord[uuid=73a649fe-261c-40d2-bad7-b0bc595c0158,claim=StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1495030753601-25550, > container=default, section=974], offset=912561, > length=288],offset=0,name=source.txt,size=288] due to > ca.uhn.hl7v2.HL7Exception: The HL7 version 2.3 > EVN is not recognized: ca.uhn.hl7v2.HL7Exception: The HL7 version 2.3 > EVN is not recognized > 2017-05-17 11:11:58,848 ERROR [Timer-Driven Process Thread-2] > o.a.n.p.hl7.ExtractHL7Attributes > ca.uhn.hl7v2.HL7Exception: The HL7 version 2.3 > EVN is not recognized > at ca.uhn.hl7v2.parser.Parser.assertVersionExists(Parser.java:527) > ~[hapi-base-2.2.jar:na] > at ca.uhn.hl7v2.parser.Parser.parse(Parser.java:208) > ~[hapi-base-2.2.jar:na] > at ca.uhn.hl7v2.parser.PipeParser.parse(PipeParser.java:1018) > ~[hapi-base-2.2.jar:na] > at > org.apache.nifi.processors.hl7.ExtractHL7Attributes.onTrigger(ExtractHL7Attributes.java:195) > ~[nifi-hl7-processors-1.0.0.2.0.2.0-17.jar:1.0.0.2.0.2.0-17] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > [nifi-api-1.0.0.2.0.2.0-17.jar:1.0.0.2.0.2.0-17] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1064) > [nifi-framework-core-1.0.0.2.0.2.0-17.jar:1.0.0.2.0.2.0-17] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) > [nifi-framework-core-1.0.0.2.0.2.0-17.jar:1.0.0.2.0.2.0-17] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) > [nifi-framework-core-1.0.0.2.0.2.0-17.jar:1.0.0.2.0.2.0-17] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) > [nifi-framework-core-1.0.0.2.0.2.0-17.jar:1.0.0.2.0.2.0-17] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_77] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > [na:1.8.0_77] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > [na:1.8.0_77] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > [na:1.8.0_77] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_77] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_77] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77] > 2017-05-17 11:11:58,852 ERROR [Timer-Driven Process Thread-1] > o.a.n.p.hl7.ExtractHL7Attributes > ExtractHL7Attributes[id=bea89fef-86db-1094--81c2e524] > ExtractHL7Attributes[id=bea89fef-86db-1094--81c2e524] failed to > process due to java.lang.NullPointerException; rolling back session: > java.lang.NullPointerException > 2017-05-17 11:11:58,852 ERROR [Timer-Driven Process Thread-1] > o.a.n.p.hl7.ExtractHL7Attributes > java.lang.NullPointerException: null > 2017-05-17 11:11:58,852 ERROR [Timer-Driven Process Thread-1] > o.a.n.p.hl7.ExtractHL7Attributes > ExtractHL7Attributes[id=bea89fef-86db-1094--81c2e524] >
[jira] [Commented] (NIFI-3973) Create a new Kudu Processor to ingest data
[ https://issues.apache.org/jira/browse/NIFI-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121918#comment-16121918 ] Cam Mach commented on NIFI-3973: [~sanysand...@gmail.com] You're welcome to review the PR. It has been out there for months, but it's active. Thanks > Create a new Kudu Processor to ingest data > -- > > Key: NIFI-3973 > URL: https://issues.apache.org/jira/browse/NIFI-3973 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Cam Mach >Assignee: Cam Quoc Mach > Original Estimate: 336h > Remaining Estimate: 336h > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2020: [NiFi-3973] Add PutKudu Processor for ingesting data to Ku...
Github user cammachusa commented on the issue: https://github.com/apache/nifi/pull/2020 @rickysaltzer , remind :-) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level
[ https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121861#comment-16121861 ] ASF GitHub Bot commented on NIFI-4224: -- Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2051#discussion_r132502384 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java --- @@ -325,6 +441,859 @@ public Response updateProcessGroup( ); } + +@GET +@Consumes(MediaType.WILDCARD) +@Produces(MediaType.APPLICATION_JSON) +@Path("{groupId}/variable-registry/update-requests/{updateId}") +@ApiOperation(value = "Gets a process group's variable registry", response = VariableRegistryUpdateRequestEntity.class, authorizations = { +@Authorization(value = "Read - /process-groups/{uuid}", type = "") +}) +@ApiResponses(value = { +@ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."), +@ApiResponse(code = 401, message = "Client could not be authenticated."), +@ApiResponse(code = 403, message = "Client is not authorized to make this request."), +@ApiResponse(code = 404, message = "The specified resource could not be found."), +@ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.") +}) +public Response getVariableRegistryUpdateRequest( +@ApiParam(value = "The process group id.", required = true) @PathParam("groupId") final String groupId, +@ApiParam(value = "The ID of the Variable Registry Update Request", required = true) @PathParam("updateId") final String updateId) { + +if (groupId == null || updateId == null) { +throw new IllegalArgumentException("Group ID and Update ID must both be specified."); +} + +if (isReplicateRequest()) { +return replicate(HttpMethod.GET); +} + +// authorize access +serviceFacade.authorizeAccess(lookup -> { +final Authorizable processGroup = lookup.getProcessGroup(groupId).getAuthorizable(); +processGroup.authorize(authorizer, RequestAction.READ, NiFiUserUtils.getNiFiUser()); +}); + +final VariableRegistryUpdateRequest request = varRegistryUpdateRequests.get(updateId); +if (request == null) { +throw new ResourceNotFoundException("Could not find a Variable Registry Update Request with identifier " + updateId); +} + +if (!groupId.equals(request.getProcessGroupId())) { +throw new ResourceNotFoundException("Could not find a Variable Registry Update Request with identifier " + updateId + " for Process Group with identifier " + groupId); +} + +final VariableRegistryUpdateRequestEntity entity = new VariableRegistryUpdateRequestEntity(); +entity.setId(request.getRequestId()); + entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request)); +entity.setUri(generateResourceUri("process-groups", groupId, "variable-registry", updateId)); +return generateOkResponse(entity).build(); +} + + +@DELETE +@Consumes(MediaType.WILDCARD) +@Produces(MediaType.APPLICATION_JSON) +@Path("{groupId}/variable-registry/update-requests/{updateId}") +@ApiOperation(value = "Deletes an update request for a process group's variable registry. If the request is not yet complete, it will automatically be cancelled.", +response = VariableRegistryUpdateRequestEntity.class, authorizations = { +@Authorization(value = "Read - /process-groups/{uuid}", type = "") +}) +@ApiResponses(value = { +@ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."), +@ApiResponse(code = 401, message = "Client could not be authenticated."), +@ApiResponse(code = 403, message = "Client is not authorized to make this request."), +@ApiResponse(code = 404, message = "The specified resource could not be found."), +@ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.") +}) +public Response deleteVariableRegistryUpdateRequest( +@ApiParam(value = "The
[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2051#discussion_r132502384 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java --- @@ -325,6 +441,859 @@ public Response updateProcessGroup( ); } + +@GET +@Consumes(MediaType.WILDCARD) +@Produces(MediaType.APPLICATION_JSON) +@Path("{groupId}/variable-registry/update-requests/{updateId}") +@ApiOperation(value = "Gets a process group's variable registry", response = VariableRegistryUpdateRequestEntity.class, authorizations = { +@Authorization(value = "Read - /process-groups/{uuid}", type = "") +}) +@ApiResponses(value = { +@ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."), +@ApiResponse(code = 401, message = "Client could not be authenticated."), +@ApiResponse(code = 403, message = "Client is not authorized to make this request."), +@ApiResponse(code = 404, message = "The specified resource could not be found."), +@ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.") +}) +public Response getVariableRegistryUpdateRequest( +@ApiParam(value = "The process group id.", required = true) @PathParam("groupId") final String groupId, +@ApiParam(value = "The ID of the Variable Registry Update Request", required = true) @PathParam("updateId") final String updateId) { + +if (groupId == null || updateId == null) { +throw new IllegalArgumentException("Group ID and Update ID must both be specified."); +} + +if (isReplicateRequest()) { +return replicate(HttpMethod.GET); +} + +// authorize access +serviceFacade.authorizeAccess(lookup -> { +final Authorizable processGroup = lookup.getProcessGroup(groupId).getAuthorizable(); +processGroup.authorize(authorizer, RequestAction.READ, NiFiUserUtils.getNiFiUser()); +}); + +final VariableRegistryUpdateRequest request = varRegistryUpdateRequests.get(updateId); +if (request == null) { +throw new ResourceNotFoundException("Could not find a Variable Registry Update Request with identifier " + updateId); +} + +if (!groupId.equals(request.getProcessGroupId())) { +throw new ResourceNotFoundException("Could not find a Variable Registry Update Request with identifier " + updateId + " for Process Group with identifier " + groupId); +} + +final VariableRegistryUpdateRequestEntity entity = new VariableRegistryUpdateRequestEntity(); +entity.setId(request.getRequestId()); + entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request)); +entity.setUri(generateResourceUri("process-groups", groupId, "variable-registry", updateId)); +return generateOkResponse(entity).build(); +} + + +@DELETE +@Consumes(MediaType.WILDCARD) +@Produces(MediaType.APPLICATION_JSON) +@Path("{groupId}/variable-registry/update-requests/{updateId}") +@ApiOperation(value = "Deletes an update request for a process group's variable registry. If the request is not yet complete, it will automatically be cancelled.", +response = VariableRegistryUpdateRequestEntity.class, authorizations = { +@Authorization(value = "Read - /process-groups/{uuid}", type = "") +}) +@ApiResponses(value = { +@ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."), +@ApiResponse(code = 401, message = "Client could not be authenticated."), +@ApiResponse(code = 403, message = "Client is not authorized to make this request."), +@ApiResponse(code = 404, message = "The specified resource could not be found."), +@ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.") +}) +public Response deleteVariableRegistryUpdateRequest( +@ApiParam(value = "The process group id.", required = true) @PathParam("groupId") final String groupId, +@ApiParam(value = "The ID of the Variable Registry Update Request", required = true) @PathParam("updateId") final String updateId) { +
[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level
[ https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121762#comment-16121762 ] ASF GitHub Bot commented on NIFI-4224: -- Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2051#discussion_r132477480 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java --- @@ -325,6 +441,859 @@ public Response updateProcessGroup( ); } + +@GET +@Consumes(MediaType.WILDCARD) +@Produces(MediaType.APPLICATION_JSON) +@Path("{groupId}/variable-registry/update-requests/{updateId}") +@ApiOperation(value = "Gets a process group's variable registry", response = VariableRegistryUpdateRequestEntity.class, authorizations = { +@Authorization(value = "Read - /process-groups/{uuid}", type = "") +}) +@ApiResponses(value = { +@ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."), +@ApiResponse(code = 401, message = "Client could not be authenticated."), +@ApiResponse(code = 403, message = "Client is not authorized to make this request."), +@ApiResponse(code = 404, message = "The specified resource could not be found."), +@ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.") +}) +public Response getVariableRegistryUpdateRequest( +@ApiParam(value = "The process group id.", required = true) @PathParam("groupId") final String groupId, +@ApiParam(value = "The ID of the Variable Registry Update Request", required = true) @PathParam("updateId") final String updateId) { + +if (groupId == null || updateId == null) { +throw new IllegalArgumentException("Group ID and Update ID must both be specified."); +} + +if (isReplicateRequest()) { +return replicate(HttpMethod.GET); +} + +// authorize access +serviceFacade.authorizeAccess(lookup -> { +final Authorizable processGroup = lookup.getProcessGroup(groupId).getAuthorizable(); +processGroup.authorize(authorizer, RequestAction.READ, NiFiUserUtils.getNiFiUser()); +}); + +final VariableRegistryUpdateRequest request = varRegistryUpdateRequests.get(updateId); +if (request == null) { +throw new ResourceNotFoundException("Could not find a Variable Registry Update Request with identifier " + updateId); +} + +if (!groupId.equals(request.getProcessGroupId())) { +throw new ResourceNotFoundException("Could not find a Variable Registry Update Request with identifier " + updateId + " for Process Group with identifier " + groupId); +} + +final VariableRegistryUpdateRequestEntity entity = new VariableRegistryUpdateRequestEntity(); +entity.setId(request.getRequestId()); + entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request)); +entity.setUri(generateResourceUri("process-groups", groupId, "variable-registry", updateId)); +return generateOkResponse(entity).build(); +} + + +@DELETE +@Consumes(MediaType.WILDCARD) +@Produces(MediaType.APPLICATION_JSON) +@Path("{groupId}/variable-registry/update-requests/{updateId}") +@ApiOperation(value = "Deletes an update request for a process group's variable registry. If the request is not yet complete, it will automatically be cancelled.", +response = VariableRegistryUpdateRequestEntity.class, authorizations = { +@Authorization(value = "Read - /process-groups/{uuid}", type = "") +}) +@ApiResponses(value = { +@ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."), +@ApiResponse(code = 401, message = "Client could not be authenticated."), +@ApiResponse(code = 403, message = "Client is not authorized to make this request."), +@ApiResponse(code = 404, message = "The specified resource could not be found."), +@ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.") +}) +public Response deleteVariableRegistryUpdateRequest( +@ApiParam(value = "The
[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2051#discussion_r132477480 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java --- @@ -325,6 +441,859 @@ public Response updateProcessGroup( ); } + +@GET +@Consumes(MediaType.WILDCARD) +@Produces(MediaType.APPLICATION_JSON) +@Path("{groupId}/variable-registry/update-requests/{updateId}") +@ApiOperation(value = "Gets a process group's variable registry", response = VariableRegistryUpdateRequestEntity.class, authorizations = { +@Authorization(value = "Read - /process-groups/{uuid}", type = "") +}) +@ApiResponses(value = { +@ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."), +@ApiResponse(code = 401, message = "Client could not be authenticated."), +@ApiResponse(code = 403, message = "Client is not authorized to make this request."), +@ApiResponse(code = 404, message = "The specified resource could not be found."), +@ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.") +}) +public Response getVariableRegistryUpdateRequest( +@ApiParam(value = "The process group id.", required = true) @PathParam("groupId") final String groupId, +@ApiParam(value = "The ID of the Variable Registry Update Request", required = true) @PathParam("updateId") final String updateId) { + +if (groupId == null || updateId == null) { +throw new IllegalArgumentException("Group ID and Update ID must both be specified."); +} + +if (isReplicateRequest()) { +return replicate(HttpMethod.GET); +} + +// authorize access +serviceFacade.authorizeAccess(lookup -> { +final Authorizable processGroup = lookup.getProcessGroup(groupId).getAuthorizable(); +processGroup.authorize(authorizer, RequestAction.READ, NiFiUserUtils.getNiFiUser()); +}); + +final VariableRegistryUpdateRequest request = varRegistryUpdateRequests.get(updateId); +if (request == null) { +throw new ResourceNotFoundException("Could not find a Variable Registry Update Request with identifier " + updateId); +} + +if (!groupId.equals(request.getProcessGroupId())) { +throw new ResourceNotFoundException("Could not find a Variable Registry Update Request with identifier " + updateId + " for Process Group with identifier " + groupId); +} + +final VariableRegistryUpdateRequestEntity entity = new VariableRegistryUpdateRequestEntity(); +entity.setId(request.getRequestId()); + entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request)); +entity.setUri(generateResourceUri("process-groups", groupId, "variable-registry", updateId)); +return generateOkResponse(entity).build(); +} + + +@DELETE +@Consumes(MediaType.WILDCARD) +@Produces(MediaType.APPLICATION_JSON) +@Path("{groupId}/variable-registry/update-requests/{updateId}") +@ApiOperation(value = "Deletes an update request for a process group's variable registry. If the request is not yet complete, it will automatically be cancelled.", +response = VariableRegistryUpdateRequestEntity.class, authorizations = { +@Authorization(value = "Read - /process-groups/{uuid}", type = "") +}) +@ApiResponses(value = { +@ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."), +@ApiResponse(code = 401, message = "Client could not be authenticated."), +@ApiResponse(code = 403, message = "Client is not authorized to make this request."), +@ApiResponse(code = 404, message = "The specified resource could not be found."), +@ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.") +}) +public Response deleteVariableRegistryUpdateRequest( +@ApiParam(value = "The process group id.", required = true) @PathParam("groupId") final String groupId, +@ApiParam(value = "The ID of the Variable Registry Update Request", required = true) @PathParam("updateId") final String updateId) { +
[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2051#discussion_r132476153 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java --- @@ -325,6 +441,859 @@ public Response updateProcessGroup( ); } + +@GET +@Consumes(MediaType.WILDCARD) +@Produces(MediaType.APPLICATION_JSON) +@Path("{groupId}/variable-registry/update-requests/{updateId}") +@ApiOperation(value = "Gets a process group's variable registry", response = VariableRegistryUpdateRequestEntity.class, authorizations = { +@Authorization(value = "Read - /process-groups/{uuid}", type = "") +}) +@ApiResponses(value = { +@ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."), +@ApiResponse(code = 401, message = "Client could not be authenticated."), +@ApiResponse(code = 403, message = "Client is not authorized to make this request."), +@ApiResponse(code = 404, message = "The specified resource could not be found."), +@ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.") +}) +public Response getVariableRegistryUpdateRequest( +@ApiParam(value = "The process group id.", required = true) @PathParam("groupId") final String groupId, +@ApiParam(value = "The ID of the Variable Registry Update Request", required = true) @PathParam("updateId") final String updateId) { + +if (groupId == null || updateId == null) { +throw new IllegalArgumentException("Group ID and Update ID must both be specified."); +} + +if (isReplicateRequest()) { +return replicate(HttpMethod.GET); +} + +// authorize access +serviceFacade.authorizeAccess(lookup -> { +final Authorizable processGroup = lookup.getProcessGroup(groupId).getAuthorizable(); +processGroup.authorize(authorizer, RequestAction.READ, NiFiUserUtils.getNiFiUser()); +}); + +final VariableRegistryUpdateRequest request = varRegistryUpdateRequests.get(updateId); +if (request == null) { +throw new ResourceNotFoundException("Could not find a Variable Registry Update Request with identifier " + updateId); +} + +if (!groupId.equals(request.getProcessGroupId())) { +throw new ResourceNotFoundException("Could not find a Variable Registry Update Request with identifier " + updateId + " for Process Group with identifier " + groupId); +} + +final VariableRegistryUpdateRequestEntity entity = new VariableRegistryUpdateRequestEntity(); +entity.setId(request.getRequestId()); + entity.setRequestDto(dtoFactory.createVariableRegistryUpdateRequestDto(request)); +entity.setUri(generateResourceUri("process-groups", groupId, "variable-registry", updateId)); +return generateOkResponse(entity).build(); +} + + +@DELETE +@Consumes(MediaType.WILDCARD) +@Produces(MediaType.APPLICATION_JSON) +@Path("{groupId}/variable-registry/update-requests/{updateId}") +@ApiOperation(value = "Deletes an update request for a process group's variable registry. If the request is not yet complete, it will automatically be cancelled.", +response = VariableRegistryUpdateRequestEntity.class, authorizations = { +@Authorization(value = "Read - /process-groups/{uuid}", type = "") +}) +@ApiResponses(value = { +@ApiResponse(code = 400, message = "NiFi was unable to complete the request because it was invalid. The request should not be retried without modification."), +@ApiResponse(code = 401, message = "Client could not be authenticated."), +@ApiResponse(code = 403, message = "Client is not authorized to make this request."), +@ApiResponse(code = 404, message = "The specified resource could not be found."), +@ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.") +}) +public Response deleteVariableRegistryUpdateRequest( +@ApiParam(value = "The process group id.", required = true) @PathParam("groupId") final String groupId, +@ApiParam(value = "The ID of the Variable Registry Update Request", required = true) @PathParam("updateId") final String updateId) { +
[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level
[ https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121741#comment-16121741 ] ASF GitHub Bot commented on NIFI-4224: -- Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2051#discussion_r132472005 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java --- @@ -314,7 +430,7 @@ public Response updateProcessGroup( Authorizable authorizable = lookup.getProcessGroup(id).getAuthorizable(); authorizable.authorize(authorizer, RequestAction.WRITE, NiFiUserUtils.getNiFiUser()); }, -null, +() -> serviceFacade.verifyUpdateProcessGroup(requestProcessGroupDTO), --- End diff -- Good call. Originally, I had it updating the variables here but then refactored quite a bit. I think it's best to go ahead and leave in the addition of the verifyUpdateProcessGroup method, but make the verification a NOP. We do this in a few other places, as well, so I feel it's best to leave the 'plumbing' there, since it's already been built. It will make it easier to update later. > Add Variable Registry at Process Group level > > > Key: NIFI-4224 > URL: https://issues.apache.org/jira/browse/NIFI-4224 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne > > Currently, NiFi exposes a variable registry that is configurable by adding > the name of a properties file to nifi.properties and then treating the > referenced properties file as key/value pairs for the variable registry. > This, however, is very limiting, as it provides a global scope for all > variables, and it requires a restart of NiFi in order to pick up any updates > to the file. We should expose a Process Group-level Variable Registry. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2051: NIFI-4224: Initial implementation of Process Group ...
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2051#discussion_r132472005 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java --- @@ -314,7 +430,7 @@ public Response updateProcessGroup( Authorizable authorizable = lookup.getProcessGroup(id).getAuthorizable(); authorizable.authorize(authorizer, RequestAction.WRITE, NiFiUserUtils.getNiFiUser()); }, -null, +() -> serviceFacade.verifyUpdateProcessGroup(requestProcessGroupDTO), --- End diff -- Good call. Originally, I had it updating the variables here but then refactored quite a bit. I think it's best to go ahead and leave in the addition of the verifyUpdateProcessGroup method, but make the verification a NOP. We do this in a few other places, as well, so I feel it's best to leave the 'plumbing' there, since it's already been built. It will make it easier to update later. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4224) Add Variable Registry at Process Group level
[ https://issues.apache.org/jira/browse/NIFI-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121651#comment-16121651 ] ASF GitHub Bot commented on NIFI-4224: -- Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/2051 @mcgilman thanks for the review. Will update and push a new commit. > Add Variable Registry at Process Group level > > > Key: NIFI-4224 > URL: https://issues.apache.org/jira/browse/NIFI-4224 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne > > Currently, NiFi exposes a variable registry that is configurable by adding > the name of a properties file to nifi.properties and then treating the > referenced properties file as key/value pairs for the variable registry. > This, however, is very limiting, as it provides a global scope for all > variables, and it requires a restart of NiFi in order to pick up any updates > to the file. We should expose a Process Group-level Variable Registry. -- This message was sent by Atlassian JIRA (v6.4.14#64029)