[jira] [Updated] (NIFI-4970) EOF Exception in InvokeHttp when body's response is empty
[
https://issues.apache.org/jira/browse/NIFI-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eduardo Mota Fontes updated NIFI-4970:
--
Affects Version/s: 1.11.3
Status: Patch Available (was: Open)
[https://github.com/apache/nifi/pull/4109]
It fixes EOFException when response body is empty and Content-Encoding = gzip.
> EOF Exception in InvokeHttp when body's response is empty
> --
>
> Key: NIFI-4970
> URL: https://issues.apache.org/jira/browse/NIFI-4970
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.11.3, 1.5.0
> Environment: Apache NiFi - Version 1.5.0.3.1.1.0-35
>Reporter: Francois Brillon
>Assignee: Eduardo Mota Fontes
>Priority: Major
> Fix For: 1.12.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> *Description*
> A POST to an API that returns an empty body on success (status code 200) will
> generate an EOF Exception, causing the processor to always remain in error
> and routing all flow files to failures, even if the API call succeeded.
> An example of such API is the Streaming API of PowerBI:
> [https://docs.microsoft.com/en-us/power-bi/service-real-time-streaming]
> *Exception Stack Traces*
> When the +property "Put Response Body In Attribute" is not set+, the
> exception is as follows:
> {code:java}
> rocessor.exception.FlowFileAccessException: Unable to create ContentClaim due
> to java.io.EOFException: org.apache.nifi.pro
> cessor.exception.FlowFileAccessException: Failed to import data from
> buffer(okio.GzipSource@159311b9).inputStream() for St
> andardFlowFileRecord[uuid=05a89e7b-d500-4d48-b034-52c7324fa6e6,claim=,offset=0,name=rtm-vehicle-position-20180313-182039.p
> b,size=0] due to org.apache.nifi.processor.exception.FlowFileAccessException:
> Unable to create ContentClaim due to java.io
> .EOFException
> org.apache.nifi.processor.exception.FlowFileAccessException: Failed to import
> data from buffer(okio.GzipSource@159311b9).i
> nputStream() for
> StandardFlowFileRecord[uuid=05a89e7b-d500-4d48-b034-52c7324fa6e6,claim=,offset=0,name=rtm-vehicle-positio
> n-20180313-182039.pb,size=0] due to
> org.apache.nifi.processor.exception.FlowFileAccessException: Unable to create
> ContentC
> laim due to java.io.EOFException
> at
> org.apache.nifi.controller.repository.StandardProcessSession.importFrom(StandardProcessSession.java:2942)
> at
> org.apache.nifi.processors.standard.InvokeHTTP.onTrigger(InvokeHTTP.java:817)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.nifi.processor.exception.FlowFileAccessException:
> Unable to create ContentClaim due to java.io.EOFException
> at
> org.apache.nifi.controller.repository.StandardProcessSession.importFrom(StandardProcessSession.java:2935)
> ... 13 common frames omitted
> Caused by: java.io.EOFException: null
> at okio.RealBufferedSource.require(RealBufferedSource.java:59)
> at okio.GzipSource.consumeHeader(GzipSource.java:114)
> at okio.GzipSource.read(GzipSource.java:73)
> at okio.RealBufferedSource$1.read(RealBufferedSource.java:409)
> at java.io.InputStream.read(InputStream.java:101)
> at org.apache.nifi.stream.io.StreamUtils.copy(StreamUtils.java:35)
> at
> org.apache.nifi.controller.repository.FileSystemRepository.importFrom(FileSystemRepository.java:734)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.importFrom(StandardProcessSession.java:2932)
> ... 13 common frames omitted
> {code}
> When +"Put Body Response in Attribute" property is set+, the exception is as
> follows :
> {code:java}
> 2018-03-13 18:32:13,165 ERROR [Timer-Driven Process Thread-3]
> o.a.nifi.processors.standard.InvokeHTTP
>
[GitHub] [nifi] eduardofontes opened a new pull request #4109: NIFI-4970 - EOF Exception in InvokeHttp when body's response is empty with gzip
eduardofontes opened a new pull request #4109: NIFI-4970 - EOF Exception in
InvokeHttp when body's response is empty with gzip
URL: https://github.com/apache/nifi/pull/4109
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on both JDK 8 and
JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (NIFI-4970) EOF Exception in InvokeHttp when body's response is empty
[
https://issues.apache.org/jira/browse/NIFI-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eduardo Mota Fontes updated NIFI-4970:
--
Fix Version/s: 1.12.0
> EOF Exception in InvokeHttp when body's response is empty
> --
>
> Key: NIFI-4970
> URL: https://issues.apache.org/jira/browse/NIFI-4970
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.5.0
> Environment: Apache NiFi - Version 1.5.0.3.1.1.0-35
>Reporter: Francois Brillon
>Assignee: Eduardo Mota Fontes
>Priority: Major
> Fix For: 1.12.0
>
>
> *Description*
> A POST to an API that returns an empty body on success (status code 200) will
> generate an EOF Exception, causing the processor to always remain in error
> and routing all flow files to failures, even if the API call succeeded.
> An example of such API is the Streaming API of PowerBI:
> [https://docs.microsoft.com/en-us/power-bi/service-real-time-streaming]
> *Exception Stack Traces*
> When the +property "Put Response Body In Attribute" is not set+, the
> exception is as follows:
> {code:java}
> rocessor.exception.FlowFileAccessException: Unable to create ContentClaim due
> to java.io.EOFException: org.apache.nifi.pro
> cessor.exception.FlowFileAccessException: Failed to import data from
> buffer(okio.GzipSource@159311b9).inputStream() for St
> andardFlowFileRecord[uuid=05a89e7b-d500-4d48-b034-52c7324fa6e6,claim=,offset=0,name=rtm-vehicle-position-20180313-182039.p
> b,size=0] due to org.apache.nifi.processor.exception.FlowFileAccessException:
> Unable to create ContentClaim due to java.io
> .EOFException
> org.apache.nifi.processor.exception.FlowFileAccessException: Failed to import
> data from buffer(okio.GzipSource@159311b9).i
> nputStream() for
> StandardFlowFileRecord[uuid=05a89e7b-d500-4d48-b034-52c7324fa6e6,claim=,offset=0,name=rtm-vehicle-positio
> n-20180313-182039.pb,size=0] due to
> org.apache.nifi.processor.exception.FlowFileAccessException: Unable to create
> ContentC
> laim due to java.io.EOFException
> at
> org.apache.nifi.controller.repository.StandardProcessSession.importFrom(StandardProcessSession.java:2942)
> at
> org.apache.nifi.processors.standard.InvokeHTTP.onTrigger(InvokeHTTP.java:817)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.nifi.processor.exception.FlowFileAccessException:
> Unable to create ContentClaim due to java.io.EOFException
> at
> org.apache.nifi.controller.repository.StandardProcessSession.importFrom(StandardProcessSession.java:2935)
> ... 13 common frames omitted
> Caused by: java.io.EOFException: null
> at okio.RealBufferedSource.require(RealBufferedSource.java:59)
> at okio.GzipSource.consumeHeader(GzipSource.java:114)
> at okio.GzipSource.read(GzipSource.java:73)
> at okio.RealBufferedSource$1.read(RealBufferedSource.java:409)
> at java.io.InputStream.read(InputStream.java:101)
> at org.apache.nifi.stream.io.StreamUtils.copy(StreamUtils.java:35)
> at
> org.apache.nifi.controller.repository.FileSystemRepository.importFrom(FileSystemRepository.java:734)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.importFrom(StandardProcessSession.java:2932)
> ... 13 common frames omitted
> {code}
> When +"Put Body Response in Attribute" property is set+, the exception is as
> follows :
> {code:java}
> 2018-03-13 18:32:13,165 ERROR [Timer-Driven Process Thread-3]
> o.a.nifi.processors.standard.InvokeHTTP
> InvokeHTTP[id=5aae3a34-35d9-1268-6e6d-b774d4467e5a] Routing to Failure due to
> exception: java.io.EOFException: java.io.EOFException
> java.io.EOFException: null
> at okio.RealBufferedSource.require(RealBufferedSource.java:59)
>
[jira] [Assigned] (NIFI-4970) EOF Exception in InvokeHttp when body's response is empty
[
https://issues.apache.org/jira/browse/NIFI-4970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eduardo Mota Fontes reassigned NIFI-4970:
-
Assignee: Eduardo Mota Fontes
> EOF Exception in InvokeHttp when body's response is empty
> --
>
> Key: NIFI-4970
> URL: https://issues.apache.org/jira/browse/NIFI-4970
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.5.0
> Environment: Apache NiFi - Version 1.5.0.3.1.1.0-35
>Reporter: Francois Brillon
>Assignee: Eduardo Mota Fontes
>Priority: Major
>
> *Description*
> A POST to an API that returns an empty body on success (status code 200) will
> generate an EOF Exception, causing the processor to always remain in error
> and routing all flow files to failures, even if the API call succeeded.
> An example of such API is the Streaming API of PowerBI:
> [https://docs.microsoft.com/en-us/power-bi/service-real-time-streaming]
> *Exception Stack Traces*
> When the +property "Put Response Body In Attribute" is not set+, the
> exception is as follows:
> {code:java}
> rocessor.exception.FlowFileAccessException: Unable to create ContentClaim due
> to java.io.EOFException: org.apache.nifi.pro
> cessor.exception.FlowFileAccessException: Failed to import data from
> buffer(okio.GzipSource@159311b9).inputStream() for St
> andardFlowFileRecord[uuid=05a89e7b-d500-4d48-b034-52c7324fa6e6,claim=,offset=0,name=rtm-vehicle-position-20180313-182039.p
> b,size=0] due to org.apache.nifi.processor.exception.FlowFileAccessException:
> Unable to create ContentClaim due to java.io
> .EOFException
> org.apache.nifi.processor.exception.FlowFileAccessException: Failed to import
> data from buffer(okio.GzipSource@159311b9).i
> nputStream() for
> StandardFlowFileRecord[uuid=05a89e7b-d500-4d48-b034-52c7324fa6e6,claim=,offset=0,name=rtm-vehicle-positio
> n-20180313-182039.pb,size=0] due to
> org.apache.nifi.processor.exception.FlowFileAccessException: Unable to create
> ContentC
> laim due to java.io.EOFException
> at
> org.apache.nifi.controller.repository.StandardProcessSession.importFrom(StandardProcessSession.java:2942)
> at
> org.apache.nifi.processors.standard.InvokeHTTP.onTrigger(InvokeHTTP.java:817)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.nifi.processor.exception.FlowFileAccessException:
> Unable to create ContentClaim due to java.io.EOFException
> at
> org.apache.nifi.controller.repository.StandardProcessSession.importFrom(StandardProcessSession.java:2935)
> ... 13 common frames omitted
> Caused by: java.io.EOFException: null
> at okio.RealBufferedSource.require(RealBufferedSource.java:59)
> at okio.GzipSource.consumeHeader(GzipSource.java:114)
> at okio.GzipSource.read(GzipSource.java:73)
> at okio.RealBufferedSource$1.read(RealBufferedSource.java:409)
> at java.io.InputStream.read(InputStream.java:101)
> at org.apache.nifi.stream.io.StreamUtils.copy(StreamUtils.java:35)
> at
> org.apache.nifi.controller.repository.FileSystemRepository.importFrom(FileSystemRepository.java:734)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.importFrom(StandardProcessSession.java:2932)
> ... 13 common frames omitted
> {code}
> When +"Put Body Response in Attribute" property is set+, the exception is as
> follows :
> {code:java}
> 2018-03-13 18:32:13,165 ERROR [Timer-Driven Process Thread-3]
> o.a.nifi.processors.standard.InvokeHTTP
> InvokeHTTP[id=5aae3a34-35d9-1268-6e6d-b774d4467e5a] Routing to Failure due to
> exception: java.io.EOFException: java.io.EOFException
> java.io.EOFException: null
> at okio.RealBufferedSource.require(RealBufferedSource.java:59)
> at
[GitHub] [nifi] alopresto commented on issue #4077: NIFI-5346 Introduces new PGP controller service and PGP processors.
alopresto commented on issue #4077: NIFI-5346 Introduces new PGP controller service and PGP processors. URL: https://github.com/apache/nifi/pull/4077#issuecomment-594281260 I also don't see any changes to the existing `EncryptContent` processor. We should clearly indicate that the PGP operations in the new processors will be faster and easier to use and configure, update the documentation for both, and explain to the users and developers why this decision was made. We don't want to break BC on a minor release, but users should be encouraged to change now so we can drop the PGP operations from the legacy processor in a major release. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (NIFI-7025) Add kerberos password property to NiFi Hive components
[ https://issues.apache.org/jira/browse/NIFI-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Storck updated NIFI-7025: -- Status: Patch Available (was: Reopened) > Add kerberos password property to NiFi Hive components > -- > > Key: NIFI-7025 > URL: https://issues.apache.org/jira/browse/NIFI-7025 > Project: Apache NiFi > Issue Type: Sub-task > Components: Extensions >Reporter: Jeff Storck >Assignee: Jeff Storck >Priority: Major > Fix For: 1.12.0 > > Time Spent: 2h 50m > Remaining Estimate: 0h > > In addition to the principal/keytab and KerberosCredentialsService options > for accessing kerberized services from NiFi Hive components, a password field > should be added. > Components should validate that only one set of options should be configured: > * principal and keytab > * principal and password > * KerberosCredentialsService > The components that will be affected by this change: > * Hive3ConnectionPool > * Hive_1_1ConnectionPool > * HiveConnectionPool > * PutHive3Streaming > * PutHiveStreaming -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] jtstorck opened a new pull request #4108: NIFI-7025: Wrap Hive 3 calls with UGI.doAs
jtstorck opened a new pull request #4108: NIFI-7025: Wrap Hive 3 calls with
UGI.doAs
URL: https://github.com/apache/nifi/pull/4108
Updated PutHive3Streaming to wrap calls to Hive in UGI.doAs methods
Fixed misleading logging message after the principal has been authenticated
with the KDC
When connecting to unsecured Hive 3, a UGI with "simple" auth will be used
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Wrap Hive 3 calls with UGI.doAs._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [x] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [x] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on both JDK 8 and
JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] alopresto commented on a change in pull request #4077: NIFI-5346 Introduces new PGP controller service and PGP processors.
alopresto commented on a change in pull request #4077: NIFI-5346 Introduces new PGP controller service and PGP processors. URL: https://github.com/apache/nifi/pull/4077#discussion_r387369251 ## File path: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/pgp/PGPControllerService.java ## @@ -0,0 +1,893 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.processors.standard.pgp; + +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.context.PropertyContext; +import org.apache.nifi.controller.AbstractControllerService; +import org.apache.nifi.controller.ControllerServiceInitializationContext; +import org.apache.nifi.expression.ExpressionLanguageScope; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.processors.standard.util.PGPUtil; +import org.apache.nifi.reporting.InitializationException; +import org.bouncycastle.bcpg.ArmoredOutputStream; +import org.bouncycastle.jce.provider.BouncyCastleProvider; +import org.bouncycastle.openpgp.PGPCompressedData; +import org.bouncycastle.openpgp.PGPCompressedDataGenerator; +import org.bouncycastle.openpgp.PGPEncryptedData; +import org.bouncycastle.openpgp.PGPEncryptedDataGenerator; +import org.bouncycastle.openpgp.PGPEncryptedDataList; +import org.bouncycastle.openpgp.PGPException; +import org.bouncycastle.openpgp.PGPLiteralData; +import org.bouncycastle.openpgp.PGPLiteralDataGenerator; +import org.bouncycastle.openpgp.PGPObjectFactory; +import org.bouncycastle.openpgp.PGPPBEEncryptedData; +import org.bouncycastle.openpgp.PGPPrivateKey; +import org.bouncycastle.openpgp.PGPPublicKey; +import org.bouncycastle.openpgp.PGPPublicKeyEncryptedData; +import org.bouncycastle.openpgp.PGPPublicKeyRing; +import org.bouncycastle.openpgp.PGPSecretKey; +import org.bouncycastle.openpgp.PGPSecretKeyRing; +import org.bouncycastle.openpgp.PGPSecretKeyRingCollection; +import org.bouncycastle.openpgp.PGPSignature; +import org.bouncycastle.openpgp.PGPSignatureGenerator; +import org.bouncycastle.openpgp.PGPSignatureList; +import org.bouncycastle.openpgp.jcajce.JcaPGPObjectFactory; +import org.bouncycastle.openpgp.jcajce.JcaPGPPublicKeyRingCollection; +import org.bouncycastle.openpgp.operator.KeyFingerPrintCalculator; +import org.bouncycastle.openpgp.operator.PBEDataDecryptorFactory; +import org.bouncycastle.openpgp.operator.PBESecretKeyDecryptor; +import org.bouncycastle.openpgp.operator.PGPDigestCalculatorProvider; +import org.bouncycastle.openpgp.operator.PublicKeyDataDecryptorFactory; +import org.bouncycastle.openpgp.operator.bc.BcKeyFingerprintCalculator; +import org.bouncycastle.openpgp.operator.bc.BcPGPDigestCalculatorProvider; +import org.bouncycastle.openpgp.operator.jcajce.JcaPGPContentSignerBuilder; +import org.bouncycastle.openpgp.operator.jcajce.JcaPGPContentVerifierBuilderProvider; +import org.bouncycastle.openpgp.operator.jcajce.JcaPGPDigestCalculatorProviderBuilder; +import org.bouncycastle.openpgp.operator.jcajce.JcePBEDataDecryptorFactoryBuilder; +import org.bouncycastle.openpgp.operator.jcajce.JcePBEKeyEncryptionMethodGenerator; +import org.bouncycastle.openpgp.operator.jcajce.JcePBESecretKeyDecryptorBuilder; +import org.bouncycastle.openpgp.operator.jcajce.JcePGPDataEncryptorBuilder; +import org.bouncycastle.openpgp.operator.jcajce.JcePublicKeyDataDecryptorFactoryBuilder; +import org.bouncycastle.openpgp.operator.jcajce.JcePublicKeyKeyEncryptionMethodGenerator; +import org.bouncycastle.util.io.Streams; + +import java.io.ByteArrayInputStream; +import java.io.File; +import java.io.FileInputStream; +import java.io.FileNotFoundException; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.nio.charset.StandardCharsets; +import java.security.Security; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.Date; +import
[GitHub] [nifi] alopresto commented on a change in pull request #4077: NIFI-5346 Introduces new PGP controller service and PGP processors.
alopresto commented on a change in pull request #4077: NIFI-5346 Introduces new PGP controller service and PGP processors. URL: https://github.com/apache/nifi/pull/4077#discussion_r387368454 ## File path: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/pgp/PGPControllerService.java ## @@ -0,0 +1,893 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.processors.standard.pgp; + +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.context.PropertyContext; +import org.apache.nifi.controller.AbstractControllerService; +import org.apache.nifi.controller.ControllerServiceInitializationContext; +import org.apache.nifi.expression.ExpressionLanguageScope; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.processors.standard.util.PGPUtil; +import org.apache.nifi.reporting.InitializationException; +import org.bouncycastle.bcpg.ArmoredOutputStream; +import org.bouncycastle.jce.provider.BouncyCastleProvider; +import org.bouncycastle.openpgp.PGPCompressedData; +import org.bouncycastle.openpgp.PGPCompressedDataGenerator; +import org.bouncycastle.openpgp.PGPEncryptedData; +import org.bouncycastle.openpgp.PGPEncryptedDataGenerator; +import org.bouncycastle.openpgp.PGPEncryptedDataList; +import org.bouncycastle.openpgp.PGPException; +import org.bouncycastle.openpgp.PGPLiteralData; +import org.bouncycastle.openpgp.PGPLiteralDataGenerator; +import org.bouncycastle.openpgp.PGPObjectFactory; +import org.bouncycastle.openpgp.PGPPBEEncryptedData; +import org.bouncycastle.openpgp.PGPPrivateKey; +import org.bouncycastle.openpgp.PGPPublicKey; +import org.bouncycastle.openpgp.PGPPublicKeyEncryptedData; +import org.bouncycastle.openpgp.PGPPublicKeyRing; +import org.bouncycastle.openpgp.PGPSecretKey; +import org.bouncycastle.openpgp.PGPSecretKeyRing; +import org.bouncycastle.openpgp.PGPSecretKeyRingCollection; +import org.bouncycastle.openpgp.PGPSignature; +import org.bouncycastle.openpgp.PGPSignatureGenerator; +import org.bouncycastle.openpgp.PGPSignatureList; +import org.bouncycastle.openpgp.jcajce.JcaPGPObjectFactory; +import org.bouncycastle.openpgp.jcajce.JcaPGPPublicKeyRingCollection; +import org.bouncycastle.openpgp.operator.KeyFingerPrintCalculator; +import org.bouncycastle.openpgp.operator.PBEDataDecryptorFactory; +import org.bouncycastle.openpgp.operator.PBESecretKeyDecryptor; +import org.bouncycastle.openpgp.operator.PGPDigestCalculatorProvider; +import org.bouncycastle.openpgp.operator.PublicKeyDataDecryptorFactory; +import org.bouncycastle.openpgp.operator.bc.BcKeyFingerprintCalculator; +import org.bouncycastle.openpgp.operator.bc.BcPGPDigestCalculatorProvider; +import org.bouncycastle.openpgp.operator.jcajce.JcaPGPContentSignerBuilder; +import org.bouncycastle.openpgp.operator.jcajce.JcaPGPContentVerifierBuilderProvider; +import org.bouncycastle.openpgp.operator.jcajce.JcaPGPDigestCalculatorProviderBuilder; +import org.bouncycastle.openpgp.operator.jcajce.JcePBEDataDecryptorFactoryBuilder; +import org.bouncycastle.openpgp.operator.jcajce.JcePBEKeyEncryptionMethodGenerator; +import org.bouncycastle.openpgp.operator.jcajce.JcePBESecretKeyDecryptorBuilder; +import org.bouncycastle.openpgp.operator.jcajce.JcePGPDataEncryptorBuilder; +import org.bouncycastle.openpgp.operator.jcajce.JcePublicKeyDataDecryptorFactoryBuilder; +import org.bouncycastle.openpgp.operator.jcajce.JcePublicKeyKeyEncryptionMethodGenerator; +import org.bouncycastle.util.io.Streams; + +import java.io.ByteArrayInputStream; +import java.io.File; +import java.io.FileInputStream; +import java.io.FileNotFoundException; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.nio.charset.StandardCharsets; +import java.security.Security; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.Date; +import
[jira] [Reopened] (NIFI-7025) Add kerberos password property to NiFi Hive components
[ https://issues.apache.org/jira/browse/NIFI-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Storck reopened NIFI-7025: --- Reopening to address an issue with PutHive3Streaming encountering authentication issues caused by not wrapping calls to Hive with UGI.doAs. > Add kerberos password property to NiFi Hive components > -- > > Key: NIFI-7025 > URL: https://issues.apache.org/jira/browse/NIFI-7025 > Project: Apache NiFi > Issue Type: Sub-task > Components: Extensions >Reporter: Jeff Storck >Assignee: Jeff Storck >Priority: Major > Fix For: 1.12.0 > > Time Spent: 2h 40m > Remaining Estimate: 0h > > In addition to the principal/keytab and KerberosCredentialsService options > for accessing kerberized services from NiFi Hive components, a password field > should be added. > Components should validate that only one set of options should be configured: > * principal and keytab > * principal and password > * KerberosCredentialsService > The components that will be affected by this change: > * Hive3ConnectionPool > * Hive_1_1ConnectionPool > * HiveConnectionPool > * PutHive3Streaming > * PutHiveStreaming -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-7121) Fix Argon2 constructor comment
[
https://issues.apache.org/jira/browse/NIFI-7121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy LoPresto resolved NIFI-7121.
-
Fix Version/s: 1.12.0
Resolution: Fixed
> Fix Argon2 constructor comment
> --
>
> Key: NIFI-7121
> URL: https://issues.apache.org/jira/browse/NIFI-7121
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 1.11.1
>Reporter: Andy LoPresto
>Assignee: M Tien
>Priority: Minor
> Labels: documentation
> Fix For: 1.12.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The comment on line 74 of {{Argon2SecureHasher}} is not correct - the salt is
> not unique, but rather static for this constructor.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7121) Fix Argon2 constructor comment
[
https://issues.apache.org/jira/browse/NIFI-7121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050648#comment-17050648
]
ASF subversion and git services commented on NIFI-7121:
---
Commit 0b2816baa4a44ac8e2128e4730796cc193d8bced in nifi's branch
refs/heads/master from M Tien
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=0b2816b ]
NIFI-7121 Updated comment to state a 'static' salt is used in the constructor.
(#4098)
Signed-off-by: Andy LoPresto
> Fix Argon2 constructor comment
> --
>
> Key: NIFI-7121
> URL: https://issues.apache.org/jira/browse/NIFI-7121
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 1.11.1
>Reporter: Andy LoPresto
>Assignee: M Tien
>Priority: Minor
> Labels: documentation
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The comment on line 74 of {{Argon2SecureHasher}} is not correct - the salt is
> not unique, but rather static for this constructor.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] alopresto merged pull request #4098: NIFI-7121 Updated comment to state a 'static' salt is used in the con…
alopresto merged pull request #4098: NIFI-7121 Updated comment to state a 'static' salt is used in the con… URL: https://github.com/apache/nifi/pull/4098 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (NIFI-7218) Fix typos in documentation
[ https://issues.apache.org/jira/browse/NIFI-7218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto updated NIFI-7218: Fix Version/s: 1.12.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Fix typos in documentation > -- > > Key: NIFI-7218 > URL: https://issues.apache.org/jira/browse/NIFI-7218 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation Website >Affects Versions: 1.11.3 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Trivial > Labels: beginner, documentation > Fix For: 1.12.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > There are outstanding typos in the public documentation that should be fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7218) Fix typos in documentation
[ https://issues.apache.org/jira/browse/NIFI-7218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto updated NIFI-7218: Status: Patch Available (was: In Progress) > Fix typos in documentation > -- > > Key: NIFI-7218 > URL: https://issues.apache.org/jira/browse/NIFI-7218 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation Website >Affects Versions: 1.11.3 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Trivial > Labels: beginner, documentation > Time Spent: 0.5h > Remaining Estimate: 0h > > There are outstanding typos in the public documentation that should be fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7218) Fix typos in documentation
[ https://issues.apache.org/jira/browse/NIFI-7218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050646#comment-17050646 ] ASF subversion and git services commented on NIFI-7218: --- Commit cb08382da4cb888c1d8cdfab078a57c0c2986501 in nifi's branch refs/heads/master from Andy LoPresto [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=cb08382 ] NIFI-7218 Fixed typo in Overview docs. (#4107) > Fix typos in documentation > -- > > Key: NIFI-7218 > URL: https://issues.apache.org/jira/browse/NIFI-7218 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation Website >Affects Versions: 1.11.3 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Trivial > Labels: beginner, documentation > Time Spent: 0.5h > Remaining Estimate: 0h > > There are outstanding typos in the public documentation that should be fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] alopresto merged pull request #4107: NIFI-7218 Fixed typo in Overview docs.
alopresto merged pull request #4107: NIFI-7218 Fixed typo in Overview docs. URL: https://github.com/apache/nifi/pull/4107 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (MINIFICPP-1170) Fix C2 RESTART
[ https://issues.apache.org/jira/browse/MINIFICPP-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Murtuza Shareef updated MINIFICPP-1170: --- Summary: Fix C2 RESTART (was: Fix C2 STOP, START and RESTART) > Fix C2 RESTART > -- > > Key: MINIFICPP-1170 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1170 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Murtuza Shareef >Assignee: Murtuza Shareef >Priority: Major > Labels: minifi-cpp > > C2 RESTART doe not restart the agent. > Define the behavior of the agent in response RESTART and implement it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1170) Fix C2 RESTART
[ https://issues.apache.org/jira/browse/MINIFICPP-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Murtuza Shareef updated MINIFICPP-1170: --- Description: C2 RESTART does not restart the agent. Define the behavior of the agent in response RESTART and implement it. was: C2 RESTART doe not restart the agent. Define the behavior of the agent in response RESTART and implement it. > Fix C2 RESTART > -- > > Key: MINIFICPP-1170 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1170 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Murtuza Shareef >Assignee: Murtuza Shareef >Priority: Major > Labels: minifi-cpp > > C2 RESTART does not restart the agent. > Define the behavior of the agent in response RESTART and implement it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1170) Fix C2 STOP, START and RESTART
[ https://issues.apache.org/jira/browse/MINIFICPP-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Murtuza Shareef updated MINIFICPP-1170: --- Description: C2 RESTART doe not restart the agent. Define the behavior of the agent in response RESTART and implement it. was: C2 STOP seems to terminate the agent. However, C2 START does not start it. Neither does C2 RESTART actually restarts the agent. Define the behavior of the agent in response to these operations and implement it. > Fix C2 STOP, START and RESTART > -- > > Key: MINIFICPP-1170 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1170 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Murtuza Shareef >Assignee: Murtuza Shareef >Priority: Major > Labels: minifi-cpp > > C2 RESTART doe not restart the agent. > Define the behavior of the agent in response RESTART and implement it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1170) Fix C2 RESTART
[ https://issues.apache.org/jira/browse/MINIFICPP-1170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Murtuza Shareef updated MINIFICPP-1170: --- Description: C2 RESTART does not restart the agent. Define the behavior of the agent in response to RESTART and implement it. was: C2 RESTART does not restart the agent. Define the behavior of the agent in response RESTART and implement it. > Fix C2 RESTART > -- > > Key: MINIFICPP-1170 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1170 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Murtuza Shareef >Assignee: Murtuza Shareef >Priority: Major > Labels: minifi-cpp > > C2 RESTART does not restart the agent. > Define the behavior of the agent in response to RESTART and implement it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] m-hogue commented on issue #4107: NIFI-7218 Fixed typo in Overview docs.
m-hogue commented on issue #4107: NIFI-7218 Fixed typo in Overview docs. URL: https://github.com/apache/nifi/pull/4107#issuecomment-594139951 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] alopresto opened a new pull request #4107: NIFI-7218 Fixed typo in Overview docs.
alopresto opened a new pull request #4107: NIFI-7218 Fixed typo in Overview
docs.
URL: https://github.com/apache/nifi/pull/4107
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Fixed typo in opening line of Overview doc._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [x] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on both JDK 8 and
JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [x] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] m-hogue commented on issue #4105: NIFI-5925: Added controller services to set of components that are searched
m-hogue commented on issue #4105: NIFI-5925: Added controller services to set of components that are searched URL: https://github.com/apache/nifi/pull/4105#issuecomment-594136422 CI workflow failed for an unrelated reason. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (NIFI-7218) Fix typos in documentation
Andy LoPresto created NIFI-7218: --- Summary: Fix typos in documentation Key: NIFI-7218 URL: https://issues.apache.org/jira/browse/NIFI-7218 Project: Apache NiFi Issue Type: Improvement Components: Documentation Website Affects Versions: 1.11.3 Reporter: Andy LoPresto Assignee: Andy LoPresto There are outstanding typos in the public documentation that should be fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] msharee9 commented on a change in pull request #743: Minificpp 1169 - Simplify C2 metrics collection and reporting
msharee9 commented on a change in pull request #743: Minificpp 1169 - Simplify
C2 metrics collection and reporting
URL: https://github.com/apache/nifi-minifi-cpp/pull/743#discussion_r387232949
##
File path: conf/minifi.properties
##
@@ -52,6 +52,8 @@
nifi.database.content.repository.directory.default=${MINIFI_HOME}/content_reposi
#nifi.c2.rest.url=
#nifi.c2.rest.url.ack=
nifi.c2.root.classes=DeviceInfoNode,AgentInformation,FlowInformation
+## Minimize heartbeat payload size by excluding agent manifest from the
heartbeat
Review comment:
This will not be true anymore
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi-minifi-cpp] msharee9 commented on a change in pull request #743: Minificpp 1169 - Simplify C2 metrics collection and reporting
msharee9 commented on a change in pull request #743: Minificpp 1169 - Simplify C2 metrics collection and reporting URL: https://github.com/apache/nifi-minifi-cpp/pull/743#discussion_r387232703 ## File path: C2.md ## @@ -49,6 +49,11 @@ Release 0.6.0: Please note that all c2 properties now exist as `nifi.c2.*`. If y files contain the former naming convention of `c2.*`, we will continue to support that as an alternate key, but you are encouraged to switch your configuration options as soon as possible. Review comment: This will not be true anymore. Will take care of this This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] msharee9 opened a new pull request #743: Minificpp 1169
msharee9 opened a new pull request #743: Minificpp 1169 URL: https://github.com/apache/nifi-minifi-cpp/pull/743 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (NIFI-7217) Within the DistributedMapCacheServer, it would be beneficial to more easily purge the entire entry list and specific entries with a particular Cache Entry Identifier
Henry Sowell created NIFI-7217: -- Summary: Within the DistributedMapCacheServer, it would be beneficial to more easily purge the entire entry list and specific entries with a particular Cache Entry Identifier Key: NIFI-7217 URL: https://issues.apache.org/jira/browse/NIFI-7217 Project: Apache NiFi Issue Type: Improvement Reporter: Henry Sowell Currently, I use the DetectDuplicate processor, leveraging the DistributedMapCacheService and DistributedMapCacheServer. I have a flow where we need to periodically reprocess data and need to purge what's stored in the DistributedMapCacheServer. Because our DMCserver was configured to persist cache entries to disk, we are required to manually stop the server, and delete the cache on disk, then restart the server to purge entries. It would be beneficial to be able to more easily purge the entire entry list and specific entries with a specific Cache Entry Identifier. This could be accomplished in a variety of ways (e.g., a button from the controller services menu, within Detect Duplicate processor, etc.). Similar requests have come up from others in the community: https://issues.apache.org/jira/browse/NIFI-4173 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] alopresto commented on issue #4099: NIFI-7170: Add option to disable anonymous authentication
alopresto commented on issue #4099: NIFI-7170: Add option to disable anonymous authentication URL: https://github.com/apache/nifi/pull/4099#issuecomment-594110779 Is there any code that prevents a valid user identity from being the string ``? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] alopresto commented on a change in pull request #4099: NIFI-7170: Add option to disable anonymous authentication
alopresto commented on a change in pull request #4099: NIFI-7170: Add option to
disable anonymous authentication
URL: https://github.com/apache/nifi/pull/4099#discussion_r387222503
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/src/test/java/org/apache/nifi/web/security/anonymous/NiFiAnonymousAuthenticationProviderTest.java
##
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.security.anonymous;
+
+import org.apache.nifi.authorization.Authorizer;
+import org.apache.nifi.authorization.user.NiFiUserDetails;
+import org.apache.nifi.util.NiFiProperties;
+import org.apache.nifi.util.StringUtils;
+import org.apache.nifi.web.security.InvalidAuthenticationException;
+import org.apache.nifi.web.security.token.NiFiAuthenticationToken;
+import org.junit.Test;
+import org.mockito.Mockito;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import static org.junit.Assert.assertTrue;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+public class NiFiAnonymousAuthenticationProviderTest {
+
+private static final Logger logger =
LoggerFactory.getLogger(NiFiAnonymousAuthenticationProviderTest.class);
+
+@Test
+public void testAnonymousDisabledNotSecure() throws Exception {
Review comment:
I'm confused as to what scenario this is emulating. To me, it appears to be
a NiFi instance which does not have HTTPS enabled, and has anonymous
authentication explicitly disabled. Why does this return an authentication
token for the anonymous user?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] alopresto commented on a change in pull request #4099: NIFI-7170: Add option to disable anonymous authentication
alopresto commented on a change in pull request #4099: NIFI-7170: Add option to
disable anonymous authentication
URL: https://github.com/apache/nifi/pull/4099#discussion_r387221126
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/src/main/java/org/apache/nifi/web/security/anonymous/NiFiAnonymousAuthenticationRequestToken.java
##
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.security.anonymous;
+
+import org.apache.nifi.web.security.NiFiAuthenticationRequestToken;
+
+import static
org.apache.nifi.authorization.user.StandardNiFiUser.ANONYMOUS_IDENTITY;
+
+/**
+ * This is an authentication request for an anonymous user.
+ */
+public class NiFiAnonymousAuthenticationRequestToken extends
NiFiAuthenticationRequestToken {
+
+final boolean secureRequest;
Review comment:
Not sure I understand the purpose of this boolean - aren't all requests
which cause a token to be generated secure?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] alopresto commented on a change in pull request #4099: NIFI-7170: Add option to disable anonymous authentication
alopresto commented on a change in pull request #4099: NIFI-7170: Add option to
disable anonymous authentication
URL: https://github.com/apache/nifi/pull/4099#discussion_r387220392
##
File path:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/src/main/java/org/apache/nifi/web/security/anonymous/NiFiAnonymousAuthenticationProvider.java
##
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.security.anonymous;
+
+import org.apache.nifi.authorization.Authorizer;
+import org.apache.nifi.authorization.user.NiFiUserDetails;
+import org.apache.nifi.authorization.user.StandardNiFiUser;
+import org.apache.nifi.util.NiFiProperties;
+import org.apache.nifi.web.security.InvalidAuthenticationException;
+import org.apache.nifi.web.security.NiFiAuthenticationProvider;
+import org.apache.nifi.web.security.token.NiFiAuthenticationToken;
+import org.springframework.security.core.Authentication;
+import org.springframework.security.core.AuthenticationException;
+
+/**
+ *
+ */
+public class NiFiAnonymousAuthenticationProvider extends
NiFiAuthenticationProvider {
+
+final NiFiProperties properties;
+
+public NiFiAnonymousAuthenticationProvider(NiFiProperties nifiProperties,
Authorizer authorizer) {
+super(nifiProperties, authorizer);
+this.properties = nifiProperties;
+}
+
+@Override
+public Authentication authenticate(Authentication authentication) throws
AuthenticationException {
+final NiFiAnonymousAuthenticationRequestToken request =
(NiFiAnonymousAuthenticationRequestToken) authentication;
+
+if (request.isSecureRequest() &&
!properties.isAnonymousAuthenticationAllowed()) {
+throw new InvalidAuthenticationException("Anonymous authentication
is not been configured.");
Review comment:
Typo: _is_ -> _has_.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] alopresto commented on a change in pull request #4099: NIFI-7170: Add option to disable anonymous authentication
alopresto commented on a change in pull request #4099: NIFI-7170: Add option to disable anonymous authentication URL: https://github.com/apache/nifi/pull/4099#discussion_r387198406 ## File path: nifi-docs/src/main/asciidoc/administration-guide.adoc ## @@ -3269,6 +3271,7 @@ These properties pertain to various security features in NiFi. Many of these pro |`nifi.security.truststoreType`|The truststore type. It is blank by default. |`nifi.security.truststorePasswd`|The truststore password. It is blank by default. |`nifi.security.user.authorizer`|Specifies which of the configured Authorizers in the _authorizers.xml_ file to use. By default, it is set to `file-provider`. +|`nifi.security.allow.anonymous.authentication`|Whether anonymous authentication is allowed when running over HTTPS. If set to true, this setting will also ensure that one way SSL is enabled. Review comment: I would suggest changing the second sentence here to "If set to true, client certificates are not required to connect via TLS." This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] alopresto commented on a change in pull request #4099: NIFI-7170: Add option to disable anonymous authentication
alopresto commented on a change in pull request #4099: NIFI-7170: Add option to
disable anonymous authentication
URL: https://github.com/apache/nifi/pull/4099#discussion_r387197313
##
File path:
nifi-commons/nifi-properties/src/main/java/org/apache/nifi/util/NiFiProperties.java
##
@@ -904,10 +905,19 @@ public boolean isLoginIdentityProviderEnabled() {
return
!StringUtils.isBlank(getProperty(NiFiProperties.SECURITY_USER_LOGIN_IDENTITY_PROVIDER));
}
+/**
+ * @return True if property value is 'true'; False otherwise.
+ */
+public Boolean isAnonymousAuthenticationAllowed() {
+final String anonymousAuthenticationAllowed =
getProperty(SECURITY_ANONYMOUS_AUTHENTICATION, "false");
+
+return "true".equalsIgnoreCase(anonymousAuthenticationAllowed);
Review comment:
I think we should trim the retrieved property before comparing. Users
frequently have trailing spaces in their `nifi.properties` files, which can
cause string inequality when to the human eye it looks valid, and this is
difficult to diagnose.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (NIFI-6790) Load-balanced queues cause huge number of open pipes
[ https://issues.apache.org/jira/browse/NIFI-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050363#comment-17050363 ] Harald Dobbernack commented on NIFI-6790: - Thank you Shawn for the good news!! I found the Jira Number https://issues.apache.org/jira/browse/NIFI-7114 > Load-balanced queues cause huge number of open pipes > > > Key: NIFI-6790 > URL: https://issues.apache.org/jira/browse/NIFI-6790 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.9.2 > Environment: Ubuntu,OpenJDK8 >Reporter: Matthew Knight >Priority: Critical > Labels: Cluster, LoadBalancing > Attachments: flow_exception.png, lsof_loadbalanced.png, > lsof_notloadbalanced.png > > Original Estimate: 672h > Remaining Estimate: 672h > > Even very basic flows using load balance connecting queues result in huge > numbers of open pipes which don't seem to get closed and result in the entire > cluster being brought to a standstill because of "too many open files". I > tried apples-to-apples with equivalent very basic flows, one load balanced > and one not, to show how easy this has been to reproduce. I'm attaching some > screenshots that show the "too many open files" issue in action in my NiFi > cluster, along with the output of LSOF for the primary node of a cluster with > vs. without load balancing. When load balancing is enabled there is a huge > number of 'pipe' and 'eventpoll' items, without load balancing things are a > bit more balanced. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6790) Load-balanced queues cause huge number of open pipes
[ https://issues.apache.org/jira/browse/NIFI-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050353#comment-17050353 ] Shawn Weeks commented on NIFI-6790: --- This was fixed in 1.11.3. I don’t have the other jira issue number handy though. > Load-balanced queues cause huge number of open pipes > > > Key: NIFI-6790 > URL: https://issues.apache.org/jira/browse/NIFI-6790 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.9.2 > Environment: Ubuntu,OpenJDK8 >Reporter: Matthew Knight >Priority: Critical > Labels: Cluster, LoadBalancing > Attachments: flow_exception.png, lsof_loadbalanced.png, > lsof_notloadbalanced.png > > Original Estimate: 672h > Remaining Estimate: 672h > > Even very basic flows using load balance connecting queues result in huge > numbers of open pipes which don't seem to get closed and result in the entire > cluster being brought to a standstill because of "too many open files". I > tried apples-to-apples with equivalent very basic flows, one load balanced > and one not, to show how easy this has been to reproduce. I'm attaching some > screenshots that show the "too many open files" issue in action in my NiFi > cluster, along with the output of LSOF for the primary node of a cluster with > vs. without load balancing. When load balancing is enabled there is a huge > number of 'pipe' and 'eventpoll' items, without load balancing things are a > bit more balanced. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-registry] ekovacs commented on issue #263: NIFIREG-366: updating pom parent version
ekovacs commented on issue #263: NIFIREG-366: updating pom parent version URL: https://github.com/apache/nifi-registry/pull/263#issuecomment-594021138 @bbende, could you please take a look? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-registry] ekovacs opened a new pull request #263: NIFIREG-366: updating pom parent version
ekovacs opened a new pull request #263: NIFIREG-366: updating pom parent version URL: https://github.com/apache/nifi-registry/pull/263 updating parent pom version to same as NiFi uses. associated ticket: https://issues.apache.org/jira/browse/NIFIREG-366 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] pvillard31 commented on issue #4106: NIFI-5644 Fixed typo in getWrappedQuery method of AbstractDatabaseFet…
pvillard31 commented on issue #4106: NIFI-5644 Fixed typo in getWrappedQuery method of AbstractDatabaseFet… URL: https://github.com/apache/nifi/pull/4106#issuecomment-594017927 Merged, thanks @karthik-kadajji This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (NIFI-5644) Fix typo in AbstractDatabaseFetchProcessor.java
[ https://issues.apache.org/jira/browse/NIFI-5644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-5644. -- Fix Version/s: 1.12.0 Resolution: Fixed > Fix typo in AbstractDatabaseFetchProcessor.java > --- > > Key: NIFI-5644 > URL: https://issues.apache.org/jira/browse/NIFI-5644 > Project: Apache NiFi > Issue Type: Task > Components: Extensions >Reporter: Diego Queiroz >Assignee: karthik kadajji >Priority: Trivial > Fix For: 1.12.0 > > Time Spent: 20m > Remaining Estimate: 0h > > dbAdaper -> dbAdapter -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-5644) Fix typo in AbstractDatabaseFetchProcessor.java
[ https://issues.apache.org/jira/browse/NIFI-5644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050324#comment-17050324 ] ASF subversion and git services commented on NIFI-5644: --- Commit 2cf4fde6868aa7148148fa45371c255b75579faf in nifi's branch refs/heads/master from karthik.kadajji [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=2cf4fde ] NIFI-5644 Fixed typo in getWrappedQuery method of AbstractDatabaseFetchProcessor class Signed-off-by: Pierre Villard This closes #4106. > Fix typo in AbstractDatabaseFetchProcessor.java > --- > > Key: NIFI-5644 > URL: https://issues.apache.org/jira/browse/NIFI-5644 > Project: Apache NiFi > Issue Type: Task > Components: Extensions >Reporter: Diego Queiroz >Assignee: karthik kadajji >Priority: Trivial > Time Spent: 10m > Remaining Estimate: 0h > > dbAdaper -> dbAdapter -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4106: NIFI-5644 Fixed typo in getWrappedQuery method of AbstractDatabaseFet…
asfgit closed pull request #4106: NIFI-5644 Fixed typo in getWrappedQuery method of AbstractDatabaseFet… URL: https://github.com/apache/nifi/pull/4106 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6790) Load-balanced queues cause huge number of open pipes
[ https://issues.apache.org/jira/browse/NIFI-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050322#comment-17050322 ] Harald Dobbernack commented on NIFI-6790: - We (NiFi Beginners) have been experiencing 'Too many open files' Problems with an one node development system (NiFi 1.11.1) , but many Processors and some Wait Processors relying on nofication via DistributedCache fed by Notify Processor which get's fed by listfile Processors checking every 30sec with yield and penalty set to 30sec. Even if the only active flows are in the wait connection of the wait processor and nothing else is happening then the open pipes will accumalate over time. In our use case the waiting flows can easily be waiting a few days. From Friday Evening (starting with about 4000 open Files for the nifi user) till Monday Morning with only three active flows we managed to surpass 5 open files. We will now as a workaround restart the service regularly, but I am hoping a solution can be found. As a side note: we had to put 'export FD_MAX=5' in the nifi-env.sh, as the settings described in https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html under Maximum File Handles in _/etc/security/limits.conf_ did not suffice to be able to set the max open file limit of nifi over 4096 > Load-balanced queues cause huge number of open pipes > > > Key: NIFI-6790 > URL: https://issues.apache.org/jira/browse/NIFI-6790 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.9.2 > Environment: Ubuntu,OpenJDK8 >Reporter: Matthew Knight >Priority: Critical > Labels: Cluster, LoadBalancing > Attachments: flow_exception.png, lsof_loadbalanced.png, > lsof_notloadbalanced.png > > Original Estimate: 672h > Remaining Estimate: 672h > > Even very basic flows using load balance connecting queues result in huge > numbers of open pipes which don't seem to get closed and result in the entire > cluster being brought to a standstill because of "too many open files". I > tried apples-to-apples with equivalent very basic flows, one load balanced > and one not, to show how easy this has been to reproduce. I'm attaching some > screenshots that show the "too many open files" issue in action in my NiFi > cluster, along with the output of LSOF for the primary node of a cluster with > vs. without load balancing. When load balancing is enabled there is a huge > number of 'pipe' and 'eventpoll' items, without load balancing things are a > bit more balanced. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] m-hogue commented on a change in pull request #4105: NIFI-5925: Added controller services to set of components that are searched
m-hogue commented on a change in pull request #4105: NIFI-5925: Added controller services to set of components that are searched URL: https://github.com/apache/nifi/pull/4105#discussion_r386706840 ## File path: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerSearchService.java ## @@ -175,13 +176,67 @@ public void search(final SearchResultsDTO results, final String search, final Pr } } Review comment: Only question i have for the reviewer is if the `getControllerServices()` call here should be recursive. I elected to not make it recursive since we're recursing through `ProcessGroups` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-3303) escapeJson in ReplaceText
[
https://issues.apache.org/jira/browse/NIFI-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050293#comment-17050293
]
Robin Lutz commented on NIFI-3303:
--
This Bug is still present in 1.11.2 .
I attached a small example how to work around it. [^ReplaceText_Bug.xml]
Besically what you need to do is escape the Text twice:
${'$1':escapeJson():escapeJson()} and the processor will unescape it once back.
The issue might be related to this issue:
https://issues.apache.org/jira/browse/NIFI-3837 . It helped me come up with a
work around.
> escapeJson in ReplaceText
> -
>
> Key: NIFI-3303
> URL: https://issues.apache.org/jira/browse/NIFI-3303
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.1.1
>Reporter: tianzk
>Priority: Major
> Attachments: ReplaceText_Bug.xml, config.png, dataflow.png
>
>
> I have some problems while using excapeJson and unescapeJson in ReplaceText
> processor.
> When I give a string: He didn’t say, “Stop”! to ReplaceText as input,and
> configure ReplaceText like: attachment config.png
> The output of ReplaceText is same with the input: He didn’t say, “Stop!”
> ,nothing changed.
> As described in NiFI Documentation the output should be: He didn’t say,
> \"Stop!\”.Did I miss something?
> Also there are problems with unescapeJson.If input is: He didn’t say,
> \”Sto\\\"p!\”,the return string will be: He didn’t say, ”Sto"p!”.
> My dataflow:(GetFile just read a file with a string as content.)
> dataflow.png
> Thanks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-3303) escapeJson in ReplaceText
[ https://issues.apache.org/jira/browse/NIFI-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robin Lutz updated NIFI-3303: - Attachment: ReplaceText_Bug.xml > escapeJson in ReplaceText > - > > Key: NIFI-3303 > URL: https://issues.apache.org/jira/browse/NIFI-3303 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.1.1 >Reporter: tianzk >Priority: Major > Attachments: ReplaceText_Bug.xml, config.png, dataflow.png > > > I have some problems while using excapeJson and unescapeJson in ReplaceText > processor. > When I give a string: He didn’t say, “Stop”! to ReplaceText as input,and > configure ReplaceText like: attachment config.png > The output of ReplaceText is same with the input: He didn’t say, “Stop!” > ,nothing changed. > As described in NiFI Documentation the output should be: He didn’t say, > \"Stop!\”.Did I miss something? > Also there are problems with unescapeJson.If input is: He didn’t say, > \”Sto\\\"p!\”,the return string will be: He didn’t say, ”Sto"p!”. > My dataflow:(GetFile just read a file with a string as content.) > dataflow.png > Thanks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7216) FetchSFTP can't delete or move files upon completion
Matt Rodriguez created NIFI-7216:
Summary: FetchSFTP can't delete or move files upon completion
Key: NIFI-7216
URL: https://issues.apache.org/jira/browse/NIFI-7216
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 1.10.0
Reporter: Matt Rodriguez
Possibly similar to NIFI-7177 as they're not able to delete files with the
GetSFTP processor, but they're getting a different error than I am.
I'm using the FetchSFTP processor to get data from a third party SFTP server, I
don't have a lot of details on version or configuration on their end. I have
noticed that neither the "Move File" or "Delete File" Completion Strategy
options work.
When using "Delete File" as the Completion Strategy I get no alert/bulletin
from NiFi to show that anything went wrong, however, the original file has not
been deleted. I can manually delete the file using an SFTP client with the same
username/password that NiFi connects as.
When using "Move File" as the Completion Strategy I get this warning message
when NiFi tries to move the file after completion:
{code:java}
2020-03-03 13:59:54,375 WARN [Timer-Driven Process Thread-2]
o.a.nifi.processors.standard.FetchSFTP
FetchSFTP[id=0344354b-3a49-316e-a571-adcca7b3e70e] Successfully fetched the
content for
StandardFlowFileRecord[uuid=dea58017-8f3f-4991-91a3-1ff8ca167b6c,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1583243711464-8, container=default,
section=8], offset=4625, length=7672936],offset=0,name=TEST.csv,size=7672936]
from ftp1.X.com:22/Test/TEST.csv but failed to rename the remote file due
to java.io.FileNotFoundException: No such file or directory: {code}
As with deletion, I am able manually move (rename) the file using an SFTP
client with the same username/password that NiFi connects as.
However, I will add that I'm only able to delete or move the file PRIOR to NiFi
fetching the file. If I try to do it immediately after NiFi has fetched the
file, I'll get the exact same "no such file or directory" error from the SFTP
server when using my local client. If I wait some arbitrary amount of time
after NiFi has fetched the file, I am then able to delete or move the file.
I've also noticed the below sequence of events:
# NiFi lists the files on the SFTP server
# NiFi fetches the files on the SFTP server
# NiFi attempts to delete or move the files on the SFTP server and fails
# I immediately attempt to delete or move the files on the SFTP server and fail
# I stop the FetchSFTP processor in NiFi
# I immediately attempt to delete or move the files on the SFTP server and it
succeeds
This leads me to believe that there is some sort of locking behavior happening
where the fetch operation keeps the file locked for some arbitrary amount of
time or until the fetch processor is stopped, which is preventing any other
operation from taking place on the file.
Interestingly enough, I was able to get both the "Delete File" and "Move File"
Completion Strategies to work when I spun up a basic SFTP server on my own. It
sounds like there is something specific with how locks are handled with this
third part SFTP server I'm connecting to and how NiFi uses (or re-uses) SFTP
clients/connections.
As mentioned above, I unfortunately don't have a lot of information about this
SFTP server I'm connecting to, the vendor has been very tight-lipped about
their configuration for some reason.
If there's anything else you need me to provide, please let me know.
Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] karthik-kadajji opened a new pull request #4106: NIFI-5644 Fixed typo in getWrappedQuery method of AbstractDatabaseFet…
karthik-kadajji opened a new pull request #4106: NIFI-5644 Fixed typo in
getWrappedQuery method of AbstractDatabaseFet…
URL: https://github.com/apache/nifi/pull/4106
…chProcessor class
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [Yes ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ Yes] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ yes] Has your PR been rebased against the latest commit within the
target branch (typically `master`)?
- [ yes] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [yes ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [No ] Have you written or updated unit tests to verify your changes?
It is a typo change.
- [No ] Have you verified that the full build is successful on both JDK 8
and JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (NIFIREG-366) update NiFi Registry's parent pom version
Endre Kovacs created NIFIREG-366:
Summary: update NiFi Registry's parent pom version
Key: NIFIREG-366
URL: https://issues.apache.org/jira/browse/NIFIREG-366
Project: NiFi Registry
Issue Type: Task
Reporter: Endre Kovacs
Assignee: Endre Kovacs
currently Apache NiFi uses version *23* of its parent pom.
{code}
org.apache
apache
23
{code}
let's update it in NiFi Registry as well!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (NIFIREG-365) upgrade apache rat to latest released version (0.12)
[ https://issues.apache.org/jira/browse/NIFIREG-365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Endre Kovacs closed NIFIREG-365. Resolution: Invalid actually it is not apache-rat which has to be upgraded, but *nifi-registry*'s parent pom version, from which rat, and other plugin's will get their version. > upgrade apache rat to latest released version (0.12) > > > Key: NIFIREG-365 > URL: https://issues.apache.org/jira/browse/NIFIREG-365 > Project: NiFi Registry > Issue Type: Task >Reporter: Endre Kovacs >Assignee: Endre Kovacs >Priority: Trivial > > NiFi itself uses apache rat 0.12 for license header checks. > NiFi registry have not upgraded it yet, and is using 0.11. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7215) ScanHbase : Get row key when set to col-qual-and-val in json format
CHANDAN KUMAR created NIFI-7215: --- Summary: ScanHbase : Get row key when set to col-qual-and-val in json format Key: NIFI-7215 URL: https://issues.apache.org/jira/browse/NIFI-7215 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Affects Versions: 1.11.3 Reporter: CHANDAN KUMAR ScanHBase processor does not return the row key, when configured to use col-qual-and-val JSON Format. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFIREG-365) upgrade apache rat to latest released version (0.12)
Endre Kovacs created NIFIREG-365: Summary: upgrade apache rat to latest released version (0.12) Key: NIFIREG-365 URL: https://issues.apache.org/jira/browse/NIFIREG-365 Project: NiFi Registry Issue Type: Task Reporter: Endre Kovacs Assignee: Endre Kovacs NiFi itself uses apache rat 0.12 for license header checks. NiFi registry have not upgraded it yet, and is using 0.11. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] arpadboda commented on issue #670: MINIFICPP-1074 - TailFileTests was not working on WIN32 due do CLRF
arpadboda commented on issue #670: MINIFICPP-1074 - TailFileTests was not working on WIN32 due do CLRF URL: https://github.com/apache/nifi-minifi-cpp/pull/670#issuecomment-593932890 Closing this a #731 is going to resolve this testcase issue as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #670: MINIFICPP-1074 - TailFileTests was not working on WIN32 due do CLRF
arpadboda closed pull request #670: MINIFICPP-1074 - TailFileTests was not working on WIN32 due do CLRF URL: https://github.com/apache/nifi-minifi-cpp/pull/670 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #696: WIP: MINIFICPP-1103 - Stabilize MergeFileTests
arpadboda closed pull request #696: WIP: MINIFICPP-1103 - Stabilize MergeFileTests URL: https://github.com/apache/nifi-minifi-cpp/pull/696 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #710: MINIFICPP - 1110, 1111 - PublishKafka, OPC processors should config and
arpadboda closed pull request #710: MINIFICPP - 1110, - PublishKafka, OPC processors should config and URL: https://github.com/apache/nifi-minifi-cpp/pull/710 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (NIFI-5644) Fix typo in AbstractDatabaseFetchProcessor.java
[ https://issues.apache.org/jira/browse/NIFI-5644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] karthik kadajji reassigned NIFI-5644: - Assignee: karthik kadajji > Fix typo in AbstractDatabaseFetchProcessor.java > --- > > Key: NIFI-5644 > URL: https://issues.apache.org/jira/browse/NIFI-5644 > Project: Apache NiFi > Issue Type: Task > Components: Extensions >Reporter: Diego Queiroz >Assignee: karthik kadajji >Priority: Trivial > > dbAdaper -> dbAdapter -- This message was sent by Atlassian Jira (v8.3.4#803005)
