[jira] [Updated] (NIFI-5421) Provide support for Kerberized zookeeper in docker instances
[ https://issues.apache.org/jira/browse/NIFI-5421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarthak updated NIFI-5421: -- Description: We should make the Docker image configurable to support all of the common security options available in NiFi > Provide support for Kerberized zookeeper in docker instances > > > Key: NIFI-5421 > URL: https://issues.apache.org/jira/browse/NIFI-5421 > Project: Apache NiFi > Issue Type: Improvement > Components: Docker >Affects Versions: 1.7.0 >Reporter: Sarthak >Priority: Major > > We should make the Docker image configurable to support all of the common > security options available in NiFi -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-5421) Provide support for Kerberized zookeeper in docker instances
Sarthak created NIFI-5421: - Summary: Provide support for Kerberized zookeeper in docker instances Key: NIFI-5421 URL: https://issues.apache.org/jira/browse/NIFI-5421 Project: Apache NiFi Issue Type: Improvement Components: Docker Affects Versions: 1.7.0 Reporter: Sarthak -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-5420) Allow StandardProcessSession to calculate duration for provenance events
[ https://issues.apache.org/jira/browse/NIFI-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-5420: --- Status: Patch Available (was: In Progress) > Allow StandardProcessSession to calculate duration for provenance events > > > Key: NIFI-5420 > URL: https://issues.apache.org/jira/browse/NIFI-5420 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > > Besides some automatic provenance events being emitted by the framework, the > onus is on each processor to register provenance events and to provide a > duration for the amount of time spent on the operation that results in the > provenance event(s). However, supplying the duration is not mandatory, and if > not provided, a value of -1 is stored to indicate that the duration is > unknown. > Unfortunately there are many processors that do not track the operation time > and thus do not store it in the provenance event. However, the time from when > a flow file is fetched or created in a session to the time the provenance > event is recorded is a legitimate measure of the time the flow file spent > "active" in the session, which is the same as the duration of a provenance > event. > This Jira proposes to record the "start" time of each flow file (more > specifically, it's StandardRepositoryRecord), and upon session.commit(), > record the duration of the flow file while active in the session into the > provenance event(s), only if the duration has not already been supplied to > the provenance events. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5420) Allow StandardProcessSession to calculate duration for provenance events
[
https://issues.apache.org/jira/browse/NIFI-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542314#comment-16542314
]
ASF GitHub Bot commented on NIFI-5420:
--
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2886#discussion_r202205608
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/StandardProvenanceReporter.java
---
@@ -226,7 +228,7 @@ public void send(final FlowFile flowFile, final String
transitUri, final String
@Override
public void send(final FlowFile flowFile, final String transitUri,
final boolean force) {
-send(flowFile, transitUri, -1L, true);
+send(flowFile, transitUri, -1L, force);
--- End diff --
This one I fixed while I was in there, the only thing that calls it is
InvokeHttp which passes in true, so it hasn't been a bug, but it could be some
day :P
> Allow StandardProcessSession to calculate duration for provenance events
>
>
> Key: NIFI-5420
> URL: https://issues.apache.org/jira/browse/NIFI-5420
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>Priority: Major
>
> Besides some automatic provenance events being emitted by the framework, the
> onus is on each processor to register provenance events and to provide a
> duration for the amount of time spent on the operation that results in the
> provenance event(s). However, supplying the duration is not mandatory, and if
> not provided, a value of -1 is stored to indicate that the duration is
> unknown.
> Unfortunately there are many processors that do not track the operation time
> and thus do not store it in the provenance event. However, the time from when
> a flow file is fetched or created in a session to the time the provenance
> event is recorded is a legitimate measure of the time the flow file spent
> "active" in the session, which is the same as the duration of a provenance
> event.
> This Jira proposes to record the "start" time of each flow file (more
> specifically, it's StandardRepositoryRecord), and upon session.commit(),
> record the duration of the flow file while active in the session into the
> provenance event(s), only if the duration has not already been supplied to
> the provenance events.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2886: NIFI-5420: Allow StandardProcessSession to calculat...
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2886#discussion_r202205608
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/StandardProvenanceReporter.java
---
@@ -226,7 +228,7 @@ public void send(final FlowFile flowFile, final String
transitUri, final String
@Override
public void send(final FlowFile flowFile, final String transitUri,
final boolean force) {
-send(flowFile, transitUri, -1L, true);
+send(flowFile, transitUri, -1L, force);
--- End diff --
This one I fixed while I was in there, the only thing that calls it is
InvokeHttp which passes in true, so it hasn't been a bug, but it could be some
day :P
---
[jira] [Assigned] (NIFI-5420) Allow StandardProcessSession to calculate duration for provenance events
[ https://issues.apache.org/jira/browse/NIFI-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess reassigned NIFI-5420: -- Assignee: Matt Burgess > Allow StandardProcessSession to calculate duration for provenance events > > > Key: NIFI-5420 > URL: https://issues.apache.org/jira/browse/NIFI-5420 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > > Besides some automatic provenance events being emitted by the framework, the > onus is on each processor to register provenance events and to provide a > duration for the amount of time spent on the operation that results in the > provenance event(s). However, supplying the duration is not mandatory, and if > not provided, a value of -1 is stored to indicate that the duration is > unknown. > Unfortunately there are many processors that do not track the operation time > and thus do not store it in the provenance event. However, the time from when > a flow file is fetched or created in a session to the time the provenance > event is recorded is a legitimate measure of the time the flow file spent > "active" in the session, which is the same as the duration of a provenance > event. > This Jira proposes to record the "start" time of each flow file (more > specifically, it's StandardRepositoryRecord), and upon session.commit(), > record the duration of the flow file while active in the session into the > provenance event(s), only if the duration has not already been supplied to > the provenance events. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5420) Allow StandardProcessSession to calculate duration for provenance events
[ https://issues.apache.org/jira/browse/NIFI-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542212#comment-16542212 ] ASF GitHub Bot commented on NIFI-5420: -- Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/2886 @markap14 Mind taking a look at this one? Please and thanks! > Allow StandardProcessSession to calculate duration for provenance events > > > Key: NIFI-5420 > URL: https://issues.apache.org/jira/browse/NIFI-5420 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > > Besides some automatic provenance events being emitted by the framework, the > onus is on each processor to register provenance events and to provide a > duration for the amount of time spent on the operation that results in the > provenance event(s). However, supplying the duration is not mandatory, and if > not provided, a value of -1 is stored to indicate that the duration is > unknown. > Unfortunately there are many processors that do not track the operation time > and thus do not store it in the provenance event. However, the time from when > a flow file is fetched or created in a session to the time the provenance > event is recorded is a legitimate measure of the time the flow file spent > "active" in the session, which is the same as the duration of a provenance > event. > This Jira proposes to record the "start" time of each flow file (more > specifically, it's StandardRepositoryRecord), and upon session.commit(), > record the duration of the flow file while active in the session into the > provenance event(s), only if the duration has not already been supplied to > the provenance events. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2886: NIFI-5420: Allow StandardProcessSession to calculate durat...
Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/2886 @markap14 Mind taking a look at this one? Please and thanks! ---
[jira] [Commented] (NIFI-5420) Allow StandardProcessSession to calculate duration for provenance events
[ https://issues.apache.org/jira/browse/NIFI-5420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542211#comment-16542211 ] ASF GitHub Bot commented on NIFI-5420: -- GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/2886 NIFI-5420: Allow StandardProcessSession to calculate duration for provenance events Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-5420 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2886.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2886 commit c9aa7caae52eda571a36cabbbf6829bc15583f3b Author: Matthew Burgess Date: 2018-07-12T21:13:32Z NIFI-5420: Allow StandardProcessSession to calculate duration for provenance events > Allow StandardProcessSession to calculate duration for provenance events > > > Key: NIFI-5420 > URL: https://issues.apache.org/jira/browse/NIFI-5420 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > > Besides some automatic provenance events being emitted by the framework, the > onus is on each processor to register provenance events and to provide a > duration for the amount of time spent on the operation that results in the > provenance event(s). However, supplying the duration is not mandatory, and if > not provided, a value of -1 is stored to indicate that the duration is > unknown. > Unfortunately there are many processors that do not track the operation time > and thus do not store it in the provenance event. However, the time from when > a flow file is fetched or created in a session to the time the provenance > event is recorded is a legitimate measure of the time the flow file spent > "active" in the session, which is the same as the duration of a provenance > event. > This Jira proposes to record the "start" time of each flow file (more > specifically, it's StandardRepositoryRecord), and upon session.commit(), > record the duration of the flow file while active in the session into the > provenance event(s), only if the duration has not already been supplied to > the provenance events. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2886: NIFI-5420: Allow StandardProcessSession to calculat...
GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/2886 NIFI-5420: Allow StandardProcessSession to calculate duration for provenance events Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-5420 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2886.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2886 commit c9aa7caae52eda571a36cabbbf6829bc15583f3b Author: Matthew Burgess Date: 2018-07-12T21:13:32Z NIFI-5420: Allow StandardProcessSession to calculate duration for provenance events ---
[jira] [Created] (NIFI-5420) Allow StandardProcessSession to calculate duration for provenance events
Matt Burgess created NIFI-5420: -- Summary: Allow StandardProcessSession to calculate duration for provenance events Key: NIFI-5420 URL: https://issues.apache.org/jira/browse/NIFI-5420 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Matt Burgess Besides some automatic provenance events being emitted by the framework, the onus is on each processor to register provenance events and to provide a duration for the amount of time spent on the operation that results in the provenance event(s). However, supplying the duration is not mandatory, and if not provided, a value of -1 is stored to indicate that the duration is unknown. Unfortunately there are many processors that do not track the operation time and thus do not store it in the provenance event. However, the time from when a flow file is fetched or created in a session to the time the provenance event is recorded is a legitimate measure of the time the flow file spent "active" in the session, which is the same as the duration of a provenance event. This Jira proposes to record the "start" time of each flow file (more specifically, it's StandardRepositoryRecord), and upon session.commit(), record the duration of the flow file while active in the session into the provenance event(s), only if the duration has not already been supplied to the provenance events. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4194) NullPointerException in InvokeHTTP processor when trusted hostname property is used
[
https://issues.apache.org/jira/browse/NIFI-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nathan Gough updated NIFI-4194:
---
Attachment: Screen Shot 2018-07-12 at 4.39.27 PM.png
> NullPointerException in InvokeHTTP processor when trusted hostname property
> is used
> ---
>
> Key: NIFI-4194
> URL: https://issues.apache.org/jira/browse/NIFI-4194
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Assignee: Nathan Gough
>Priority: Major
> Labels: hostname, okhttp, security, tls
> Attachments: Screen Shot 2018-07-12 at 3.29.50 PM.png, Screen Shot
> 2018-07-12 at 4.39.27 PM.png, flow_20170717-192339_trusted_hostname.xml
>
>
> When a user configures the {{InvokeHTTP}} processor with HTTPS (using an
> {{SSLContextService}}) and populates the {{trustedHostname}} property, the
> processor will throw a {{NullPointerException}} because the OkHttp client
> does not have a valid {{HostnameVerifier}} configured when the
> {{@OnScheduled}} method is called and that verifier is delegated to the
> processor. This results in the stacktrace below:
> {code}
> 2017-07-17 19:15:49,341 ERROR [Timer-Driven Process Thread-6]
> o.a.nifi.processors.standard.InvokeHTTP
> InvokeHTTP[id=53784003-015d-1000-ffb0-e07173729c9c] Routing to Failure due to
> exception: java.lang.NullPointerException: java.lang.NullPointerException
> java.lang.NullPointerException: null
> at
> org.apache.nifi.processors.standard.InvokeHTTP$OverrideHostnameVerifier.verify(InvokeHTTP.java:1050)
> at
> com.squareup.okhttp.internal.io.RealConnection.connectTls(RealConnection.java:192)
> at
> com.squareup.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:145)
> at
> com.squareup.okhttp.internal.io.RealConnection.connect(RealConnection.java:108)
> at
> com.squareup.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:184)
> at
> com.squareup.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:126)
> at
> com.squareup.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:95)
> at
> com.squareup.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:283)
> at
> com.squareup.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:224)
> at com.squareup.okhttp.Call.getResponse(Call.java:286)
> at
> com.squareup.okhttp.Call$ApplicationInterceptorChain.proceed(Call.java:243)
> at
> com.squareup.okhttp.Call.getResponseWithInterceptorChain(Call.java:205)
> at com.squareup.okhttp.Call.execute(Call.java:80)
> at
> org.apache.nifi.processors.standard.InvokeHTTP.onTrigger(InvokeHTTP.java:630)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> When that method is invoked, the existence of the {{HostnameVerifier}} should
> be checked before it is provided to the {{OverrideHostnameVerifier}}, and a
> default value of {{OkHostnameVerifier.INSTANCE}} (public static singleton)
> should be provided if none is available.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-4194) NullPointerException in InvokeHTTP processor when trusted hostname property is used
[
https://issues.apache.org/jira/browse/NIFI-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nathan Gough updated NIFI-4194:
---
Attachment: Screen Shot 2018-07-12 at 3.29.50 PM.png
> NullPointerException in InvokeHTTP processor when trusted hostname property
> is used
> ---
>
> Key: NIFI-4194
> URL: https://issues.apache.org/jira/browse/NIFI-4194
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Assignee: Nathan Gough
>Priority: Major
> Labels: hostname, okhttp, security, tls
> Attachments: Screen Shot 2018-07-12 at 3.29.50 PM.png,
> flow_20170717-192339_trusted_hostname.xml
>
>
> When a user configures the {{InvokeHTTP}} processor with HTTPS (using an
> {{SSLContextService}}) and populates the {{trustedHostname}} property, the
> processor will throw a {{NullPointerException}} because the OkHttp client
> does not have a valid {{HostnameVerifier}} configured when the
> {{@OnScheduled}} method is called and that verifier is delegated to the
> processor. This results in the stacktrace below:
> {code}
> 2017-07-17 19:15:49,341 ERROR [Timer-Driven Process Thread-6]
> o.a.nifi.processors.standard.InvokeHTTP
> InvokeHTTP[id=53784003-015d-1000-ffb0-e07173729c9c] Routing to Failure due to
> exception: java.lang.NullPointerException: java.lang.NullPointerException
> java.lang.NullPointerException: null
> at
> org.apache.nifi.processors.standard.InvokeHTTP$OverrideHostnameVerifier.verify(InvokeHTTP.java:1050)
> at
> com.squareup.okhttp.internal.io.RealConnection.connectTls(RealConnection.java:192)
> at
> com.squareup.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:145)
> at
> com.squareup.okhttp.internal.io.RealConnection.connect(RealConnection.java:108)
> at
> com.squareup.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:184)
> at
> com.squareup.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:126)
> at
> com.squareup.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:95)
> at
> com.squareup.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:283)
> at
> com.squareup.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:224)
> at com.squareup.okhttp.Call.getResponse(Call.java:286)
> at
> com.squareup.okhttp.Call$ApplicationInterceptorChain.proceed(Call.java:243)
> at
> com.squareup.okhttp.Call.getResponseWithInterceptorChain(Call.java:205)
> at com.squareup.okhttp.Call.execute(Call.java:80)
> at
> org.apache.nifi.processors.standard.InvokeHTTP.onTrigger(InvokeHTTP.java:630)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> When that method is invoked, the existence of the {{HostnameVerifier}} should
> be checked before it is provided to the {{OverrideHostnameVerifier}}, and a
> default value of {{OkHostnameVerifier.INSTANCE}} (public static singleton)
> should be provided if none is available.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4194) NullPointerException in InvokeHTTP processor when trusted hostname property is used
[
https://issues.apache.org/jira/browse/NIFI-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542189#comment-16542189
]
Nathan Gough commented on NIFI-4194:
[~tony0918]
Hi Tony,
This issue was fixed by an upgrade to OkHttp version from 2.7.1 in NiFi v1.3.0,
to 3.8.1 in NiFi v1.4.0. This changed the way the HostnameVerifier is used by
the InvokeHTTP processor. I have verified this by building NiFi v1.3.0 and
experienced this exception:
!Screen Shot 2018-07-12 at 3.29.50 PM.png!
> NullPointerException in InvokeHTTP processor when trusted hostname property
> is used
> ---
>
> Key: NIFI-4194
> URL: https://issues.apache.org/jira/browse/NIFI-4194
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Andy LoPresto
>Assignee: Nathan Gough
>Priority: Major
> Labels: hostname, okhttp, security, tls
> Attachments: flow_20170717-192339_trusted_hostname.xml
>
>
> When a user configures the {{InvokeHTTP}} processor with HTTPS (using an
> {{SSLContextService}}) and populates the {{trustedHostname}} property, the
> processor will throw a {{NullPointerException}} because the OkHttp client
> does not have a valid {{HostnameVerifier}} configured when the
> {{@OnScheduled}} method is called and that verifier is delegated to the
> processor. This results in the stacktrace below:
> {code}
> 2017-07-17 19:15:49,341 ERROR [Timer-Driven Process Thread-6]
> o.a.nifi.processors.standard.InvokeHTTP
> InvokeHTTP[id=53784003-015d-1000-ffb0-e07173729c9c] Routing to Failure due to
> exception: java.lang.NullPointerException: java.lang.NullPointerException
> java.lang.NullPointerException: null
> at
> org.apache.nifi.processors.standard.InvokeHTTP$OverrideHostnameVerifier.verify(InvokeHTTP.java:1050)
> at
> com.squareup.okhttp.internal.io.RealConnection.connectTls(RealConnection.java:192)
> at
> com.squareup.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:145)
> at
> com.squareup.okhttp.internal.io.RealConnection.connect(RealConnection.java:108)
> at
> com.squareup.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:184)
> at
> com.squareup.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:126)
> at
> com.squareup.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:95)
> at
> com.squareup.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:283)
> at
> com.squareup.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:224)
> at com.squareup.okhttp.Call.getResponse(Call.java:286)
> at
> com.squareup.okhttp.Call$ApplicationInterceptorChain.proceed(Call.java:243)
> at
> com.squareup.okhttp.Call.getResponseWithInterceptorChain(Call.java:205)
> at com.squareup.okhttp.Call.execute(Call.java:80)
> at
> org.apache.nifi.processors.standard.InvokeHTTP.onTrigger(InvokeHTTP.java:630)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> When that method is invoked, the existence of the {{HostnameVerifier}} should
> be checked before it is provided to the {{OverrideHostnameVerifier}}, and a
> default value of {{OkHostnameVerifier.INSTANCE}} (public static singleton)
> should be provided if none is available.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2816: NIFI-5337 Syslog 5424 Record Reader and nifi-syslog...
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2816#discussion_r202168428
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/syslog/Syslog5424Reader.java
---
@@ -0,0 +1,151 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.syslog;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.SchemaAccessStrategy;
+import org.apache.nifi.schema.access.SchemaField;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.schemaregistry.services.SchemaRegistry;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.RecordReaderFactory;
+import org.apache.nifi.serialization.SchemaRegistryService;
+import org.apache.nifi.serialization.SimpleRecordSchema;
+import org.apache.nifi.serialization.record.RecordField;
+import org.apache.nifi.serialization.record.RecordFieldType;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.syslog.attributes.Syslog5424Attributes;
+import org.apache.nifi.syslog.attributes.SyslogAttributes;
+import org.apache.nifi.syslog.keyproviders.SimpleKeyProvider;
+import org.apache.nifi.syslog.parsers.StrictSyslog5424Parser;
+import org.apache.nifi.syslog.utils.NifiStructuredDataPolicy;
+import org.apache.nifi.syslog.utils.NilHandlingPolicy;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.EnumSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+@Tags({"syslog 5424", "syslog", "logs", "logfiles", "parse", "text",
"record", "reader"})
+@CapabilityDescription("Provides a mechanism for reading RFC 5424
compliant Syslog data, such as log files, and structuring the data" +
+" so that it can be processed.")
+public class Syslog5424Reader extends SchemaRegistryService implements
RecordReaderFactory {
+static final AllowableValue RFC_5424_SCHEMA = new
AllowableValue("default-5424-schema", "Use RFC 5424 Schema",
+"The schema will be the default schema per RFC 5424.");
+public static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("Character Set")
+.description("Specifies which character set of the Syslog
messages")
+.required(true)
+.defaultValue("UTF-8")
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+private volatile StrictSyslog5424Parser parser;
+private volatile RecordSchema recordSchema;
+
+@Override
+protected List getSupportedPropertyDescriptors() {
+final List properties = new ArrayList<>(1);
+properties.add(CHARSET);
+return properties;
+}
+
+
+@OnEnabled
+public void onEnabled(final ConfigurationContext context) {
+final String charsetName = context.getProperty(CHARSET).getValue();
+parser = new StrictSyslog5424Parser(Charset.forName(charsetName),
NilHandlingPolicy.NULL, NifiStructuredDataPolicy.MAP_OF_MAPS, new
SimpleKeyProvider());
+recordSchema = createRecordSchema();
+}
+
+@Override
+protected List getSchemaAccessStrategyValues() {
+final
[jira] [Commented] (NIFI-5337) Add Syslog 5424 Record Reader and create nifi-syslog-utils
[
https://issues.apache.org/jira/browse/NIFI-5337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542172#comment-16542172
]
ASF GitHub Bot commented on NIFI-5337:
--
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2816#discussion_r202168428
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/syslog/Syslog5424Reader.java
---
@@ -0,0 +1,151 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.syslog;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.SchemaAccessStrategy;
+import org.apache.nifi.schema.access.SchemaField;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.schemaregistry.services.SchemaRegistry;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.RecordReaderFactory;
+import org.apache.nifi.serialization.SchemaRegistryService;
+import org.apache.nifi.serialization.SimpleRecordSchema;
+import org.apache.nifi.serialization.record.RecordField;
+import org.apache.nifi.serialization.record.RecordFieldType;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.syslog.attributes.Syslog5424Attributes;
+import org.apache.nifi.syslog.attributes.SyslogAttributes;
+import org.apache.nifi.syslog.keyproviders.SimpleKeyProvider;
+import org.apache.nifi.syslog.parsers.StrictSyslog5424Parser;
+import org.apache.nifi.syslog.utils.NifiStructuredDataPolicy;
+import org.apache.nifi.syslog.utils.NilHandlingPolicy;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.EnumSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+@Tags({"syslog 5424", "syslog", "logs", "logfiles", "parse", "text",
"record", "reader"})
+@CapabilityDescription("Provides a mechanism for reading RFC 5424
compliant Syslog data, such as log files, and structuring the data" +
+" so that it can be processed.")
+public class Syslog5424Reader extends SchemaRegistryService implements
RecordReaderFactory {
+static final AllowableValue RFC_5424_SCHEMA = new
AllowableValue("default-5424-schema", "Use RFC 5424 Schema",
+"The schema will be the default schema per RFC 5424.");
+public static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("Character Set")
+.description("Specifies which character set of the Syslog
messages")
+.required(true)
+.defaultValue("UTF-8")
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+private volatile StrictSyslog5424Parser parser;
+private volatile RecordSchema recordSchema;
+
+@Override
+protected List getSupportedPropertyDescriptors() {
+final List properties = new ArrayList<>(1);
+properties.add(CHARSET);
+return properties;
+}
+
+
+@OnEnabled
+public void onEnabled(final ConfigurationContext context) {
+final String charsetName = context.getProperty(CHARSET).getValue();
+parser = new StrictSyslog5424Parser(Charset.forName(charsetName),
[GitHub] nifi pull request #2816: NIFI-5337 Syslog 5424 Record Reader and nifi-syslog...
Github user bbende commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2816#discussion_r202165722
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/syslog/Syslog5424Reader.java
---
@@ -0,0 +1,151 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.syslog;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.SchemaAccessStrategy;
+import org.apache.nifi.schema.access.SchemaField;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.schemaregistry.services.SchemaRegistry;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.RecordReaderFactory;
+import org.apache.nifi.serialization.SchemaRegistryService;
+import org.apache.nifi.serialization.SimpleRecordSchema;
+import org.apache.nifi.serialization.record.RecordField;
+import org.apache.nifi.serialization.record.RecordFieldType;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.syslog.attributes.Syslog5424Attributes;
+import org.apache.nifi.syslog.attributes.SyslogAttributes;
+import org.apache.nifi.syslog.keyproviders.SimpleKeyProvider;
+import org.apache.nifi.syslog.parsers.StrictSyslog5424Parser;
+import org.apache.nifi.syslog.utils.NifiStructuredDataPolicy;
+import org.apache.nifi.syslog.utils.NilHandlingPolicy;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.EnumSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+@Tags({"syslog 5424", "syslog", "logs", "logfiles", "parse", "text",
"record", "reader"})
+@CapabilityDescription("Provides a mechanism for reading RFC 5424
compliant Syslog data, such as log files, and structuring the data" +
+" so that it can be processed.")
+public class Syslog5424Reader extends SchemaRegistryService implements
RecordReaderFactory {
+static final AllowableValue RFC_5424_SCHEMA = new
AllowableValue("default-5424-schema", "Use RFC 5424 Schema",
+"The schema will be the default schema per RFC 5424.");
+public static final PropertyDescriptor CHARSET = new
PropertyDescriptor.Builder()
+.name("Character Set")
+.description("Specifies which character set of the Syslog
messages")
+.required(true)
+.defaultValue("UTF-8")
+.addValidator(StandardValidators.CHARACTER_SET_VALIDATOR)
+.build();
+
+private volatile StrictSyslog5424Parser parser;
+private volatile RecordSchema recordSchema;
+
+@Override
+protected List getSupportedPropertyDescriptors() {
+final List properties = new ArrayList<>(1);
+properties.add(CHARSET);
+return properties;
+}
+
+
+@OnEnabled
+public void onEnabled(final ConfigurationContext context) {
+final String charsetName = context.getProperty(CHARSET).getValue();
+parser = new StrictSyslog5424Parser(Charset.forName(charsetName),
NilHandlingPolicy.NULL, NifiStructuredDataPolicy.MAP_OF_MAPS, new
SimpleKeyProvider());
+recordSchema = createRecordSchema();
+}
+
+@Override
+protected List getSchemaAccessStrategyValues() {
+final List
[GitHub] nifi-registry issue #130: NIFIREG-185 Added element id's to facilitate User ...
Github user scottyaslan commented on the issue: https://github.com/apache/nifi-registry/pull/130 Reviewing... ---
[jira] [Commented] (NIFIREG-185) Add element id's for User testing
[ https://issues.apache.org/jira/browse/NIFIREG-185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542163#comment-16542163 ] ASF GitHub Bot commented on NIFIREG-185: Github user scottyaslan commented on the issue: https://github.com/apache/nifi-registry/pull/130 Reviewing... > Add element id's for User testing > - > > Key: NIFIREG-185 > URL: https://issues.apache.org/jira/browse/NIFIREG-185 > Project: NiFi Registry > Issue Type: Task >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Minor > > I need some element id's added to perform some automated UI testing around > User test cases (User creation/deletion/duplicates etc.). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-registry pull request #130: NIFIREG-185 Added element id's to facilitat...
GitHub user andrewmlim opened a pull request: https://github.com/apache/nifi-registry/pull/130 NIFIREG-185 Added element id's to facilitate User testing use cases You can merge this pull request into a Git repository by running: $ git pull https://github.com/andrewmlim/nifi-registry NIFIREG-185 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-registry/pull/130.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #130 commit e6a765f4a80530787b0391708197ca820136cf06 Author: Andrew Lim Date: 2018-07-12T20:03:29Z NIFIREG-185 Added elment id's to facilitate User testing use cases ---
[jira] [Commented] (NIFIREG-185) Add element id's for User testing
[ https://issues.apache.org/jira/browse/NIFIREG-185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542156#comment-16542156 ] ASF GitHub Bot commented on NIFIREG-185: GitHub user andrewmlim opened a pull request: https://github.com/apache/nifi-registry/pull/130 NIFIREG-185 Added element id's to facilitate User testing use cases You can merge this pull request into a Git repository by running: $ git pull https://github.com/andrewmlim/nifi-registry NIFIREG-185 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-registry/pull/130.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #130 commit e6a765f4a80530787b0391708197ca820136cf06 Author: Andrew Lim Date: 2018-07-12T20:03:29Z NIFIREG-185 Added elment id's to facilitate User testing use cases > Add element id's for User testing > - > > Key: NIFIREG-185 > URL: https://issues.apache.org/jira/browse/NIFIREG-185 > Project: NiFi Registry > Issue Type: Task >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Minor > > I need some element id's added to perform some automated UI testing around > User test cases (User creation/deletion/duplicates etc.). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5414) Release Apache NiFi 1.7.1
[ https://issues.apache.org/jira/browse/NIFI-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542151#comment-16542151 ] ASF subversion and git services commented on NIFI-5414: --- Commit 241e5411a24435ca2c39ab7bea1c5eb9ed195cb3 in nifi's branch refs/heads/NIFI-5414-RC1 from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=241e541 ] NIFI-5414-RC1 prepare release nifi-1.7.1-RC1 > Release Apache NiFi 1.7.1 > - > > Key: NIFI-5414 > URL: https://issues.apache.org/jira/browse/NIFI-5414 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Affects Versions: 1.7.1 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Blocker > Labels: release > Fix For: 1.7.1 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5414) Release Apache NiFi 1.7.1
[ https://issues.apache.org/jira/browse/NIFI-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542150#comment-16542150 ] ASF subversion and git services commented on NIFI-5414: --- Commit 8bbef92c589d4854ceba50f7e7d47f37bc5c3abc in nifi's branch refs/heads/NIFI-5414-RC1 from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=8bbef92 ] NIFI-5414 Manually set pom versions to 1.7.1-SNAPSHOT to allow mvn release plugin to sign release. > Release Apache NiFi 1.7.1 > - > > Key: NIFI-5414 > URL: https://issues.apache.org/jira/browse/NIFI-5414 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Affects Versions: 1.7.1 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Blocker > Labels: release > Fix For: 1.7.1 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5414) Release Apache NiFi 1.7.1
[ https://issues.apache.org/jira/browse/NIFI-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542152#comment-16542152 ] ASF subversion and git services commented on NIFI-5414: --- Commit 36f6318174b2088b1fa269021e75d2e6dd03ae28 in nifi's branch refs/heads/NIFI-5414-RC1 from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=36f6318 ] NIFI-5414-RC1 prepare for next development iteration > Release Apache NiFi 1.7.1 > - > > Key: NIFI-5414 > URL: https://issues.apache.org/jira/browse/NIFI-5414 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Affects Versions: 1.7.1 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Blocker > Labels: release > Fix For: 1.7.1 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5414) Release Apache NiFi 1.7.1
[ https://issues.apache.org/jira/browse/NIFI-5414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542149#comment-16542149 ] ASF subversion and git services commented on NIFI-5414: --- Commit e959630c22c9a52ec717141f6cf9f018830a38bf in nifi's branch refs/heads/NIFI-5414-RC1 from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=e959630 ] NIFI-5414 Fixed checkstyle error due to unused import. > Release Apache NiFi 1.7.1 > - > > Key: NIFI-5414 > URL: https://issues.apache.org/jira/browse/NIFI-5414 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Affects Versions: 1.7.1 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Blocker > Labels: release > Fix For: 1.7.1 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4889) Logout not working properly with OIDC
[ https://issues.apache.org/jira/browse/NIFI-4889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16542118#comment-16542118 ] ASF GitHub Bot commented on NIFI-4889: -- Github user Trojan295 commented on the issue: https://github.com/apache/nifi/pull/2830 Rebased from current master > Logout not working properly with OIDC > - > > Key: NIFI-4889 > URL: https://issues.apache.org/jira/browse/NIFI-4889 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.5.0 > Environment: Browser: Chrome / Firefox > Configuration of NiFi: > - SSL certificate for the server (no client auth) > - OIDC configuration including end_session_endpoint (see the link > https://auth.s.orchestracities.com/auth/realms/default/.well-known/openid-configuration) >Reporter: Federico Michele Facca >Priority: Critical > > Click on logout, i would expect to logout and getting redirect to the auth > page. But given that the session is not closed on the oauth provider, i get > logged in again. > I suppose the solution would be to invoke the end_session_endpoint provided > in the openid discovery configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2830: NIFI-4889: Logout not working properly with OIDC
Github user Trojan295 commented on the issue: https://github.com/apache/nifi/pull/2830 Rebased from current master ---
[jira] [Assigned] (NIFIREG-185) Add element id's for User testing
[ https://issues.apache.org/jira/browse/NIFIREG-185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Lim reassigned NIFIREG-185: -- Assignee: Andrew Lim > Add element id's for User testing > - > > Key: NIFIREG-185 > URL: https://issues.apache.org/jira/browse/NIFIREG-185 > Project: NiFi Registry > Issue Type: Task >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Minor > > I need some element id's added to perform some automated UI testing around > User test cases (User creation/deletion/duplicates etc.). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5337) Add Syslog 5424 Record Reader and create nifi-syslog-utils
[
https://issues.apache.org/jira/browse/NIFI-5337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541959#comment-16541959
]
ASF GitHub Bot commented on NIFI-5337:
--
Github user bbende commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2816#discussion_r202109378
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/syslog/Syslog5424RecordReader.java
---
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.syslog;
+
+import org.apache.nifi.serialization.MalformedRecordException;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.record.MapRecord;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.syslog.attributes.SyslogAttributes;
+import org.apache.nifi.syslog.events.Syslog5424Event;
+import org.apache.nifi.syslog.parsers.StrictSyslog5424Parser;
+import org.apache.nifi.util.StringUtils;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.nio.ByteBuffer;
+import java.sql.Timestamp;
+import java.time.Instant;
+import java.time.format.DateTimeFormatter;
+import java.util.HashMap;
+import java.util.Map;
+
+public class Syslog5424RecordReader implements RecordReader {
+private final BufferedReader reader;
+private RecordSchema schema;
+private final StrictSyslog5424Parser parser;
+
+public Syslog5424RecordReader(StrictSyslog5424Parser parser,
InputStream in, RecordSchema schema){
+this.reader = new BufferedReader(new InputStreamReader(in));
+this.schema = schema;
+this.parser = parser;
+}
+
+@Override
+public Record nextRecord(boolean coerceTypes, boolean
dropUnknownFields) throws IOException, MalformedRecordException {
+String line = reader.readLine();
+
+if ( line == null || StringUtils.isBlank(line)) {
--- End diff --
Yea I also considered the idea of reading until a non-blank line or null,
but then I thought about the case of something like ListenTCPRecord where the
InputStream could be an unbounded stream over a socket, and you could
potentially end up stuck inside the nextRecord method reading blank lines
forever (unlikely of course).
> Add Syslog 5424 Record Reader and create nifi-syslog-utils
> --
>
> Key: NIFI-5337
> URL: https://issues.apache.org/jira/browse/NIFI-5337
> Project: Apache NiFi
> Issue Type: Sub-task
>Reporter: Otto Fowler
>Assignee: Otto Fowler
>Priority: Major
>
> Create Record Reader for Syslog 5424 messages ( including structured data )
> built on the ParseSyslog5424 processor infrastructure.
>
> Create nifi-syslog-utils to centralize syslog support to be shared better the
> standard processors and the serialization library
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2816: NIFI-5337 Syslog 5424 Record Reader and nifi-syslog...
Github user bbende commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2816#discussion_r202109378
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/syslog/Syslog5424RecordReader.java
---
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.syslog;
+
+import org.apache.nifi.serialization.MalformedRecordException;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.record.MapRecord;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.syslog.attributes.SyslogAttributes;
+import org.apache.nifi.syslog.events.Syslog5424Event;
+import org.apache.nifi.syslog.parsers.StrictSyslog5424Parser;
+import org.apache.nifi.util.StringUtils;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.nio.ByteBuffer;
+import java.sql.Timestamp;
+import java.time.Instant;
+import java.time.format.DateTimeFormatter;
+import java.util.HashMap;
+import java.util.Map;
+
+public class Syslog5424RecordReader implements RecordReader {
+private final BufferedReader reader;
+private RecordSchema schema;
+private final StrictSyslog5424Parser parser;
+
+public Syslog5424RecordReader(StrictSyslog5424Parser parser,
InputStream in, RecordSchema schema){
+this.reader = new BufferedReader(new InputStreamReader(in));
+this.schema = schema;
+this.parser = parser;
+}
+
+@Override
+public Record nextRecord(boolean coerceTypes, boolean
dropUnknownFields) throws IOException, MalformedRecordException {
+String line = reader.readLine();
+
+if ( line == null || StringUtils.isBlank(line)) {
--- End diff --
Yea I also considered the idea of reading until a non-blank line or null,
but then I thought about the case of something like ListenTCPRecord where the
InputStream could be an unbounded stream over a socket, and you could
potentially end up stuck inside the nextRecord method reading blank lines
forever (unlikely of course).
---
[jira] [Commented] (NIFI-5337) Add Syslog 5424 Record Reader and create nifi-syslog-utils
[
https://issues.apache.org/jira/browse/NIFI-5337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541945#comment-16541945
]
ASF GitHub Bot commented on NIFI-5337:
--
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2816#discussion_r202107396
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/syslog/Syslog5424RecordReader.java
---
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.syslog;
+
+import org.apache.nifi.serialization.MalformedRecordException;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.record.MapRecord;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.syslog.attributes.SyslogAttributes;
+import org.apache.nifi.syslog.events.Syslog5424Event;
+import org.apache.nifi.syslog.parsers.StrictSyslog5424Parser;
+import org.apache.nifi.util.StringUtils;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.nio.ByteBuffer;
+import java.sql.Timestamp;
+import java.time.Instant;
+import java.time.format.DateTimeFormatter;
+import java.util.HashMap;
+import java.util.Map;
+
+public class Syslog5424RecordReader implements RecordReader {
+private final BufferedReader reader;
+private RecordSchema schema;
+private final StrictSyslog5424Parser parser;
+
+public Syslog5424RecordReader(StrictSyslog5424Parser parser,
InputStream in, RecordSchema schema){
+this.reader = new BufferedReader(new InputStreamReader(in));
+this.schema = schema;
+this.parser = parser;
+}
+
+@Override
+public Record nextRecord(boolean coerceTypes, boolean
dropUnknownFields) throws IOException, MalformedRecordException {
+String line = reader.readLine();
+
+if ( line == null || StringUtils.isBlank(line)) {
--- End diff --
OK OK, @bbende, sorry I understand what you actually meant, and have made
changes accordingly
> Add Syslog 5424 Record Reader and create nifi-syslog-utils
> --
>
> Key: NIFI-5337
> URL: https://issues.apache.org/jira/browse/NIFI-5337
> Project: Apache NiFi
> Issue Type: Sub-task
>Reporter: Otto Fowler
>Assignee: Otto Fowler
>Priority: Major
>
> Create Record Reader for Syslog 5424 messages ( including structured data )
> built on the ParseSyslog5424 processor infrastructure.
>
> Create nifi-syslog-utils to centralize syslog support to be shared better the
> standard processors and the serialization library
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2816: NIFI-5337 Syslog 5424 Record Reader and nifi-syslog...
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2816#discussion_r202107396
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/syslog/Syslog5424RecordReader.java
---
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.syslog;
+
+import org.apache.nifi.serialization.MalformedRecordException;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.record.MapRecord;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.syslog.attributes.SyslogAttributes;
+import org.apache.nifi.syslog.events.Syslog5424Event;
+import org.apache.nifi.syslog.parsers.StrictSyslog5424Parser;
+import org.apache.nifi.util.StringUtils;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.nio.ByteBuffer;
+import java.sql.Timestamp;
+import java.time.Instant;
+import java.time.format.DateTimeFormatter;
+import java.util.HashMap;
+import java.util.Map;
+
+public class Syslog5424RecordReader implements RecordReader {
+private final BufferedReader reader;
+private RecordSchema schema;
+private final StrictSyslog5424Parser parser;
+
+public Syslog5424RecordReader(StrictSyslog5424Parser parser,
InputStream in, RecordSchema schema){
+this.reader = new BufferedReader(new InputStreamReader(in));
+this.schema = schema;
+this.parser = parser;
+}
+
+@Override
+public Record nextRecord(boolean coerceTypes, boolean
dropUnknownFields) throws IOException, MalformedRecordException {
+String line = reader.readLine();
+
+if ( line == null || StringUtils.isBlank(line)) {
--- End diff --
OK OK, @bbende, sorry I understand what you actually meant, and have made
changes accordingly
---
[jira] [Commented] (NIFI-5337) Add Syslog 5424 Record Reader and create nifi-syslog-utils
[
https://issues.apache.org/jira/browse/NIFI-5337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541936#comment-16541936
]
ASF GitHub Bot commented on NIFI-5337:
--
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2816#discussion_r202105880
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/syslog/Syslog5424RecordReader.java
---
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.syslog;
+
+import org.apache.nifi.serialization.MalformedRecordException;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.record.MapRecord;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.syslog.attributes.SyslogAttributes;
+import org.apache.nifi.syslog.events.Syslog5424Event;
+import org.apache.nifi.syslog.parsers.StrictSyslog5424Parser;
+import org.apache.nifi.util.StringUtils;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.nio.ByteBuffer;
+import java.sql.Timestamp;
+import java.time.Instant;
+import java.time.format.DateTimeFormatter;
+import java.util.HashMap;
+import java.util.Map;
+
+public class Syslog5424RecordReader implements RecordReader {
+private final BufferedReader reader;
+private RecordSchema schema;
+private final StrictSyslog5424Parser parser;
+
+public Syslog5424RecordReader(StrictSyslog5424Parser parser,
InputStream in, RecordSchema schema){
+this.reader = new BufferedReader(new InputStreamReader(in));
+this.schema = schema;
+this.parser = parser;
+}
+
+@Override
+public Record nextRecord(boolean coerceTypes, boolean
dropUnknownFields) throws IOException, MalformedRecordException {
+String line = reader.readLine();
+
+if ( line == null || StringUtils.isBlank(line)) {
--- End diff --
Ok, looked into this more. The return null behavior was modeled on grok
handling, which does the same. I think this is the right thing to do, but I
*could* see trying to deadline() until EOF or not null
> Add Syslog 5424 Record Reader and create nifi-syslog-utils
> --
>
> Key: NIFI-5337
> URL: https://issues.apache.org/jira/browse/NIFI-5337
> Project: Apache NiFi
> Issue Type: Sub-task
>Reporter: Otto Fowler
>Assignee: Otto Fowler
>Priority: Major
>
> Create Record Reader for Syslog 5424 messages ( including structured data )
> built on the ParseSyslog5424 processor infrastructure.
>
> Create nifi-syslog-utils to centralize syslog support to be shared better the
> standard processors and the serialization library
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2816: NIFI-5337 Syslog 5424 Record Reader and nifi-syslog...
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2816#discussion_r202105880
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/syslog/Syslog5424RecordReader.java
---
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.syslog;
+
+import org.apache.nifi.serialization.MalformedRecordException;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.record.MapRecord;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.syslog.attributes.SyslogAttributes;
+import org.apache.nifi.syslog.events.Syslog5424Event;
+import org.apache.nifi.syslog.parsers.StrictSyslog5424Parser;
+import org.apache.nifi.util.StringUtils;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.nio.ByteBuffer;
+import java.sql.Timestamp;
+import java.time.Instant;
+import java.time.format.DateTimeFormatter;
+import java.util.HashMap;
+import java.util.Map;
+
+public class Syslog5424RecordReader implements RecordReader {
+private final BufferedReader reader;
+private RecordSchema schema;
+private final StrictSyslog5424Parser parser;
+
+public Syslog5424RecordReader(StrictSyslog5424Parser parser,
InputStream in, RecordSchema schema){
+this.reader = new BufferedReader(new InputStreamReader(in));
+this.schema = schema;
+this.parser = parser;
+}
+
+@Override
+public Record nextRecord(boolean coerceTypes, boolean
dropUnknownFields) throws IOException, MalformedRecordException {
+String line = reader.readLine();
+
+if ( line == null || StringUtils.isBlank(line)) {
--- End diff --
Ok, looked into this more. The return null behavior was modeled on grok
handling, which does the same. I think this is the right thing to do, but I
*could* see trying to deadline() until EOF or not null
---
[jira] [Commented] (NIFI-5337) Add Syslog 5424 Record Reader and create nifi-syslog-utils
[
https://issues.apache.org/jira/browse/NIFI-5337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541924#comment-16541924
]
ASF GitHub Bot commented on NIFI-5337:
--
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2816#discussion_r202103469
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/syslog/Syslog5424RecordReader.java
---
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.syslog;
+
+import org.apache.nifi.serialization.MalformedRecordException;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.record.MapRecord;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.syslog.attributes.SyslogAttributes;
+import org.apache.nifi.syslog.events.Syslog5424Event;
+import org.apache.nifi.syslog.parsers.StrictSyslog5424Parser;
+import org.apache.nifi.util.StringUtils;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.nio.ByteBuffer;
+import java.sql.Timestamp;
+import java.time.Instant;
+import java.time.format.DateTimeFormatter;
+import java.util.HashMap;
+import java.util.Map;
+
+public class Syslog5424RecordReader implements RecordReader {
+private final BufferedReader reader;
+private RecordSchema schema;
+private final StrictSyslog5424Parser parser;
+
+public Syslog5424RecordReader(StrictSyslog5424Parser parser,
InputStream in, RecordSchema schema){
+this.reader = new BufferedReader(new InputStreamReader(in));
+this.schema = schema;
+this.parser = parser;
+}
+
+@Override
+public Record nextRecord(boolean coerceTypes, boolean
dropUnknownFields) throws IOException, MalformedRecordException {
+String line = reader.readLine();
+
+if ( line == null || StringUtils.isBlank(line)) {
--- End diff --
Although, I wonder if we could instead read until we git a non-blank line?
In other words skip blanks?
> Add Syslog 5424 Record Reader and create nifi-syslog-utils
> --
>
> Key: NIFI-5337
> URL: https://issues.apache.org/jira/browse/NIFI-5337
> Project: Apache NiFi
> Issue Type: Sub-task
>Reporter: Otto Fowler
>Assignee: Otto Fowler
>Priority: Major
>
> Create Record Reader for Syslog 5424 messages ( including structured data )
> built on the ParseSyslog5424 processor infrastructure.
>
> Create nifi-syslog-utils to centralize syslog support to be shared better the
> standard processors and the serialization library
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2816: NIFI-5337 Syslog 5424 Record Reader and nifi-syslog...
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2816#discussion_r202103469
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/syslog/Syslog5424RecordReader.java
---
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.syslog;
+
+import org.apache.nifi.serialization.MalformedRecordException;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.record.MapRecord;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.syslog.attributes.SyslogAttributes;
+import org.apache.nifi.syslog.events.Syslog5424Event;
+import org.apache.nifi.syslog.parsers.StrictSyslog5424Parser;
+import org.apache.nifi.util.StringUtils;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.nio.ByteBuffer;
+import java.sql.Timestamp;
+import java.time.Instant;
+import java.time.format.DateTimeFormatter;
+import java.util.HashMap;
+import java.util.Map;
+
+public class Syslog5424RecordReader implements RecordReader {
+private final BufferedReader reader;
+private RecordSchema schema;
+private final StrictSyslog5424Parser parser;
+
+public Syslog5424RecordReader(StrictSyslog5424Parser parser,
InputStream in, RecordSchema schema){
+this.reader = new BufferedReader(new InputStreamReader(in));
+this.schema = schema;
+this.parser = parser;
+}
+
+@Override
+public Record nextRecord(boolean coerceTypes, boolean
dropUnknownFields) throws IOException, MalformedRecordException {
+String line = reader.readLine();
+
+if ( line == null || StringUtils.isBlank(line)) {
--- End diff --
Although, I wonder if we could instead read until we git a non-blank line?
In other words skip blanks?
---
[jira] [Commented] (NIFI-5337) Add Syslog 5424 Record Reader and create nifi-syslog-utils
[
https://issues.apache.org/jira/browse/NIFI-5337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541919#comment-16541919
]
ASF GitHub Bot commented on NIFI-5337:
--
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2816#discussion_r202103018
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/syslog/Syslog5424RecordReader.java
---
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.syslog;
+
+import org.apache.nifi.serialization.MalformedRecordException;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.record.MapRecord;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.syslog.attributes.SyslogAttributes;
+import org.apache.nifi.syslog.events.Syslog5424Event;
+import org.apache.nifi.syslog.parsers.StrictSyslog5424Parser;
+import org.apache.nifi.util.StringUtils;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.nio.ByteBuffer;
+import java.sql.Timestamp;
+import java.time.Instant;
+import java.time.format.DateTimeFormatter;
+import java.util.HashMap;
+import java.util.Map;
+
+public class Syslog5424RecordReader implements RecordReader {
+private final BufferedReader reader;
+private RecordSchema schema;
+private final StrictSyslog5424Parser parser;
+
+public Syslog5424RecordReader(StrictSyslog5424Parser parser,
InputStream in, RecordSchema schema){
+this.reader = new BufferedReader(new InputStreamReader(in));
+this.schema = schema;
+this.parser = parser;
+}
+
+@Override
+public Record nextRecord(boolean coerceTypes, boolean
dropUnknownFields) throws IOException, MalformedRecordException {
+String line = reader.readLine();
+
+if ( line == null || StringUtils.isBlank(line)) {
--- End diff --
OK, that makes sense
> Add Syslog 5424 Record Reader and create nifi-syslog-utils
> --
>
> Key: NIFI-5337
> URL: https://issues.apache.org/jira/browse/NIFI-5337
> Project: Apache NiFi
> Issue Type: Sub-task
>Reporter: Otto Fowler
>Assignee: Otto Fowler
>Priority: Major
>
> Create Record Reader for Syslog 5424 messages ( including structured data )
> built on the ParseSyslog5424 processor infrastructure.
>
> Create nifi-syslog-utils to centralize syslog support to be shared better the
> standard processors and the serialization library
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2816: NIFI-5337 Syslog 5424 Record Reader and nifi-syslog...
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2816#discussion_r202103018
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/syslog/Syslog5424RecordReader.java
---
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.syslog;
+
+import org.apache.nifi.serialization.MalformedRecordException;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.record.MapRecord;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.syslog.attributes.SyslogAttributes;
+import org.apache.nifi.syslog.events.Syslog5424Event;
+import org.apache.nifi.syslog.parsers.StrictSyslog5424Parser;
+import org.apache.nifi.util.StringUtils;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.nio.ByteBuffer;
+import java.sql.Timestamp;
+import java.time.Instant;
+import java.time.format.DateTimeFormatter;
+import java.util.HashMap;
+import java.util.Map;
+
+public class Syslog5424RecordReader implements RecordReader {
+private final BufferedReader reader;
+private RecordSchema schema;
+private final StrictSyslog5424Parser parser;
+
+public Syslog5424RecordReader(StrictSyslog5424Parser parser,
InputStream in, RecordSchema schema){
+this.reader = new BufferedReader(new InputStreamReader(in));
+this.schema = schema;
+this.parser = parser;
+}
+
+@Override
+public Record nextRecord(boolean coerceTypes, boolean
dropUnknownFields) throws IOException, MalformedRecordException {
+String line = reader.readLine();
+
+if ( line == null || StringUtils.isBlank(line)) {
--- End diff --
OK, that makes sense
---
[jira] [Created] (NIFI-5419) Improve alias handling in DB Fetch processors
Matt Burgess created NIFI-5419:
--
Summary: Improve alias handling in DB Fetch processors
Key: NIFI-5419
URL: https://issues.apache.org/jira/browse/NIFI-5419
Project: Apache NiFi
Issue Type: Improvement
Components: Extensions
Reporter: Matt Burgess
Often the DB Fetch processors (GenerateTableFetch, QueryDatabaseTable, etc.)
ask the configured Database Type (aka DB adapter) for DB-specific things such
as generated SQL for SELECT, INSERT, etc. However GenerateTableFetch (as an
example), when it fetches the COUNT and MAX for a specified Maximum Value
Column, hard-codes the alias as the following:
{{SELECT COUNT(*), MAX(myColumn) myColumn}}
This works for many databases in many situations, but still can result in
errors. For example, in PostgreSQL if the column name is a reserved word, then
the above format will not work; the alias needs an explicit AS, and perhaps
even quoting.
This may be as simple as hard-coding the AS keyword into the query, but we'd
need to test this on all databases we wish the processor to work with. If it is
a DB-specific issue, then we'd need to move the logic into the DB Adapters and
have the fetch processors call to the adapter to get the correct clause.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5166) Create deep learning classification and regression processor
[ https://issues.apache.org/jira/browse/NIFI-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541572#comment-16541572 ] ASF GitHub Bot commented on NIFI-5166: -- Github user mans2singh commented on the issue: https://github.com/apache/nifi/pull/2686 Hi @ijokarumawak - I've merged your changes. Please let me know if you have any more recommendations. Thanks for your help. Mans > Create deep learning classification and regression processor > > > Key: NIFI-5166 > URL: https://issues.apache.org/jira/browse/NIFI-5166 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.6.0 >Reporter: Mans Singh >Assignee: Mans Singh >Priority: Minor > Labels: Learning, classification,, deep, regression, > Original Estimate: 168h > Remaining Estimate: 168h > > We need a deep learning classification and regression processor. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2686: NIFI-5166 - Deep learning classification and regression pr...
Github user mans2singh commented on the issue: https://github.com/apache/nifi/pull/2686 Hi @ijokarumawak - I've merged your changes. Please let me know if you have any more recommendations. Thanks for your help. Mans ---
[jira] [Commented] (NIFI-4889) Logout not working properly with OIDC
[ https://issues.apache.org/jira/browse/NIFI-4889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541566#comment-16541566 ] ASF GitHub Bot commented on NIFI-4889: -- Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/2830 @Trojan295 - NiFi 1.7.0 has been released, I'd recommend you to rebase against master (so that all the pom files are updated to 1.8.0-SNAPSHOT) and do a full build to have a clean local maven repo (you shouldn't have 1.7.0-SNAPSHOT anymore). All the Travis builds are OK for your PR so it sounds OK to me. Regarding the PR itself, I'll let someone like @mcgilman having a look ;) > Logout not working properly with OIDC > - > > Key: NIFI-4889 > URL: https://issues.apache.org/jira/browse/NIFI-4889 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.5.0 > Environment: Browser: Chrome / Firefox > Configuration of NiFi: > - SSL certificate for the server (no client auth) > - OIDC configuration including end_session_endpoint (see the link > https://auth.s.orchestracities.com/auth/realms/default/.well-known/openid-configuration) >Reporter: Federico Michele Facca >Priority: Critical > > Click on logout, i would expect to logout and getting redirect to the auth > page. But given that the session is not closed on the oauth provider, i get > logged in again. > I suppose the solution would be to invoke the end_session_endpoint provided > in the openid discovery configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2830: NIFI-4889: Logout not working properly with OIDC
Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/2830 @Trojan295 - NiFi 1.7.0 has been released, I'd recommend you to rebase against master (so that all the pom files are updated to 1.8.0-SNAPSHOT) and do a full build to have a clean local maven repo (you shouldn't have 1.7.0-SNAPSHOT anymore). All the Travis builds are OK for your PR so it sounds OK to me. Regarding the PR itself, I'll let someone like @mcgilman having a look ;) ---
[jira] [Commented] (NIFI-5418) Remove duplicated jackson-databind dependency from nifi-framework-cluster
[
https://issues.apache.org/jira/browse/NIFI-5418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541554#comment-16541554
]
ASF GitHub Bot commented on NIFI-5418:
--
GitHub user HorizonNet opened a pull request:
https://github.com/apache/nifi/pull/2885
NIFI-5418 Removed duplicated jackson-databind dependency from
nifi-framework-cluster
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with NIFI- where is the JIRA number
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [x] Is your initial contribution a single, squashed commit?
### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file, including the main
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in
which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ultratendency/nifi NIFI-5418
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/2885.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2885
commit 2515277eaa6ea619f4abe8778ca363149771fdf3
Author: Jan Hentschel
Date: 2018-07-12T12:27:26Z
NIFI-5418 Removed duplicated jackson-databind dependency from
nifi-framework-cluster
> Remove duplicated jackson-databind dependency from nifi-framework-cluster
> -
>
> Key: NIFI-5418
> URL: https://issues.apache.org/jira/browse/NIFI-5418
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Priority: Trivial
>
> Maven complains a duplicated {{jackson-databind}} dependency in the
> {{nifi-framework-cluster}}, one with the hard-coded version 2.9.4 and one,
> which gets it from the property resulting in the version 2.9.5. The
> dependency with the hard-coded version should be removed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5418) Remove duplicated jackson-databind dependency from nifi-framework-cluster
[
https://issues.apache.org/jira/browse/NIFI-5418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Hentschel updated NIFI-5418:
Status: Patch Available (was: Open)
> Remove duplicated jackson-databind dependency from nifi-framework-cluster
> -
>
> Key: NIFI-5418
> URL: https://issues.apache.org/jira/browse/NIFI-5418
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Priority: Trivial
>
> Maven complains a duplicated {{jackson-databind}} dependency in the
> {{nifi-framework-cluster}}, one with the hard-coded version 2.9.4 and one,
> which gets it from the property resulting in the version 2.9.5. The
> dependency with the hard-coded version should be removed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2885: NIFI-5418 Removed duplicated jackson-databind depen...
GitHub user HorizonNet opened a pull request: https://github.com/apache/nifi/pull/2885 NIFI-5418 Removed duplicated jackson-databind dependency from nifi-framework-cluster Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ultratendency/nifi NIFI-5418 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2885.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2885 commit 2515277eaa6ea619f4abe8778ca363149771fdf3 Author: Jan Hentschel Date: 2018-07-12T12:27:26Z NIFI-5418 Removed duplicated jackson-databind dependency from nifi-framework-cluster ---
[jira] [Created] (NIFI-5418) Remove duplicated jackson-databind dependency from nifi-framework-cluster
Jan Hentschel created NIFI-5418:
---
Summary: Remove duplicated jackson-databind dependency from
nifi-framework-cluster
Key: NIFI-5418
URL: https://issues.apache.org/jira/browse/NIFI-5418
Project: Apache NiFi
Issue Type: Improvement
Reporter: Jan Hentschel
Maven complains a duplicated {{jackson-databind}} dependency in the
{{nifi-framework-cluster}}, one with the hard-coded version 2.9.4 and one,
which gets it from the property resulting in the version 2.9.5. The dependency
with the hard-coded version should be removed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-3993) Upgrade embedded ZooKeeper version
[ https://issues.apache.org/jira/browse/NIFI-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541521#comment-16541521 ] ASF GitHub Bot commented on NIFI-3993: -- GitHub user HorizonNet opened a pull request: https://github.com/apache/nifi/pull/2884 NIFI-3993 Updated the ZooKeeper version to 3.4.10 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ultratendency/nifi NIFI-3993 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2884.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2884 commit 51bef0752de1a86b3d3173437ddcc4266b3f5905 Author: Jan Hentschel Date: 2018-07-12T11:47:05Z NIFI-3993 Updated the ZooKeeper version to 3.4.10 > Upgrade embedded ZooKeeper version > -- > > Key: NIFI-3993 > URL: https://issues.apache.org/jira/browse/NIFI-3993 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Mark Bean >Priority: Major > > In a Cluster configuration, Nodes are periodically disconnected from the > Cluster, and then reconnected. These events correspond to the following error: > ERROR [CommitProcessor:1] o.apache.zookeeper.server.NIOServerCnxn Unexpected > Exception: > java.nio.channels.CancelledKeyException: null > at sun.nio.ch.SelectionKeyImpl.ensureValid(SectionKeyImpl.java:73) > at sun.nio.ch.SelectionKeyImpl.interestOps(SelctionKeyImpl.java:77) > at > org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:151) > at > org.apache.zookeeper.server.NIOServerCnXn.sendResopnse(NIOServerCnxn.java:1081) > at > org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:404) > at > org.apache.zookeeper.server.quorum.CommitProcessor.run(CommitProcessor.java:74) > This error was reported in ZooKeeper JIRA [1], and reported as fixed in > version 3.4.10, the current stable build. As additional confirmation, when > using a stand-alone ZK, 3.4.10, rather than the embedded ZK, the above error > was no longer observed. > Update NiFi to use ZK 3.4.10 > [1] https://issues.apache.org/jira/browse/ZOOKEEPER-2044 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2884: NIFI-3993 Updated the ZooKeeper version to 3.4.10
GitHub user HorizonNet opened a pull request: https://github.com/apache/nifi/pull/2884 NIFI-3993 Updated the ZooKeeper version to 3.4.10 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ultratendency/nifi NIFI-3993 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2884.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2884 commit 51bef0752de1a86b3d3173437ddcc4266b3f5905 Author: Jan Hentschel Date: 2018-07-12T11:47:05Z NIFI-3993 Updated the ZooKeeper version to 3.4.10 ---
[GitHub] nifi issue #2830: NIFI-4889: Logout not working properly with OIDC
Github user Trojan295 commented on the issue: https://github.com/apache/nifi/pull/2830 When running the test suite locally I have problems with an dependency: ``` [ERROR] Failed to execute goal on project nifi-hive-processors: Could not resolve dependencies for project org.apache.nifi:nifi-hive-processors:jar:1.7.0-SNAPSHOT: Could not find artifact org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde in central (https://repo1.maven.org/maven2) -> [Help 1] ``` I had to add an additional repository to make it work, although I didn't commit it: ``` spring-plugins http://repo.spring.io/plugins-release false true ``` Could it be, that the artifact was in Maven Central and was moved, but the CI on Travis as it cached in the local repo? ---
[jira] [Commented] (NIFI-4889) Logout not working properly with OIDC
[ https://issues.apache.org/jira/browse/NIFI-4889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541470#comment-16541470 ] ASF GitHub Bot commented on NIFI-4889: -- Github user Trojan295 commented on the issue: https://github.com/apache/nifi/pull/2830 When running the test suite locally I have problems with an dependency: ``` [ERROR] Failed to execute goal on project nifi-hive-processors: Could not resolve dependencies for project org.apache.nifi:nifi-hive-processors:jar:1.7.0-SNAPSHOT: Could not find artifact org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde in central (https://repo1.maven.org/maven2) -> [Help 1] ``` I had to add an additional repository to make it work, although I didn't commit it: ``` spring-plugins http://repo.spring.io/plugins-release false true ``` Could it be, that the artifact was in Maven Central and was moved, but the CI on Travis as it cached in the local repo? > Logout not working properly with OIDC > - > > Key: NIFI-4889 > URL: https://issues.apache.org/jira/browse/NIFI-4889 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.5.0 > Environment: Browser: Chrome / Firefox > Configuration of NiFi: > - SSL certificate for the server (no client auth) > - OIDC configuration including end_session_endpoint (see the link > https://auth.s.orchestracities.com/auth/realms/default/.well-known/openid-configuration) >Reporter: Federico Michele Facca >Priority: Critical > > Click on logout, i would expect to logout and getting redirect to the auth > page. But given that the session is not closed on the oauth provider, i get > logged in again. > I suppose the solution would be to invoke the end_session_endpoint provided > in the openid discovery configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2830: NIFI-4889: Logout not working properly with OIDC
GitHub user Trojan295 reopened a pull request: https://github.com/apache/nifi/pull/2830 NIFI-4889: Logout not working properly with OIDC - Redirect user agent to end session endpoint on the OIDC provider Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Trojan295/nifi NIFI-4889 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2830.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2830 commit e679e2391dea018854f823d6546e8f2821cacd31 Author: Damian Czaja Date: 2018-07-02T11:35:03Z NIFI-4889: Logout not working properly with OIDC - Redirect user agent to end session endpoint on the OIDC provider ---
[jira] [Commented] (NIFI-4889) Logout not working properly with OIDC
[ https://issues.apache.org/jira/browse/NIFI-4889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541438#comment-16541438 ] ASF GitHub Bot commented on NIFI-4889: -- GitHub user Trojan295 reopened a pull request: https://github.com/apache/nifi/pull/2830 NIFI-4889: Logout not working properly with OIDC - Redirect user agent to end session endpoint on the OIDC provider Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Trojan295/nifi NIFI-4889 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2830.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2830 commit e679e2391dea018854f823d6546e8f2821cacd31 Author: Damian Czaja Date: 2018-07-02T11:35:03Z NIFI-4889: Logout not working properly with OIDC - Redirect user agent to end session endpoint on the OIDC provider > Logout not working properly with OIDC > - > > Key: NIFI-4889 > URL: https://issues.apache.org/jira/browse/NIFI-4889 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.5.0 > Environment: Browser: Chrome / Firefox > Configuration of NiFi: > - SSL certificate for the server (no client auth) > - OIDC configuration including end_session_endpoint (see the link > https://auth.s.orchestracities.com/auth/realms/default/.well-known/openid-configuration) >Reporter: Federico Michele Facca >Priority: Critical > > Click on logout, i would expect to logout and getting redirect to the auth > page. But given that the session is not closed on the oauth provider, i get > logged in again. > I suppose the solution would be to invoke the end_session_endpoint provided > in the openid discovery configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5406) Add new listing strategy by tracking listed entities to ListXXXX processors
[ https://issues.apache.org/jira/browse/NIFI-5406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541395#comment-16541395 ] Koji Kawamura commented on NIFI-5406: - [~joewitt] Thanks for your comments. I've updated the PR to let user to choose listing strategy. It behaves as is by default setting. > Add new listing strategy by tracking listed entities to List processors > --- > > Key: NIFI-5406 > URL: https://issues.apache.org/jira/browse/NIFI-5406 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura >Priority: Major > > Current List processors (ListFile, ListFTP, ListSFTP ... etc) implementation > relies on file last modified timestamp to pick new or updated files. This > approach is efficient and lightweight in terms of state management, because > it only tracks latest modified timestamp and last executed timestamp. > However, timestamps do not work as expected in some file systems, causing > List processors missing files periodically. See NIFI-3332 comments for > details. > In order to pick every entity that has not seen before or has been updated > since it had seen last time, we need another set of processors using > different approach, that is by tracking listed entities: > * Add new abstract processor AbstractWatchEntries similar to > AbstractListProcessor but uses different approach > * Target entities have: name (path), size and last-modified-timestamp > * Implementation Processors have following properties: > ** 'Watch Time Window' to limit the maximum time period to hold the already > listed entries. E.g. if set as '30min', the processor keeps entities listed > in the last 30 mins. > ** 'Minimum File Age' to defer listing entities potentially being written > * Any entity added but not listed ever having last-modified-timestamp older > than configured 'Watch Time Window' will not be listed. If user needs to pick > these items, they have to make 'Watch Time Window' longer. It also increases > the size of data the processor has to persist in the K/V store. Efficiency vs > reliability trade-off. > * The already-listed entities are persisted into one of supported K/V store > through DistributedMapCacheClient service. User can chose what KVS to use > from HBase, Redis, Couchbase and File (DistributedMapCacheServer with > persistence file). > * The reason to use KVS instead of ManagedState is, to avoid hammering > Zookeeper too much with frequently updating Zk node with large amount of > data. The number of already-listed entries can be huge depending on > use-cases. Also, we can compress entities with DistributedMapCacheClient as > it supports putting byte array, while ManagedState only supports Map String>. > * On each onTrigger: > ** Processor performs listing. Listed entries meeting any of the following > condition will be written to the 'success' output FlowFile: > *** Not exists in the already-listed entities > *** Having newer last-modified-timestamp > *** Having different size > ** Already listed entries those are old enough compared to 'Watch Time > Window' are discarded from the already-listed entries. > * Initial supporting target is Local file system, FTP and SFTP -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-5406) Add new listing strategy by tracking listed entities to ListXXXX processors
[ https://issues.apache.org/jira/browse/NIFI-5406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-5406: Summary: Add new listing strategy by tracking listed entities to List processors (was: Add processors to list new or updated files by tracking listed entities) > Add new listing strategy by tracking listed entities to List processors > --- > > Key: NIFI-5406 > URL: https://issues.apache.org/jira/browse/NIFI-5406 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura >Priority: Major > > Current List processors (ListFile, ListFTP, ListSFTP ... etc) implementation > relies on file last modified timestamp to pick new or updated files. This > approach is efficient and lightweight in terms of state management, because > it only tracks latest modified timestamp and last executed timestamp. > However, timestamps do not work as expected in some file systems, causing > List processors missing files periodically. See NIFI-3332 comments for > details. > In order to pick every entity that has not seen before or has been updated > since it had seen last time, we need another set of processors using > different approach, that is by tracking listed entities: > * Add new abstract processor AbstractWatchEntries similar to > AbstractListProcessor but uses different approach > * Target entities have: name (path), size and last-modified-timestamp > * Implementation Processors have following properties: > ** 'Watch Time Window' to limit the maximum time period to hold the already > listed entries. E.g. if set as '30min', the processor keeps entities listed > in the last 30 mins. > ** 'Minimum File Age' to defer listing entities potentially being written > * Any entity added but not listed ever having last-modified-timestamp older > than configured 'Watch Time Window' will not be listed. If user needs to pick > these items, they have to make 'Watch Time Window' longer. It also increases > the size of data the processor has to persist in the K/V store. Efficiency vs > reliability trade-off. > * The already-listed entities are persisted into one of supported K/V store > through DistributedMapCacheClient service. User can chose what KVS to use > from HBase, Redis, Couchbase and File (DistributedMapCacheServer with > persistence file). > * The reason to use KVS instead of ManagedState is, to avoid hammering > Zookeeper too much with frequently updating Zk node with large amount of > data. The number of already-listed entries can be huge depending on > use-cases. Also, we can compress entities with DistributedMapCacheClient as > it supports putting byte array, while ManagedState only supports Map String>. > * On each onTrigger: > ** Processor performs listing. Listed entries meeting any of the following > condition will be written to the 'success' output FlowFile: > *** Not exists in the already-listed entities > *** Having newer last-modified-timestamp > *** Having different size > ** Already listed entries those are old enough compared to 'Watch Time > Window' are discarded from the already-listed entries. > * Initial supporting target is Local file system, FTP and SFTP -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-5417) Add additional properties to status reporting
Taras Vuyiv created NIFI-5417: - Summary: Add additional properties to status reporting Key: NIFI-5417 URL: https://issues.apache.org/jira/browse/NIFI-5417 Project: Apache NiFi Issue Type: Improvement Components: Extensions Affects Versions: 1.7.0 Reporter: Taras Vuyiv Hi team, We use _SiteToSiteStatusReportingTask_ for our monitoring system and realized that we seriously need the run status of the processors and input/output ports, which is not sent by reporting task in the status JSON. During an investigation turned out that some of the available values in the status classes of _org.apache.nifi.controller.status_ package __ are not included in the resulting JSON of the _SiteToSiteStatusReportingTask_. It's impossible to extend the reporting task in a non-hack way, so please add the missing status properties: +Processor+ * runStatus * executionNode * terminatedThreadCount * counters +Port+ * transmitting * runStatus +Connection+ * backPressureDataSizeThreshold +Remote Process Group+ * transmissionStatus * uri +Process Group+ * versionedFlowState * terminatedThreadCount * queuedContentSize Thanks! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5416) Move long running tests to special profile or integration tests
[
https://issues.apache.org/jira/browse/NIFI-5416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541283#comment-16541283
]
Andy LoPresto commented on NIFI-5416:
-
The above timings were from a build where my {{/etc/hosts}} file was causing
some issues and many timeouts were being hit. After resolving that issue, here
is every test with more than 10 seconds duration:
{code}
$ mvn clean install -Pcontrib-check -Pinclude-grpc 2>&1 | tee
../maven_test_output.txt
$ grep 'Time elapsed' ../maven_test_output.txt | cut -d ' ' -f 13- | sort -rn >
../maven_test_timings.txt
95.057 s - in org.apache.nifi.provenance.TestPersistentProvenanceRepository
58.233 s - in org.apache.nifi.processors.standard.TestListFile
42.849 s - in org.apache.nifi.controller.TestFlowController
32.951 s - in org.apache.nifi.processors.standard.TestInvokeHttpTwoWaySSL
32.646 s - in org.apache.nifi.processors.standard.TestInvokeHttpSSL
21.957 s - in org.apache.nifi.controller.scheduling.TestProcessorLifecycle
20.66 s - in org.apache.nifi.controller.repository.TestFileSystemRepository
18.409 s - in org.apache.nifi.processors.standard.TestTailFile
13.03 s - in
org.apache.nifi.controller.state.providers.zookeeper.TestZooKeeperStateProvider
12.793 s - in org.apache.nifi.toolkit.tls.standalone.TlsToolkitStandaloneTest
12.203 s - in org.apache.nifi.remote.client.http.TestHttpClient
11.106 s - in org.apache.nifi.toolkit.zkmigrator.ZooKeeperMigratorTest
10.719 s - in
org.apache.nifi.processors.aws.wag.TestInvokeInvokeAmazonGatewayApiWithControllerService
10.419 s - in org.apache.nifi.processors.standard.TestInvokeHTTP
10.318 s - in
org.apache.nifi.processors.aws.wag.TestInvokeInvokeAmazonGatewayApiWithCredFile
10.278 s - in
org.apache.nifi.processors.aws.wag.TestInvokeInvokeAmazonGatewayApiWithStaticAuth
{code}
> Move long running tests to special profile or integration tests
> ---
>
> Key: NIFI-5416
> URL: https://issues.apache.org/jira/browse/NIFI-5416
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Tools and Build
>Affects Versions: 1.7.0
>Reporter: Andy LoPresto
>Priority: Major
> Labels: build, integration-test, test
>
> Running the build with tests is a very long process; sometimes a multithread
> build can be problematic, and running the full build is over half an hour on
> commodity hardware.
> Some unit tests wait an inordinate amount of time and do not need to be run
> as part of the normal build. I've started to identify some of these tests
> just via repeated runs. Either they should be moved to the integration tests
> profile, or a new profile / JUnit category to be run only when necessary.
> A quick collection of significantly longer tests shows the possibility of >6
> minutes shaved off the build.
> * TestTailFile 18s
> * TestPostHTTP 35s
> * TestListFile 58s
> * TestListenUDP 39s
> * TestInvokeHttpTwoWaySSL 67s
> * TestInvokeHttpSSL 67s
> * TestInvokeHttp 10s
> * TestGetHTTP 35s
> * TestRELPSocketChannelHandler 36s
> * DBCPServiceTest 37s
> =>400 seconds = 6+ minutes saved
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFI-5416) Move long running tests to special profile or integration tests
Andy LoPresto created NIFI-5416: --- Summary: Move long running tests to special profile or integration tests Key: NIFI-5416 URL: https://issues.apache.org/jira/browse/NIFI-5416 Project: Apache NiFi Issue Type: Improvement Components: Tools and Build Affects Versions: 1.7.0 Reporter: Andy LoPresto Running the build with tests is a very long process; sometimes a multithread build can be problematic, and running the full build is over half an hour on commodity hardware. Some unit tests wait an inordinate amount of time and do not need to be run as part of the normal build. I've started to identify some of these tests just via repeated runs. Either they should be moved to the integration tests profile, or a new profile / JUnit category to be run only when necessary. A quick collection of significantly longer tests shows the possibility of >6 minutes shaved off the build. * TestTailFile 18s * TestPostHTTP 35s * TestListFile 58s * TestListenUDP 39s * TestInvokeHttpTwoWaySSL 67s * TestInvokeHttpSSL 67s * TestInvokeHttp 10s * TestGetHTTP 35s * TestRELPSocketChannelHandler 36s * DBCPServiceTest 37s =>400 seconds = 6+ minutes saved -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2881: NIFI-5413 Added additional documentation for PutMongoRecor...
Github user lfrancke commented on the issue: https://github.com/apache/nifi/pull/2881 +1 looks good to me (not a committer) ---
[jira] [Commented] (NIFI-5413) Write additional documentation for PutMongoRecord
[ https://issues.apache.org/jira/browse/NIFI-5413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541230#comment-16541230 ] ASF GitHub Bot commented on NIFI-5413: -- Github user lfrancke commented on the issue: https://github.com/apache/nifi/pull/2881 +1 looks good to me (not a committer) > Write additional documentation for PutMongoRecord > - > > Key: NIFI-5413 > URL: https://issues.apache.org/jira/browse/NIFI-5413 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2882: Added Apache Pulsar Processors
Github user lfrancke commented on the issue: https://github.com/apache/nifi/pull/2882 The pull request title and the commit is missing the JIRA number, could you add that? > Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. ---
[jira] [Commented] (NIFI-5415) Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy
[
https://issues.apache.org/jira/browse/NIFI-5415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541218#comment-16541218
]
ASF GitHub Bot commented on NIFI-5415:
--
Github user alopresto commented on the issue:
https://github.com/apache/nifi/pull/2883
@joewitt please review -- not sure why the test was failing originally but
for some reason it still ran for me when performing just unit tests.
> Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy
> --
>
> Key: NIFI-5415
> URL: https://issues.apache.org/jira/browse/NIFI-5415
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 1.7.0
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Blocker
> Labels: groovy, integration-test, maven, syslog, test
> Fix For: 1.7.1
>
>
> [~joewitt] made an effort to move inconsistent and network/timing-dependent
> unit tests to integration tests via NIFI-4659, NIFI-4660, NIFI-4662, and
> NIFI-4664.
> This file was renamed (from {{ListenSyslogGroovyTest.groovy}} to
> {{ITListenSyslogGroovy.groovy}}) but the class name inside was not changed. I
> believe the maven filter is on the class name, not the file name, because
> this test still runs in normal builds. It breaks inconsistently on my
> machine, so I renamed the class to ensure it is not run as part of the usual
> unit test battery. This is required for 1.7.1 to be released.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi issue #2883: NIFI-5415 Renamed ListenSyslogGroovyTest to ITListenSyslog...
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2883 @joewitt please review -- not sure why the test was failing originally but for some reason it still ran for me when performing just unit tests. ---
[jira] [Commented] (NIFI-5415) Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy
[
https://issues.apache.org/jira/browse/NIFI-5415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541217#comment-16541217
]
ASF GitHub Bot commented on NIFI-5415:
--
Github user alopresto commented on the issue:
https://github.com/apache/nifi/pull/2883
I am opening a PR for this for actual review before it goes into `master`,
but I included it in the 1.7.1 release
[a618ea5](https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=a618ea55975094087023d02e27a9bd859f671702)
because it was blocking the build for me.
> Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy
> --
>
> Key: NIFI-5415
> URL: https://issues.apache.org/jira/browse/NIFI-5415
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 1.7.0
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Blocker
> Labels: groovy, integration-test, maven, syslog, test
> Fix For: 1.7.1
>
>
> [~joewitt] made an effort to move inconsistent and network/timing-dependent
> unit tests to integration tests via NIFI-4659, NIFI-4660, NIFI-4662, and
> NIFI-4664.
> This file was renamed (from {{ListenSyslogGroovyTest.groovy}} to
> {{ITListenSyslogGroovy.groovy}}) but the class name inside was not changed. I
> believe the maven filter is on the class name, not the file name, because
> this test still runs in normal builds. It breaks inconsistently on my
> machine, so I renamed the class to ensure it is not run as part of the usual
> unit test battery. This is required for 1.7.1 to be released.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi issue #2883: NIFI-5415 Renamed ListenSyslogGroovyTest to ITListenSyslog...
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2883 I am opening a PR for this for actual review before it goes into `master`, but I included it in the 1.7.1 release [a618ea5](https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=a618ea55975094087023d02e27a9bd859f671702) because it was blocking the build for me. ---
[jira] [Commented] (NIFI-5415) Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy
[
https://issues.apache.org/jira/browse/NIFI-5415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541215#comment-16541215
]
ASF GitHub Bot commented on NIFI-5415:
--
GitHub user alopresto opened a pull request:
https://github.com/apache/nifi/pull/2883
NIFI-5415 Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy.
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with NIFI- where is the JIRA number
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [x] Is your initial contribution a single, squashed commit?
### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file, including the main
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in
which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/alopresto/nifi NIFI-5415
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/2883.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2883
commit 57ac2871f4b848842f5d867a6f13a81473a37896
Author: Andy LoPresto
Date: 2018-07-12T06:32:23Z
NIFI-5415 Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy.
> Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy
> --
>
> Key: NIFI-5415
> URL: https://issues.apache.org/jira/browse/NIFI-5415
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 1.7.0
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Blocker
> Labels: groovy, integration-test, maven, syslog, test
> Fix For: 1.7.1
>
>
> [~joewitt] made an effort to move inconsistent and network/timing-dependent
> unit tests to integration tests via NIFI-4659, NIFI-4660, NIFI-4662, and
> NIFI-4664.
> This file was renamed (from {{ListenSyslogGroovyTest.groovy}} to
> {{ITListenSyslogGroovy.groovy}}) but the class name inside was not changed. I
> believe the maven filter is on the class name, not the file name, because
> this test still runs in normal builds. It breaks inconsistently on my
> machine, so I renamed the class to ensure it is not run as part of the usual
> unit test battery. This is required for 1.7.1 to be released.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2883: NIFI-5415 Renamed ListenSyslogGroovyTest to ITListe...
GitHub user alopresto opened a pull request: https://github.com/apache/nifi/pull/2883 NIFI-5415 Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/alopresto/nifi NIFI-5415 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2883.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2883 commit 57ac2871f4b848842f5d867a6f13a81473a37896 Author: Andy LoPresto Date: 2018-07-12T06:32:23Z NIFI-5415 Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy. ---
[jira] [Updated] (NIFI-5415) Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy
[
https://issues.apache.org/jira/browse/NIFI-5415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy LoPresto updated NIFI-5415:
Status: Patch Available (was: In Progress)
> Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy
> --
>
> Key: NIFI-5415
> URL: https://issues.apache.org/jira/browse/NIFI-5415
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 1.7.0
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Blocker
> Labels: groovy, integration-test, maven, syslog, test
> Fix For: 1.7.1
>
>
> [~joewitt] made an effort to move inconsistent and network/timing-dependent
> unit tests to integration tests via NIFI-4659, NIFI-4660, NIFI-4662, and
> NIFI-4664.
> This file was renamed (from {{ListenSyslogGroovyTest.groovy}} to
> {{ITListenSyslogGroovy.groovy}}) but the class name inside was not changed. I
> believe the maven filter is on the class name, not the file name, because
> this test still runs in normal builds. It breaks inconsistently on my
> machine, so I renamed the class to ensure it is not run as part of the usual
> unit test battery. This is required for 1.7.1 to be released.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5415) Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy
[
https://issues.apache.org/jira/browse/NIFI-5415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16541213#comment-16541213
]
ASF subversion and git services commented on NIFI-5415:
---
Commit a618ea55975094087023d02e27a9bd859f671702 in nifi's branch
refs/heads/support/nifi-1.7.x from [~alopresto]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=a618ea5 ]
NIFI-5415 Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy.
> Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy
> --
>
> Key: NIFI-5415
> URL: https://issues.apache.org/jira/browse/NIFI-5415
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 1.7.0
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Blocker
> Labels: groovy, integration-test, maven, syslog, test
> Fix For: 1.7.1
>
>
> [~joewitt] made an effort to move inconsistent and network/timing-dependent
> unit tests to integration tests via NIFI-4659, NIFI-4660, NIFI-4662, and
> NIFI-4664.
> This file was renamed (from {{ListenSyslogGroovyTest.groovy}} to
> {{ITListenSyslogGroovy.groovy}}) but the class name inside was not changed. I
> believe the maven filter is on the class name, not the file name, because
> this test still runs in normal builds. It breaks inconsistently on my
> machine, so I renamed the class to ensure it is not run as part of the usual
> unit test battery. This is required for 1.7.1 to be released.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFI-5415) Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy
Andy LoPresto created NIFI-5415:
---
Summary: Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy
Key: NIFI-5415
URL: https://issues.apache.org/jira/browse/NIFI-5415
Project: Apache NiFi
Issue Type: Bug
Components: Tools and Build
Affects Versions: 1.7.0
Reporter: Andy LoPresto
Assignee: Andy LoPresto
Fix For: 1.7.1
[~joewitt] made an effort to move inconsistent and network/timing-dependent
unit tests to integration tests via NIFI-4659, NIFI-4660, NIFI-4662, and
NIFI-4664.
This file was renamed (from {{ListenSyslogGroovyTest.groovy}} to
{{ITListenSyslogGroovy.groovy}}) but the class name inside was not changed. I
believe the maven filter is on the class name, not the file name, because this
test still runs in normal builds. It breaks inconsistently on my machine, so I
renamed the class to ensure it is not run as part of the usual unit test
battery. This is required for 1.7.1 to be released.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
