[jira] [Created] (NIFI-5502) Support custom UMask for creating folders vs files
Jeremy Dyer created NIFI-5502: - Summary: Support custom UMask for creating folders vs files Key: NIFI-5502 URL: https://issues.apache.org/jira/browse/NIFI-5502 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Jeremy Dyer Assignee: Jeremy Dyer Fix For: 1.8.0 It would be convenient to have a way to specify a custom umask for either creating file or creating folders. In this manner it would be easier for users to handle creating file and folders that have different permissions -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4434) ListHDFS applies File Filter also to subdirectory names in recursive search
[ https://issues.apache.org/jira/browse/NIFI-4434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Storck updated NIFI-4434: -- Fix Version/s: 1.8.0 > ListHDFS applies File Filter also to subdirectory names in recursive search > --- > > Key: NIFI-4434 > URL: https://issues.apache.org/jira/browse/NIFI-4434 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.3.0 >Reporter: Holger Frydrych >Assignee: Jeff Storck >Priority: Major > Fix For: 1.8.0 > > > The File Filter regex configured in the ListHDFS processor is applied not > just to files found, but also to subdirectories. > If you try to set up a recursive search to list e.g. all csv files in a > directory hierarchy via a regex like ".*\.csv", it will only pick up csv > files in the base directory, not in any subdirectory. This is because > subdirectories don't typically match that regex pattern. > To fix this, either subdirectories should not be matched against the file > filter, or the file filter should be applied to the full path of all files > (relative to the base directory). The GetHDFS processor offers both options > via a switch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFIREG-192) Create/Add REGISTRY_START event that fires when the NiFi Registry initially starts
Jeremy Dyer created NIFIREG-192: --- Summary: Create/Add REGISTRY_START event that fires when the NiFi Registry initially starts Key: NIFIREG-192 URL: https://issues.apache.org/jira/browse/NIFIREG-192 Project: NiFi Registry Issue Type: New Feature Reporter: Jeremy Dyer Assignee: Jeremy Dyer Fix For: 0.3.0 Having an event that fires when the NiFi Registry starts would provide a great place for bootstrapping the registry with any sort of setup, importing, repairing type tasks that need to occur before the registry starts serving clients. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (NIFI-4434) ListHDFS applies File Filter also to subdirectory names in recursive search
[ https://issues.apache.org/jira/browse/NIFI-4434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Storck resolved NIFI-4434. --- Resolution: Fixed > ListHDFS applies File Filter also to subdirectory names in recursive search > --- > > Key: NIFI-4434 > URL: https://issues.apache.org/jira/browse/NIFI-4434 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.3.0 >Reporter: Holger Frydrych >Assignee: Jeff Storck >Priority: Major > > The File Filter regex configured in the ListHDFS processor is applied not > just to files found, but also to subdirectories. > If you try to set up a recursive search to list e.g. all csv files in a > directory hierarchy via a regex like ".*\.csv", it will only pick up csv > files in the base directory, not in any subdirectory. This is because > subdirectories don't typically match that regex pattern. > To fix this, either subdirectories should not be matched against the file > filter, or the file filter should be applied to the full path of all files > (relative to the base directory). The GetHDFS processor offers both options > via a switch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFIREG-190) Support for Event Whitelisting in the Registry Event Hooks
[ https://issues.apache.org/jira/browse/NIFIREG-190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573572#comment-16573572 ] ASF GitHub Bot commented on NIFIREG-190: GitHub user jdye64 opened a pull request: https://github.com/apache/nifi-registry/pull/133 NIFIREG-190 You can merge this pull request into a Git repository by running: $ git pull https://github.com/jdye64/nifi-registry NIFIREG-190 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-registry/pull/133.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #133 commit 6821dc10bd633fec324c2ca82616269dbc8cda0b Author: Jeremy Dyer Date: 2018-08-08T17:41:42Z NIFIREG-190 Add support for whitelist filtering Event Hook providers commit 673734cf0a2298a68c6d7921fe8b626a6706567f Author: Jeremy Dyer Date: 2018-08-08T17:43:06Z Fixed git issue where I had accidentally removed tracking for NifiRegistryApiApplication.java commit a527fbaf1ca31a9e03aa4093b9ca639057343cc0 Author: Jeremy Dyer Date: 2018-08-08T17:44:11Z Removed unused imports > Support for Event Whitelisting in the Registry Event Hooks > -- > > Key: NIFIREG-190 > URL: https://issues.apache.org/jira/browse/NIFIREG-190 > Project: NiFi Registry > Issue Type: New Feature >Affects Versions: 0.2.0 >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer >Priority: Major > Fix For: 0.3.0 > > > Today when an event hook is configured it will be invoked for all of the NiFi > Registry events. While a user can parse the arguments in the script and > manually write scripts which ignore certain events it makes more sense to > provide this event whitelisting the the registry itself. > I propose adding a new property called something like "Event Whitelist" to > the current configuration logic. If this property is not present things > should continue to operate just as they do now, AKA the script is sent all of > the events, if the property is specified it should contain a comma delimited > list of events that the hook provider should be triggered for. > This will be extremely useful when providers that do not provider any sort of > filtering logic like the script hook provider come along. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-registry pull request #133: NIFIREG-190
GitHub user jdye64 opened a pull request: https://github.com/apache/nifi-registry/pull/133 NIFIREG-190 You can merge this pull request into a Git repository by running: $ git pull https://github.com/jdye64/nifi-registry NIFIREG-190 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-registry/pull/133.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #133 commit 6821dc10bd633fec324c2ca82616269dbc8cda0b Author: Jeremy Dyer Date: 2018-08-08T17:41:42Z NIFIREG-190 Add support for whitelist filtering Event Hook providers commit 673734cf0a2298a68c6d7921fe8b626a6706567f Author: Jeremy Dyer Date: 2018-08-08T17:43:06Z Fixed git issue where I had accidentally removed tracking for NifiRegistryApiApplication.java commit a527fbaf1ca31a9e03aa4093b9ca639057343cc0 Author: Jeremy Dyer Date: 2018-08-08T17:44:11Z Removed unused imports ---
[jira] [Commented] (NIFI-5489) Support Attribute Expressions with AMQP Processors
[ https://issues.apache.org/jira/browse/NIFI-5489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573542#comment-16573542 ] ASF GitHub Bot commented on NIFI-5489: -- Github user danieljimenez commented on the issue: https://github.com/apache/nifi/pull/2936 FWIW I am currently using a SNAPSHOT from 01782bf on a real RabbitMQ server. > Support Attribute Expressions with AMQP Processors > -- > > Key: NIFI-5489 > URL: https://issues.apache.org/jira/browse/NIFI-5489 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.7.1 >Reporter: Daniel >Priority: Major > > Particularly the fields: host, virtualhost and username. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2936: NIFI-5489: Add expression language support to AMQP process...
Github user danieljimenez commented on the issue: https://github.com/apache/nifi/pull/2936 FWIW I am currently using a SNAPSHOT from 01782bf on a real RabbitMQ server. ---
[jira] [Commented] (NIFI-5489) Support Attribute Expressions with AMQP Processors
[ https://issues.apache.org/jira/browse/NIFI-5489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573537#comment-16573537 ] ASF GitHub Bot commented on NIFI-5489: -- Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2936 +1 from me, though I didn't test this against a real RabbitMQ server. @zenfenan did you want to take another look? > Support Attribute Expressions with AMQP Processors > -- > > Key: NIFI-5489 > URL: https://issues.apache.org/jira/browse/NIFI-5489 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.7.1 >Reporter: Daniel >Priority: Major > > Particularly the fields: host, virtualhost and username. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2936: NIFI-5489: Add expression language support to AMQP process...
Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/2936 +1 from me, though I didn't test this against a real RabbitMQ server. @zenfenan did you want to take another look? ---
[jira] [Commented] (NIFI-4723) PublishAMQP 1.4.0 support Expression Language
[ https://issues.apache.org/jira/browse/NIFI-4723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573510#comment-16573510 ] Daniel commented on NIFI-4723: -- See https://github.com/apache/nifi/pull/2936 > PublishAMQP 1.4.0 support Expression Language > -- > > Key: NIFI-4723 > URL: https://issues.apache.org/jira/browse/NIFI-4723 > Project: Apache NiFi > Issue Type: Improvement > Components: Examples, Extensions >Reporter: bat-hen >Priority: Major > > Hi, > there is no support to Expression Language in PublishAMQP? for: > Host Name > Port > User Name > Password > we want to do deployment with template and we cant use Expression Language in > rabbitmq.:| -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5489) Support Attribute Expressions with AMQP Processors
[ https://issues.apache.org/jira/browse/NIFI-5489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573505#comment-16573505 ] ASF GitHub Bot commented on NIFI-5489: -- Github user danieljimenez commented on the issue: https://github.com/apache/nifi/pull/2936 @mosermw Port added sir. :) > Support Attribute Expressions with AMQP Processors > -- > > Key: NIFI-5489 > URL: https://issues.apache.org/jira/browse/NIFI-5489 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.7.1 >Reporter: Daniel >Priority: Major > > Particularly the fields: host, virtualhost and username. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2936: NIFI-5489: Add expression language support to AMQP process...
Github user danieljimenez commented on the issue: https://github.com/apache/nifi/pull/2936 @mosermw Port added sir. :) ---
[jira] [Commented] (NIFI-4434) ListHDFS applies File Filter also to subdirectory names in recursive search
[ https://issues.apache.org/jira/browse/NIFI-4434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573476#comment-16573476 ] ASF GitHub Bot commented on NIFI-4434: -- Github user zenfenan commented on the issue: https://github.com/apache/nifi/pull/2937 +1 Thanks @jtstorck. Merged to the master. Thanks @ottobackwards for reviewing. > ListHDFS applies File Filter also to subdirectory names in recursive search > --- > > Key: NIFI-4434 > URL: https://issues.apache.org/jira/browse/NIFI-4434 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.3.0 >Reporter: Holger Frydrych >Assignee: Jeff Storck >Priority: Major > > The File Filter regex configured in the ListHDFS processor is applied not > just to files found, but also to subdirectories. > If you try to set up a recursive search to list e.g. all csv files in a > directory hierarchy via a regex like ".*\.csv", it will only pick up csv > files in the base directory, not in any subdirectory. This is because > subdirectories don't typically match that regex pattern. > To fix this, either subdirectories should not be matched against the file > filter, or the file filter should be applied to the full path of all files > (relative to the base directory). The GetHDFS processor offers both options > via a switch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2937: NIFI-4434 Fixed recursive listing with a custom regex filt...
Github user zenfenan commented on the issue: https://github.com/apache/nifi/pull/2937 +1 Thanks @jtstorck. Merged to the master. Thanks @ottobackwards for reviewing. ---
[jira] [Commented] (NIFI-4434) ListHDFS applies File Filter also to subdirectory names in recursive search
[ https://issues.apache.org/jira/browse/NIFI-4434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573471#comment-16573471 ] ASF GitHub Bot commented on NIFI-4434: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2937 > ListHDFS applies File Filter also to subdirectory names in recursive search > --- > > Key: NIFI-4434 > URL: https://issues.apache.org/jira/browse/NIFI-4434 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.3.0 >Reporter: Holger Frydrych >Assignee: Jeff Storck >Priority: Major > > The File Filter regex configured in the ListHDFS processor is applied not > just to files found, but also to subdirectories. > If you try to set up a recursive search to list e.g. all csv files in a > directory hierarchy via a regex like ".*\.csv", it will only pick up csv > files in the base directory, not in any subdirectory. This is because > subdirectories don't typically match that regex pattern. > To fix this, either subdirectories should not be matched against the file > filter, or the file filter should be applied to the full path of all files > (relative to the base directory). The GetHDFS processor offers both options > via a switch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2937: NIFI-4434 Fixed recursive listing with a custom reg...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2937 ---
[jira] [Commented] (NIFI-4434) ListHDFS applies File Filter also to subdirectory names in recursive search
[ https://issues.apache.org/jira/browse/NIFI-4434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573470#comment-16573470 ] ASF subversion and git services commented on NIFI-4434: --- Commit 451084e11f8c3dc2a0c68ffb739dff58de479cff in nifi's branch refs/heads/master from [~jtstorck] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=451084e ] NIFI-4434 Fixed recursive listing with a custom regex filter. Filter modes are now supported to perform listings based on directory and file names, file-names only, and full path. This closes #2937 Signed-off-by: zenfenan > ListHDFS applies File Filter also to subdirectory names in recursive search > --- > > Key: NIFI-4434 > URL: https://issues.apache.org/jira/browse/NIFI-4434 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.3.0 >Reporter: Holger Frydrych >Assignee: Jeff Storck >Priority: Major > > The File Filter regex configured in the ListHDFS processor is applied not > just to files found, but also to subdirectories. > If you try to set up a recursive search to list e.g. all csv files in a > directory hierarchy via a regex like ".*\.csv", it will only pick up csv > files in the base directory, not in any subdirectory. This is because > subdirectories don't typically match that regex pattern. > To fix this, either subdirectories should not be matched against the file > filter, or the file filter should be applied to the full path of all files > (relative to the base directory). The GetHDFS processor offers both options > via a switch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-5501) MySQL connection cleanup thread leak
[
https://issues.apache.org/jira/browse/NIFI-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tanapol Nearunchorn updated NIFI-5501:
--
Description:
There are thousands of "Abandoned connection cleanup thread" left running that
cause memory leak in NiFi.
I got a thread dump example here (full thread dump also attached):
{code:java}
"Abandoned connection cleanup thread" #18371 daemon prio=5 os_prio=0
tid=0x7f3b840e7800 nid=0x76a3 in Object.wait() [0x7f3b24ebb000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
- locked <0x000348a12628> (a java.lang.ref.ReferenceQueue$Lock)
at
com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:64)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Locked ownable synchronizers:
- <0x000348a12648> (a java.util.concurrent.ThreadPoolExecutor$Worker)
{code}
I'm not sure where these threads come from which component because in my flow,
I used CaptureChangeMySQL processor and DBCPConnectionPool controller service.
As I also found related problem here: [mysql Bug
#69526|https://bugs.mysql.com/bug.php?id=69526] but it quite a bit old.
was:
There are thousands of "Abandoned connection cleanup thread" left running that
cause memory leak in NiFi.
I got a thread dump example here (full thread dump also attached):
{code:java}
"Abandoned connection cleanup thread" #18371 daemon prio=5 os_prio=0
tid=0x7f3b840e7800 nid=0x76a3 in Object.wait() [0x7f3b24ebb000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
- locked <0x000348a12628> (a java.lang.ref.ReferenceQueue$Lock)
at
com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:64)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Locked ownable synchronizers:
- <0x000348a12648> (a java.util.concurrent.ThreadPoolExecutor$Worker)
{code}
I'm not sure where these threads come from which component because in my flow,
I used CaptureChangeMySQL processor and DBCPConnectionPool controller service.
As I also found related problem here: [mysql Bug
#69526|[https://bugs.mysql.com/bug.php?id=69526]|https://bugs.mysql.com/bug.php?id=69526]]]
but it quite a bit old.
> MySQL connection cleanup thread leak
>
>
> Key: NIFI-5501
> URL: https://issues.apache.org/jira/browse/NIFI-5501
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.7.1
> Environment: - Ubuntu 18.04 LTS
> - mysql-connector-java-5.1.46
> - Java 1.8.0_171
>Reporter: Tanapol Nearunchorn
>Priority: Major
> Attachments: nifi-threaddump.txt
>
>
> There are thousands of "Abandoned connection cleanup thread" left running
> that cause memory leak in NiFi.
> I got a thread dump example here (full thread dump also attached):
> {code:java}
> "Abandoned connection cleanup thread" #18371 daemon prio=5 os_prio=0
> tid=0x7f3b840e7800 nid=0x76a3 in Object.wait() [0x7f3b24ebb000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
> - locked <0x000348a12628> (a java.lang.ref.ReferenceQueue$Lock)
> at
> com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:64)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Locked ownable synchronizers:
> - <0x000348a12648> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> {code}
> I'm not sure where these threads come from which component because in my
> flow, I used CaptureChangeMySQL processor and DBCPConnectionPool controller
> service.
> As I also found related problem here: [mysql Bug
> #69526|https://bugs.mysql.com/bug.php?id=69526] but it quite a bit old.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5501) MySQL connection cleanup thread leak
[
https://issues.apache.org/jira/browse/NIFI-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tanapol Nearunchorn updated NIFI-5501:
--
Description:
There are thousands of "Abandoned connection cleanup thread" left running that
cause memory leak in NiFi.
I got a thread dump example here (full thread dump also attached):
{code:java}
"Abandoned connection cleanup thread" #18371 daemon prio=5 os_prio=0
tid=0x7f3b840e7800 nid=0x76a3 in Object.wait() [0x7f3b24ebb000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
- locked <0x000348a12628> (a java.lang.ref.ReferenceQueue$Lock)
at
com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:64)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Locked ownable synchronizers:
- <0x000348a12648> (a java.util.concurrent.ThreadPoolExecutor$Worker)
{code}
I'm not sure where these threads come from which component because in my flow,
I used CaptureChangeMySQL processor and DBCPConnectionPool controller service.
As I also found related problem here: [mysql Bug
#69526|[https://bugs.mysql.com/bug.php?id=69526]|https://bugs.mysql.com/bug.php?id=69526]]]
but it quite a bit old.
was:
There are thousands of "Abandoned connection cleanup thread" left running that
cause memory leak in NiFi.
I got a thread dump example here (full thread dump also attached):
{code:java}
"Abandoned connection cleanup thread" #18371 daemon prio=5 os_prio=0
tid=0x7f3b840e7800 nid=0x76a3 in Object.wait() [0x7f3b24ebb000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
- locked <0x000348a12628> (a java.lang.ref.ReferenceQueue$Lock)
at
com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:64)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Locked ownable synchronizers:
- <0x000348a12648> (a java.util.concurrent.ThreadPoolExecutor$Worker)
{code}
I'm not sure where these threads come from which component because in my flow,
I used CaptureChangeMySQL processor and DBCPConnectionPool controller service.
As I also found related problem here:
[#https://bugs.mysql.com/bug.php?id=69526]
> MySQL connection cleanup thread leak
>
>
> Key: NIFI-5501
> URL: https://issues.apache.org/jira/browse/NIFI-5501
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.7.1
> Environment: - Ubuntu 18.04 LTS
> - mysql-connector-java-5.1.46
> - Java 1.8.0_171
>Reporter: Tanapol Nearunchorn
>Priority: Major
> Attachments: nifi-threaddump.txt
>
>
> There are thousands of "Abandoned connection cleanup thread" left running
> that cause memory leak in NiFi.
> I got a thread dump example here (full thread dump also attached):
> {code:java}
> "Abandoned connection cleanup thread" #18371 daemon prio=5 os_prio=0
> tid=0x7f3b840e7800 nid=0x76a3 in Object.wait() [0x7f3b24ebb000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
> - locked <0x000348a12628> (a java.lang.ref.ReferenceQueue$Lock)
> at
> com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:64)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Locked ownable synchronizers:
> - <0x000348a12648> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> {code}
> I'm not sure where these threads come from which component because in my
> flow, I used CaptureChangeMySQL processor and DBCPConnectionPool controller
> service.
> As I also found related problem here: [mysql Bug
> #69526|[https://bugs.mysql.com/bug.php?id=69526]|https://bugs.mysql.com/bug.php?id=69526]]]
> but it quite a bit old.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5501) MySQL connection cleanup thread leak
[
https://issues.apache.org/jira/browse/NIFI-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tanapol Nearunchorn updated NIFI-5501:
--
Attachment: nifi-threaddump.txt
> MySQL connection cleanup thread leak
>
>
> Key: NIFI-5501
> URL: https://issues.apache.org/jira/browse/NIFI-5501
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.7.1
> Environment: - Ubuntu 18.04 LTS
> - mysql-connector-java-5.1.46
> - Java 1.8.0_171
>Reporter: Tanapol Nearunchorn
>Priority: Major
> Attachments: nifi-threaddump.txt
>
>
> There are thousands of "Abandoned connection cleanup thread" left running
> that cause memory leak in NiFi.
> I got a thread dump example here (full thread dump also attached):
> {code:java}
> "Abandoned connection cleanup thread" #18371 daemon prio=5 os_prio=0
> tid=0x7f3b840e7800 nid=0x76a3 in Object.wait() [0x7f3b24ebb000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
> - locked <0x000348a12628> (a java.lang.ref.ReferenceQueue$Lock)
> at
> com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:64)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Locked ownable synchronizers:
> - <0x000348a12648> (a java.util.concurrent.ThreadPoolExecutor$Worker)
> {code}
> I'm not sure where these threads come from which component because in my
> flow, I used CaptureChangeMySQL processor and DBCPConnectionPool controller
> service.
> As I also found related problem here:
> [#https://bugs.mysql.com/bug.php?id=69526]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFI-5501) MySQL connection cleanup thread leak
Tanapol Nearunchorn created NIFI-5501:
-
Summary: MySQL connection cleanup thread leak
Key: NIFI-5501
URL: https://issues.apache.org/jira/browse/NIFI-5501
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 1.7.1
Environment: - Ubuntu 18.04 LTS
- mysql-connector-java-5.1.46
- Java 1.8.0_171
Reporter: Tanapol Nearunchorn
Attachments: nifi-threaddump.txt
There are thousands of "Abandoned connection cleanup thread" left running that
cause memory leak in NiFi.
I got a thread dump example here (full thread dump also attached):
{code:java}
"Abandoned connection cleanup thread" #18371 daemon prio=5 os_prio=0
tid=0x7f3b840e7800 nid=0x76a3 in Object.wait() [0x7f3b24ebb000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
- locked <0x000348a12628> (a java.lang.ref.ReferenceQueue$Lock)
at
com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:64)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Locked ownable synchronizers:
- <0x000348a12648> (a java.util.concurrent.ThreadPoolExecutor$Worker)
{code}
I'm not sure where these threads come from which component because in my flow,
I used CaptureChangeMySQL processor and DBCPConnectionPool controller service.
As I also found related problem here:
[#https://bugs.mysql.com/bug.php?id=69526]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2942: NIFI-5500
GitHub user wietze opened a pull request: https://github.com/apache/nifi/pull/2942 NIFI-5500 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [x] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [x] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/wietze/nifi NIFI-5500 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2942.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2942 commit 25e2ee280e0b856b73de8cd0c8f039521a883f32 Author: Wietze Date: 2018-08-08T16:17:47Z Adding support for array fields when outputting to attributes when using QueryElasticsearchHttp ---
[jira] [Commented] (NIFIREG-191) Make formatting consistent in Registry docs
[
https://issues.apache.org/jira/browse/NIFIREG-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573407#comment-16573407
]
Andrew Lim commented on NIFIREG-191:
Adding comments for additional issues to be corrected.
The git command links within the Registry Admin Guide don't render:
* Git init command [https://git-scm.com/docs/git-init]
* Git clone command [https://git-scm.com/docs/git-clone]
Should make them open in a separate tab/window.
> Make formatting consistent in Registry docs
> ---
>
> Key: NIFIREG-191
> URL: https://issues.apache.org/jira/browse/NIFIREG-191
> Project: NiFi Registry
> Issue Type: Improvement
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Minor
>
> Similar to what was done in NIFI-5469, need to make the formatting of
> properties, directories and files consistent.
> Will use same convention applied to NiFi docs:
> filenames in _italics_
> directory paths/directories in {{code formatting}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (NIFI-5500) Add array support to QueryElasticsearchHttp
[ https://issues.apache.org/jira/browse/NIFI-5500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sivaprasanna Sethuraman reassigned NIFI-5500: - Assignee: Sivaprasanna Sethuraman > Add array support to QueryElasticsearchHttp > --- > > Key: NIFI-5500 > URL: https://issues.apache.org/jira/browse/NIFI-5500 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.7.1 >Reporter: Wietze B >Assignee: Sivaprasanna Sethuraman >Priority: Minor > > When using the QueryElasticsearchHttp processor (with output=Attributes) to > query a document that contains a field with an Array rather than a String or > Integer, the resulting attribute for that field will be an empty string. > This is due to the fact that the QueryElasticsearchHttp component doesn't > handle array fields correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-5500) Add array support to QueryElasticsearchHttp
Wietze B created NIFI-5500: -- Summary: Add array support to QueryElasticsearchHttp Key: NIFI-5500 URL: https://issues.apache.org/jira/browse/NIFI-5500 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.7.1 Reporter: Wietze B When using the QueryElasticsearchHttp processor (with output=Attributes) to query a document that contains a field with an Array rather than a String or Integer, the resulting attribute for that field will be an empty string. This is due to the fact that the QueryElasticsearchHttp component doesn't handle array fields correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5499) Upgrade AngularJS to v1.7.2
[ https://issues.apache.org/jira/browse/NIFI-5499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573339#comment-16573339 ] ASF GitHub Bot commented on NIFI-5499: -- Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2941 Will review... > Upgrade AngularJS to v1.7.2 > --- > > Key: NIFI-5499 > URL: https://issues.apache.org/jira/browse/NIFI-5499 > Project: Apache NiFi > Issue Type: Bug >Reporter: Scott Aslan >Assignee: Scott Aslan >Priority: Major > Fix For: 1.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2941: [NIFI-5499] upgrade AngularJS to v1.7.2
Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2941 Will review... ---
[jira] [Commented] (NIFI-5499) Upgrade AngularJS to v1.7.2
[ https://issues.apache.org/jira/browse/NIFI-5499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573331#comment-16573331 ] ASF GitHub Bot commented on NIFI-5499: -- GitHub user scottyaslan opened a pull request: https://github.com/apache/nifi/pull/2941 [NIFI-5499] upgrade AngularJS to v1.7.2 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/scottyaslan/nifi NIFI-5499 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2941.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2941 commit c452cc4c1ba80b070bad2147439178cbb17b0e32 Author: Scott Aslan Date: 2018-08-08T15:00:21Z [NIFI-5499] upgrade AngularJS to v1.7.2 > Upgrade AngularJS to v1.7.2 > --- > > Key: NIFI-5499 > URL: https://issues.apache.org/jira/browse/NIFI-5499 > Project: Apache NiFi > Issue Type: Bug >Reporter: Scott Aslan >Assignee: Scott Aslan >Priority: Major > Fix For: 1.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2941: [NIFI-5499] upgrade AngularJS to v1.7.2
GitHub user scottyaslan opened a pull request: https://github.com/apache/nifi/pull/2941 [NIFI-5499] upgrade AngularJS to v1.7.2 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/scottyaslan/nifi NIFI-5499 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2941.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2941 commit c452cc4c1ba80b070bad2147439178cbb17b0e32 Author: Scott Aslan Date: 2018-08-08T15:00:21Z [NIFI-5499] upgrade AngularJS to v1.7.2 ---
[jira] [Created] (NIFIREG-191) Make formatting consistent in Registry docs
Andrew Lim created NIFIREG-191:
--
Summary: Make formatting consistent in Registry docs
Key: NIFIREG-191
URL: https://issues.apache.org/jira/browse/NIFIREG-191
Project: NiFi Registry
Issue Type: Improvement
Reporter: Andrew Lim
Similar to what was done in NIFI-5469, need to make the formatting of
properties, directories and files consistent.
Will use same convention applied to NiFi docs:
filenames in _italics_
directory paths/directories in {{code formatting}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (NIFIREG-191) Make formatting consistent in Registry docs
[
https://issues.apache.org/jira/browse/NIFIREG-191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Lim reassigned NIFIREG-191:
--
Assignee: Andrew Lim
> Make formatting consistent in Registry docs
> ---
>
> Key: NIFIREG-191
> URL: https://issues.apache.org/jira/browse/NIFIREG-191
> Project: NiFi Registry
> Issue Type: Improvement
>Reporter: Andrew Lim
>Assignee: Andrew Lim
>Priority: Minor
>
> Similar to what was done in NIFI-5469, need to make the formatting of
> properties, directories and files consistent.
> Will use same convention applied to NiFi docs:
> filenames in _italics_
> directory paths/directories in {{code formatting}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5498) QueryDatabaseTable unable to store state in Zookeeper
[
https://issues.apache.org/jira/browse/NIFI-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel updated NIFI-5498:
-
Description:
Running NiFi 1.7.1 in a clustered environment, external zookeeper. I'm getting
the following stack trace when running any of my QueryDatabaseTable processors:
{code:java}
eryDatabaseTable[id=a26cafd1-56e4-3f52-99f5-8143abe3ce3a] failed to update
State Manager, maximum observed values will not be recorded:
java.io.IOException: Fai
led to set cluster-wide state in ZooKeeper for component with ID
a26cafd1-56e4-3f52-99f5-8143abe3ce3a
java.io.IOException: Failed to set cluster-wide state in ZooKeeper for
component with ID a26cafd1-56e4-3f52-99f5-8143abe3ce3a
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:343)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:283)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:228)
at
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1.setState(StandardStateManagerProvider.java:271)
at
org.apache.nifi.controller.state.StandardStateManager.setState(StandardStateManager.java:79)
at
org.apache.nifi.controller.lifecycle.TaskTerminationAwareStateManager.setState(TaskTerminationAwareStateManager.java:64)
at
org.apache.nifi.processors.standard.QueryDatabaseTable.onTrigger(QueryDatabaseTable.java:430)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$InvalidACLException:
KeeperErrorCode = InvalidACL for
/nifi/components/a26cafd1-56e4-3f52-99f5-8143abe3ce3a
at org.apache.zookeeper.KeeperException.create(KeeperException.java:121)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.createNode(ZooKeeperStateProvider.java:360)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:321)
... 16 common frames omitted
{code}
The processors do work, but their inability to store state limits their usage,
and the error message floods the log file. Any suggestions would be welcomed.
Other things to note:
* All my other processors (AMQP, Kafka, GetHTTP, ExecuteSQL) work okay.
* I've reproduced this in several environments of mine, including a single
node, two node and three node cluster.
* I thought originally this might be related to the fact that I didn't create
the processor, but imported it from a NiFi registry, but I created one at the
root of my NiFi flow and had the same result.
was:
Running NiFi 1.7.1 in a clustered environment, external zookeeper. I'm getting
the following stack trace when running any of my QueryDatabaseTable processors:
{code:java}
eryDatabaseTable[id=a26cafd1-56e4-3f52-99f5-8143abe3ce3a] failed to update
State Manager, maximum observed values will not be recorded:
java.io.IOException: Fai
led to set cluster-wide state in ZooKeeper for component with ID
a26cafd1-56e4-3f52-99f5-8143abe3ce3a
java.io.IOException: Failed to set cluster-wide state in ZooKeeper for
component with ID a26cafd1-56e4-3f52-99f5-8143abe3ce3a
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:343)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:283)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:228)
at
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1.setState(StandardStateManagerProvider.java:271)
at
[jira] [Created] (NIFI-5499) Upgrade AngularJS to v1.7.2
Scott Aslan created NIFI-5499: - Summary: Upgrade AngularJS to v1.7.2 Key: NIFI-5499 URL: https://issues.apache.org/jira/browse/NIFI-5499 Project: Apache NiFi Issue Type: Bug Reporter: Scott Aslan Assignee: Scott Aslan Fix For: 1.8.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-2415) GenerateTableFetch/QueryDatabaseTable cannot store state
[
https://issues.apache.org/jira/browse/NIFI-2415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573292#comment-16573292
]
Daniel commented on NIFI-2415:
--
I opened NIFI-5498.
> GenerateTableFetch/QueryDatabaseTable cannot store state
>
>
> Key: NIFI-2415
> URL: https://issues.apache.org/jira/browse/NIFI-2415
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0
>Reporter: Haimo Liu
>Assignee: Matt Burgess
>Priority: Critical
> Fix For: 1.0.0, 1.0.0-Beta
>
> Attachments: Screen Shot 2016-07-27 at 11.10.45 PM.png
>
>
> running NIFI in a clustered mode, 7/27 build
> Both GenerateTableFetch and QueryDatabaseTable cannot store state in a
> clustered mode (checked ListFile, can store state successfully)
> see attached screenshot, trace below:
> 2016-07-27 23:25:00,014 INFO [ProcessThread(sid:1 cport:-1):]
> o.a.z.server.PrepRequestProcessor Got user-level KeeperException when
> processing sessionid:0x2562f7875130009 type:create cxid:0xe4d
> zxid:0x73733 txntype:-1 reqpath:n/a Error
> Path:/nifi/components/2f810bb3-0156-1000--16128f8b
> Error:KeeperErrorCode = InvalidACL for
> /nifi/components/2f810bb3-0156-1000--16128f8b
> 2016-07-27 23:25:00,017 WARN [Framework Task Thread Thread-1]
> o.a.n.c.s.m.StandardStateManagerProvider Component with ID {} was removed
> from NiFi instance but failed to clear clustered state for the component
> java.io.IOException: Failed to set cluster-wide state in ZooKeeper for
> component with ID 2f810bb3-0156-1000--16128f8b
> at
> org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:343)
> ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:283)
> ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:228)
> ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.clear(ZooKeeperStateProvider.java:453)
> ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.controller.state.StandardStateManager.clear(StandardStateManager.java:85)
> ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.controller.state.manager.StandardStateManagerProvider.onComponentRemoved(StandardStateManagerProvider.java:276)
> ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.groups.StandardProcessGroup$1.run(StandardProcessGroup.java:730)
> [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_77]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_77]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_77]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> [na:1.8.0_77]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_77]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_77]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
> Caused by: org.apache.zookeeper.KeeperException$InvalidACLException:
> KeeperErrorCode = InvalidACL for
> /nifi/components/2f810bb3-0156-1000--16128f8b
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:121)
> ~[zookeeper-3.4.6.jar:3.4.6-1569965]
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> ~[zookeeper-3.4.6.jar:3.4.6-1569965]
> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
> ~[zookeeper-3.4.6.jar:3.4.6-1569965]
> at
> org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.createNode(ZooKeeperStateProvider.java:360)
> ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:321)
> ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> ... 13 common frames omitted
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5498) QueryDatabaseTable unable to store state in Zookeeper
[
https://issues.apache.org/jira/browse/NIFI-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel updated NIFI-5498:
-
Description:
Running NiFi 1.7.1 in a clustered environment, external zookeeper. I'm getting
the following stack trace when running any of my QueryDatabaseTable processors:
{code:java}
eryDatabaseTable[id=a26cafd1-56e4-3f52-99f5-8143abe3ce3a] failed to update
State Manager, maximum observed values will not be recorded:
java.io.IOException: Fai
led to set cluster-wide state in ZooKeeper for component with ID
a26cafd1-56e4-3f52-99f5-8143abe3ce3a
java.io.IOException: Failed to set cluster-wide state in ZooKeeper for
component with ID a26cafd1-56e4-3f52-99f5-8143abe3ce3a
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:343)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:283)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:228)
at
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1.setState(StandardStateManagerProvider.java:271)
at
org.apache.nifi.controller.state.StandardStateManager.setState(StandardStateManager.java:79)
at
org.apache.nifi.controller.lifecycle.TaskTerminationAwareStateManager.setState(TaskTerminationAwareStateManager.java:64)
at
org.apache.nifi.processors.standard.QueryDatabaseTable.onTrigger(QueryDatabaseTable.java:430)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$InvalidACLException:
KeeperErrorCode = InvalidACL for
/nifi/components/a26cafd1-56e4-3f52-99f5-8143abe3ce3a
at org.apache.zookeeper.KeeperException.create(KeeperException.java:121)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.createNode(ZooKeeperStateProvider.java:360)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:321)
... 16 common frames omitted
{code}
The processors do work, but their inability to store state limits their usage,
and the error message floods the log file. Any suggestions would be welcomed.
Other things to note:
* All my other processors (AMQP, Kafka, GetHTTP) work okay.
* I've reproduced this in several environments of mine, including a single
node, two node and three node cluster.
* I thought originally this might be related to the fact that I didn't create
the processor, but imported it from a NiFi registry, but I created one at the
root of my NiFi flow and had the same result.
was:
Running NiFi 1.7.1 in a clustered environment, external zookeeper. I'm getting
the following stack trace when running any of my QueryDatabaseTable processors:
{code:java}
eryDatabaseTable[id=a26cafd1-56e4-3f52-99f5-8143abe3ce3a] failed to update
State Manager, maximum observed values will not be recorded:
java.io.IOException: Fai
led to set cluster-wide state in ZooKeeper for component with ID
a26cafd1-56e4-3f52-99f5-8143abe3ce3a
java.io.IOException: Failed to set cluster-wide state in ZooKeeper for
component with ID a26cafd1-56e4-3f52-99f5-8143abe3ce3a
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:343)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:283)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:228)
at
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1.setState(StandardStateManagerProvider.java:271)
at
[jira] [Updated] (NIFI-5498) QueryDatabaseTable unable to store state in Zookeeper
[
https://issues.apache.org/jira/browse/NIFI-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel updated NIFI-5498:
-
Description:
Running NiFi 1.7.1 in a clustered environment, external zookeeper. I'm getting
the following stack trace when running any of my QueryDatabaseTable processors:
{code:java}
eryDatabaseTable[id=a26cafd1-56e4-3f52-99f5-8143abe3ce3a] failed to update
State Manager, maximum observed values will not be recorded:
java.io.IOException: Fai
led to set cluster-wide state in ZooKeeper for component with ID
a26cafd1-56e4-3f52-99f5-8143abe3ce3a
java.io.IOException: Failed to set cluster-wide state in ZooKeeper for
component with ID a26cafd1-56e4-3f52-99f5-8143abe3ce3a
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:343)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:283)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:228)
at
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1.setState(StandardStateManagerProvider.java:271)
at
org.apache.nifi.controller.state.StandardStateManager.setState(StandardStateManager.java:79)
at
org.apache.nifi.controller.lifecycle.TaskTerminationAwareStateManager.setState(TaskTerminationAwareStateManager.java:64)
at
org.apache.nifi.processors.standard.QueryDatabaseTable.onTrigger(QueryDatabaseTable.java:430)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$InvalidACLException:
KeeperErrorCode = InvalidACL for
/nifi/components/a26cafd1-56e4-3f52-99f5-8143abe3ce3a
at org.apache.zookeeper.KeeperException.create(KeeperException.java:121)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.createNode(ZooKeeperStateProvider.java:360)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:321)
... 16 common frames omitted
{code}
The processors do work, but their inability to store state limits their usage,
and the error message floods the log file. Any suggestions would be welcomed.
Other things to note:
All my other processors (AMQP, Kafka, GetHTTP) work okay. I've reproduced this
in several environments of mine, including a single node, two node and three
node cluster.
was:
Running NiFi 1.7.1 in a clustered environment, external zookeeper. I'm getting
the following stack trace when running any of my QueryDatabaseTable processors:
{code:java}
eryDatabaseTable[id=a26cafd1-56e4-3f52-99f5-8143abe3ce3a] failed to update
State Manager, maximum observed values will not be recorded:
java.io.IOException: Fai
led to set cluster-wide state in ZooKeeper for component with ID
a26cafd1-56e4-3f52-99f5-8143abe3ce3a
java.io.IOException: Failed to set cluster-wide state in ZooKeeper for
component with ID a26cafd1-56e4-3f52-99f5-8143abe3ce3a
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:343)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:283)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:228)
at
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1.setState(StandardStateManagerProvider.java:271)
at
org.apache.nifi.controller.state.StandardStateManager.setState(StandardStateManager.java:79)
at
org.apache.nifi.controller.lifecycle.TaskTerminationAwareStateManager.setState(TaskTerminationAwareStateManager.java:64)
at
[jira] [Created] (NIFI-5498) QueryDatabaseTable unable to store state in Zookeeper
Daniel created NIFI-5498:
Summary: QueryDatabaseTable unable to store state in Zookeeper
Key: NIFI-5498
URL: https://issues.apache.org/jira/browse/NIFI-5498
Project: Apache NiFi
Issue Type: Improvement
Components: Extensions
Affects Versions: 1.7.1
Reporter: Daniel
Running NiFi 1.7.1 in a clustered environment, external zookeeper. I'm getting
the following stack trace when running any of my QueryDatabaseTable processors:
{code:java}
eryDatabaseTable[id=a26cafd1-56e4-3f52-99f5-8143abe3ce3a] failed to update
State Manager, maximum observed values will not be recorded:
java.io.IOException: Fai
led to set cluster-wide state in ZooKeeper for component with ID
a26cafd1-56e4-3f52-99f5-8143abe3ce3a
java.io.IOException: Failed to set cluster-wide state in ZooKeeper for
component with ID a26cafd1-56e4-3f52-99f5-8143abe3ce3a
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:343)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:283)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:228)
at
org.apache.nifi.controller.state.manager.StandardStateManagerProvider$1.setState(StandardStateManagerProvider.java:271)
at
org.apache.nifi.controller.state.StandardStateManager.setState(StandardStateManager.java:79)
at
org.apache.nifi.controller.lifecycle.TaskTerminationAwareStateManager.setState(TaskTerminationAwareStateManager.java:64)
at
org.apache.nifi.processors.standard.QueryDatabaseTable.onTrigger(QueryDatabaseTable.java:430)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$InvalidACLException:
KeeperErrorCode = InvalidACL for
/nifi/components/a26cafd1-56e4-3f52-99f5-8143abe3ce3a
at org.apache.zookeeper.KeeperException.create(KeeperException.java:121)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.createNode(ZooKeeperStateProvider.java:360)
at
org.apache.nifi.controller.state.providers.zookeeper.ZooKeeperStateProvider.setState(ZooKeeperStateProvider.java:321)
... 16 common frames omitted
{code}
The processors do work, but their inability to store state limits their usage,
and the error message floods the log file. Any suggestions would be welcomed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5487) Version agnostic paths in the docker container
[ https://issues.apache.org/jira/browse/NIFI-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573261#comment-16573261 ] ASF subversion and git services commented on NIFI-5487: --- Commit 9a79c94f809a29ebb5c36ccca8a680f1875af1f2 in nifi's branch refs/heads/master from pepov [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9a79c94 ] NIFI-5487 Move the packages to a version agnostic path and create symlinks in place, add tests to cover, add missing docs on nifi-toolkit This closes #2940. Signed-off-by: Aldrin Piri > Version agnostic paths in the docker container > -- > > Key: NIFI-5487 > URL: https://issues.apache.org/jira/browse/NIFI-5487 > Project: Apache NiFi > Issue Type: Improvement > Components: Docker >Reporter: Peter Wilcsinszky >Assignee: Peter Wilcsinszky >Priority: Major > Fix For: 1.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5487) Version agnostic paths in the docker container
[ https://issues.apache.org/jira/browse/NIFI-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573263#comment-16573263 ] ASF GitHub Bot commented on NIFI-5487: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2940 > Version agnostic paths in the docker container > -- > > Key: NIFI-5487 > URL: https://issues.apache.org/jira/browse/NIFI-5487 > Project: Apache NiFi > Issue Type: Improvement > Components: Docker >Reporter: Peter Wilcsinszky >Assignee: Peter Wilcsinszky >Priority: Major > Fix For: 1.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (NIFI-5487) Version agnostic paths in the docker container
[ https://issues.apache.org/jira/browse/NIFI-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aldrin Piri resolved NIFI-5487. --- Resolution: Done > Version agnostic paths in the docker container > -- > > Key: NIFI-5487 > URL: https://issues.apache.org/jira/browse/NIFI-5487 > Project: Apache NiFi > Issue Type: Improvement > Components: Docker >Reporter: Peter Wilcsinszky >Assignee: Peter Wilcsinszky >Priority: Major > Fix For: 1.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2940: NIFI-5487 Move the packages to a version agnostic p...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2940 ---
[jira] [Commented] (NIFI-5487) Version agnostic paths in the docker container
[ https://issues.apache.org/jira/browse/NIFI-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573259#comment-16573259 ] ASF GitHub Bot commented on NIFI-5487: -- Github user apiri commented on the issue: https://github.com/apache/nifi/pull/2940 Performed a build and verified both the Maven and the Hub (using 1.7.0) variants and worked as anticipated. I think this should make it a bit easier for folks to migrate between versions. Thanks for taking care of this and I'll get the merge in. > Version agnostic paths in the docker container > -- > > Key: NIFI-5487 > URL: https://issues.apache.org/jira/browse/NIFI-5487 > Project: Apache NiFi > Issue Type: Improvement > Components: Docker >Reporter: Peter Wilcsinszky >Assignee: Peter Wilcsinszky >Priority: Major > Fix For: 1.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2940: NIFI-5487 Move the packages to a version agnostic path and...
Github user apiri commented on the issue: https://github.com/apache/nifi/pull/2940 Performed a build and verified both the Maven and the Hub (using 1.7.0) variants and worked as anticipated. I think this should make it a bit easier for folks to migrate between versions. Thanks for taking care of this and I'll get the merge in. ---
[jira] [Comment Edited] (NIFI-5497) XPATH interpretation not correct
[
https://issues.apache.org/jira/browse/NIFI-5497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573246#comment-16573246
]
Stefan Goldener edited comment on NIFI-5497 at 8/8/18 1:49 PM:
---
I still consider this as a BUG but for those who stumble upon this Jira, there
is a workaround:
{code:java}
//@*[name()="xlink:href"]{code}
or
{code:java}
//@*[local-name()="href"]{code}
was (Author: renedlog):
I still consider this as a BUG but for those who stumble upon this Jira there
is a workaround:
{code:java}
//@*[name()="xlink:href"]{code}
or
{code:java}
//@*[local-name()="href"]{code}
> XPATH interpretation not correct
>
>
> Key: NIFI-5497
> URL: https://issues.apache.org/jira/browse/NIFI-5497
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.7.1
> Environment: CentOS 7
>Reporter: Stefan Goldener
>Priority: Major
> Attachments: image-2018-08-08-13-21-37-820.png
>
>
> For example using the following XML:
> {code:java}
>
> http://www.w3.org/1999/xlink;
> a="17"
> date="2018-02-16T22:19:45+01:00"
> xlink:href="http://url.com/webservice/rest/1.0/"/>{code}
>
> With XPATH i can usually do:
> {code:java}
> //@xlink:href{code}
> to get the url.
>
> This does not work with Nifi because it evaluates the XPath expression
> somehow incorrectly.
> !image-2018-08-08-13-21-37-820.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5497) XPATH interpretation not correct
[
https://issues.apache.org/jira/browse/NIFI-5497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573246#comment-16573246
]
Stefan Goldener commented on NIFI-5497:
---
I still consider this as a BUG but for those who stumble upon this Jira there
is a workaround:
{code:java}
//@*[name()="xlink:href"]{code}
or
{code:java}
//@*[local-name()="href"]{code}
> XPATH interpretation not correct
>
>
> Key: NIFI-5497
> URL: https://issues.apache.org/jira/browse/NIFI-5497
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.7.1
> Environment: CentOS 7
>Reporter: Stefan Goldener
>Priority: Major
> Attachments: image-2018-08-08-13-21-37-820.png
>
>
> For example using the following XML:
> {code:java}
>
> http://www.w3.org/1999/xlink;
> a="17"
> date="2018-02-16T22:19:45+01:00"
> xlink:href="http://url.com/webservice/rest/1.0/"/>{code}
>
> With XPATH i can usually do:
> {code:java}
> //@xlink:href{code}
> to get the url.
>
> This does not work with Nifi because it evaluates the XPath expression
> somehow incorrectly.
> !image-2018-08-08-13-21-37-820.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4434) ListHDFS applies File Filter also to subdirectory names in recursive search
[ https://issues.apache.org/jira/browse/NIFI-4434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573220#comment-16573220 ] ASF GitHub Bot commented on NIFI-4434: -- Github user ottobackwards commented on the issue: https://github.com/apache/nifi/pull/2937 Built with -Pcontrib-check, verified the documentation, +1 from me, nice work > ListHDFS applies File Filter also to subdirectory names in recursive search > --- > > Key: NIFI-4434 > URL: https://issues.apache.org/jira/browse/NIFI-4434 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.3.0 >Reporter: Holger Frydrych >Assignee: Jeff Storck >Priority: Major > > The File Filter regex configured in the ListHDFS processor is applied not > just to files found, but also to subdirectories. > If you try to set up a recursive search to list e.g. all csv files in a > directory hierarchy via a regex like ".*\.csv", it will only pick up csv > files in the base directory, not in any subdirectory. This is because > subdirectories don't typically match that regex pattern. > To fix this, either subdirectories should not be matched against the file > filter, or the file filter should be applied to the full path of all files > (relative to the base directory). The GetHDFS processor offers both options > via a switch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2937: NIFI-4434 Fixed recursive listing with a custom regex filt...
Github user ottobackwards commented on the issue: https://github.com/apache/nifi/pull/2937 Built with -Pcontrib-check, verified the documentation, +1 from me, nice work ---
[jira] [Assigned] (NIFI-3320) Listen to SNMP Trap
[ https://issues.apache.org/jira/browse/NIFI-3320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemantha kumara M S reassigned NIFI-3320: - Assignee: Hemantha kumara M S > Listen to SNMP Trap > --- > > Key: NIFI-3320 > URL: https://issues.apache.org/jira/browse/NIFI-3320 > Project: Apache NiFi > Issue Type: New Feature >Affects Versions: 1.1.1 >Reporter: Balakrishnan R >Assignee: Hemantha kumara M S >Priority: Major > Labels: SNMP, Traps > > As part NIFI-1537 SNMP Get, Walk and Set were introduced. However SNMP Traps > sent to NIFI cannot be converted to a flowfile currently. This is a fairly > useful feature. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (NIFI-5435) Prometheus /metrics http endpoint for monitoring integration
[ https://issues.apache.org/jira/browse/NIFI-5435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemantha kumara M S reassigned NIFI-5435: - Assignee: Hemantha kumara M S > Prometheus /metrics http endpoint for monitoring integration > > > Key: NIFI-5435 > URL: https://issues.apache.org/jira/browse/NIFI-5435 > Project: Apache NiFi > Issue Type: New Feature > Components: Configuration, Core Framework, Extensions, Tools and > Build >Affects Versions: 1.5.0 >Reporter: Hari Sekhon >Assignee: Hemantha kumara M S >Priority: Major > > Feature Request to add Prometheus /metrics http endpoint for monitoring > integration: > [https://prometheus.io/docs/prometheus/latest/configuration/configuration/#%3Cscrape_config%3E] > Prometheus metrics format for that endpoint: > [https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md] > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4710) Kerberos
[ https://issues.apache.org/jira/browse/NIFI-4710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573170#comment-16573170 ] ASF GitHub Bot commented on NIFI-4710: -- Github user pepov commented on the issue: https://github.com/apache/nifi/pull/2866 @SarthakSahu we didn't mean to discourage you, I beleive it would be great to have the SPNEGO setup working and it's not far from the proposed changeset at all. When you create the user principal you can also create another one (the SPN) for NiFi as well and save the keytab. It can be as easy as: ``` addprinc -randkey HTTP/example.com ktadd -k nifi.keytab HTTP/example.com ``` Then you need to get `nifi.keytab` add it to your volume mount and configure the `nifi.kerberos.spnego.principal` and `nifi.kerberos.spnego.keytab.location` properties and you should be ready to go. To test it you need to use `kinit` on your local machine and configure your browser to be able to use SPNEGO. For an example see: https://bryanbende.com/development/2016/08/31/apache-nifi-1.0.0-kerberos-authentication > Kerberos > > > Key: NIFI-4710 > URL: https://issues.apache.org/jira/browse/NIFI-4710 > Project: Apache NiFi > Issue Type: Sub-task > Components: Docker >Reporter: Aldrin Piri >Assignee: Sarthak >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2866: NIFI-4710 Kerberos support for user auth in Docker instanc...
Github user pepov commented on the issue: https://github.com/apache/nifi/pull/2866 @SarthakSahu we didn't mean to discourage you, I beleive it would be great to have the SPNEGO setup working and it's not far from the proposed changeset at all. When you create the user principal you can also create another one (the SPN) for NiFi as well and save the keytab. It can be as easy as: ``` addprinc -randkey HTTP/example.com ktadd -k nifi.keytab HTTP/example.com ``` Then you need to get `nifi.keytab` add it to your volume mount and configure the `nifi.kerberos.spnego.principal` and `nifi.kerberos.spnego.keytab.location` properties and you should be ready to go. To test it you need to use `kinit` on your local machine and configure your browser to be able to use SPNEGO. For an example see: https://bryanbende.com/development/2016/08/31/apache-nifi-1.0.0-kerberos-authentication ---
[jira] [Commented] (NIFI-5487) Version agnostic paths in the docker container
[ https://issues.apache.org/jira/browse/NIFI-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573131#comment-16573131 ] ASF GitHub Bot commented on NIFI-5487: -- Github user pepov commented on the issue: https://github.com/apache/nifi/pull/2940 Ran `mvn integration-test -P docker` from nifi-docker after built all the necessary binaries in `nifi-assembly` and `nifi-toolkit/nifi-toolkit-assembly`. > Version agnostic paths in the docker container > -- > > Key: NIFI-5487 > URL: https://issues.apache.org/jira/browse/NIFI-5487 > Project: Apache NiFi > Issue Type: Improvement > Components: Docker >Reporter: Peter Wilcsinszky >Assignee: Peter Wilcsinszky >Priority: Major > Fix For: 1.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5487) Version agnostic paths in the docker container
[ https://issues.apache.org/jira/browse/NIFI-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573127#comment-16573127 ] ASF GitHub Bot commented on NIFI-5487: -- GitHub user pepov opened a pull request: https://github.com/apache/nifi/pull/2940 NIFI-5487 Move the packages to a version agnostic path and create sym… …links in place, add tests to cover, add missing docs on nifi-toolkit Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pepov/nifi NIFI-5487 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2940.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2940 commit eba0ea81dc3704c87867927ded51d67381b4a0eb Author: pepov Date: 2018-08-08T12:03:10Z NIFI-5487 Move the packages to a version agnostic path and create symlinks in place, add tests to cover, add missing docs on nifi-toolkit > Version agnostic paths in the docker container > -- > > Key: NIFI-5487 > URL: https://issues.apache.org/jira/browse/NIFI-5487 > Project: Apache NiFi > Issue Type: Improvement > Components: Docker >Reporter: Peter Wilcsinszky >Assignee: Peter Wilcsinszky >Priority: Major > Fix For: 1.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2940: NIFI-5487 Move the packages to a version agnostic p...
GitHub user pepov opened a pull request: https://github.com/apache/nifi/pull/2940 NIFI-5487 Move the packages to a version agnostic path and create sym⦠â¦links in place, add tests to cover, add missing docs on nifi-toolkit Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pepov/nifi NIFI-5487 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2940.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2940 commit eba0ea81dc3704c87867927ded51d67381b4a0eb Author: pepov Date: 2018-08-08T12:03:10Z NIFI-5487 Move the packages to a version agnostic path and create symlinks in place, add tests to cover, add missing docs on nifi-toolkit ---
[GitHub] nifi issue #2940: NIFI-5487 Move the packages to a version agnostic path and...
Github user pepov commented on the issue: https://github.com/apache/nifi/pull/2940 Ran `mvn integration-test -P docker` from nifi-docker after built all the necessary binaries in `nifi-assembly` and `nifi-toolkit/nifi-toolkit-assembly`. ---
[jira] [Created] (NIFI-5497) XPATH interpretation not correct
Stefan Goldener created NIFI-5497:
-
Summary: XPATH interpretation not correct
Key: NIFI-5497
URL: https://issues.apache.org/jira/browse/NIFI-5497
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.7.1
Environment: CentOS 7
Reporter: Stefan Goldener
Attachments: image-2018-08-08-13-21-37-820.png
For example using the following XML:
{code:java}
http://www.w3.org/1999/xlink;
a="17"
date="2018-02-16T22:19:45+01:00"
xlink:href="http://url.com/webservice/rest/1.0/"/>{code}
With XPATH i can usually do:
{code:java}
//@xlink:href{code}
to get the url.
This does not work with Nifi because it evaluates the XPath expression somehow
incorrectly.
!image-2018-08-08-13-21-37-820.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
