[jira] [Commented] (NIFI-4169) PutWebSocket processor with blank WebSocket session id attribute cannot transfer to failure queue
[ https://issues.apache.org/jira/browse/NIFI-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083447#comment-16083447 ] Y Wikander commented on NIFI-4169: -- Could you expand on your last bullet point? Are you suggesting cloning each flowfile that fails to a flowfile with the specific 'websocket.session.id' as an attribute? If not, what if the flowfile failed to send to more than _one_ websocket session? 'websocket.session.id' would contain a comma separated list? What would someone actually _do_ with the websocket.session.id value? Have PutWebSocket use it to control which session id(s) to attempt to send to again if it was routed back into PutWebSocket as 'failure'? > PutWebSocket processor with blank WebSocket session id attribute cannot > transfer to failure queue > - > > Key: NIFI-4169 > URL: https://issues.apache.org/jira/browse/NIFI-4169 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Y Wikander >Priority: Critical > Labels: patch > Attachments: > 0001-websocket-when-sendMessage-fails-under-blank-session.patch > > > If a PutWebSocket processor is setup with a blank WebSocket session id > attribute (see NIFI-3318; Send message from PutWebSocket to all connected > clients) and it is not connected to a websocket server it will log the > failure and mark the flowfile with Success (rather than Failure) -- and the > data is effectively lost. > If there are multiple connected clients, and some succeed and others fail, > routing Failure back into the PutWebSocket could result in duplicate data to > some clients. > Other NiFi processors seem to err on the side of "at least once". -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4170) PutWebSocket processor does not support 'Penalty duration' and 'Yield duration' settings
[ https://issues.apache.org/jira/browse/NIFI-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083420#comment-16083420 ] Y Wikander commented on NIFI-4170: -- I'll pursue your line of thinking. The discussion turning to processor level misconfiguration makes me ponder the Validators. Not that I really wanted to go deep into the weeds; but would Validators be the "correct" way? Such that the processor would be stopped and the yellow warning symbol would show up in the GUI. This is presuming that Users would be more interested in pursuing those (warning) messages vs. assumed transitory error messages that will probably go away on their own (if you wait long enough). To that end, can Validators be triggered _while_ a processor is running? And, does a processor stop itself as part this action? > PutWebSocket processor does not support 'Penalty duration' and 'Yield > duration' settings > > > Key: NIFI-4170 > URL: https://issues.apache.org/jira/browse/NIFI-4170 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Y Wikander >Priority: Minor > Labels: patch > Attachments: > 0002-websocket-PutWebSocket-processor-support-Penalty-dur.patch > > > PutWebSocket processor does not support 'Penalty duration' and 'Yield > duration' settings. > I'm assuming that calling content.yield() will also cover 'Penalty duration'. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4170) PutWebSocket processor does not support 'Penalty duration' and 'Yield duration' settings
[ https://issues.apache.org/jira/browse/NIFI-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083337#comment-16083337 ] Koji Kawamura commented on NIFI-4170: - That's a good point. I agree with that. I think following rule would be the best: - If the issue is likely caused by incoming FlowFile, the FlowFile should be penalized. No need to yield the processor because other FlowFile might have correct attributes and just works. -- When processor didn't find a CS, [here|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-websocket-bundle/nifi-websocket-processors/src/main/java/org/apache/nifi/processors/websocket/PutWebSocket.java#L183] and [here|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-websocket-bundle/nifi-websocket-processors/src/main/java/org/apache/nifi/processors/websocket/PutWebSocket.java#L186] - If the issue is likely caused by processor level miss configuration or target server (WebSocket client or server), then we yield the processor. Optionally we can penalize the FlowFile, too (just for the use-case where 'failure' is connected back to itself). -- When processor catches Exception, [here|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-websocket-bundle/nifi-websocket-processors/src/main/java/org/apache/nifi/processors/websocket/PutWebSocket.java#L238] So, adding penalize at transferToFailure(), and adding context.yield() when processor catches Exception would do above. How do you think? > PutWebSocket processor does not support 'Penalty duration' and 'Yield > duration' settings > > > Key: NIFI-4170 > URL: https://issues.apache.org/jira/browse/NIFI-4170 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.3.0, 1.4.0 >Reporter: Y Wikander >Priority: Minor > Labels: patch > Fix For: 1.3.0, 1.4.0 > > Attachments: > 0002-websocket-PutWebSocket-processor-support-Penalty-dur.patch > > > PutWebSocket processor does not support 'Penalty duration' and 'Yield > duration' settings. > I'm assuming that calling content.yield() will also cover 'Penalty duration'. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4169) PutWebSocket processor with blank WebSocket session id attribute cannot transfer to failure queue
[ https://issues.apache.org/jira/browse/NIFI-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-4169: Affects Version/s: (was: 1.4.0) > PutWebSocket processor with blank WebSocket session id attribute cannot > transfer to failure queue > - > > Key: NIFI-4169 > URL: https://issues.apache.org/jira/browse/NIFI-4169 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Y Wikander >Priority: Critical > Labels: patch > Attachments: > 0001-websocket-when-sendMessage-fails-under-blank-session.patch > > > If a PutWebSocket processor is setup with a blank WebSocket session id > attribute (see NIFI-3318; Send message from PutWebSocket to all connected > clients) and it is not connected to a websocket server it will log the > failure and mark the flowfile with Success (rather than Failure) -- and the > data is effectively lost. > If there are multiple connected clients, and some succeed and others fail, > routing Failure back into the PutWebSocket could result in duplicate data to > some clients. > Other NiFi processors seem to err on the side of "at least once". -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4170) PutWebSocket processor does not support 'Penalty duration' and 'Yield duration' settings
[ https://issues.apache.org/jira/browse/NIFI-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-4170: Affects Version/s: (was: 1.4.0) Fix Version/s: (was: 1.4.0) (was: 1.3.0) > PutWebSocket processor does not support 'Penalty duration' and 'Yield > duration' settings > > > Key: NIFI-4170 > URL: https://issues.apache.org/jira/browse/NIFI-4170 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Y Wikander >Priority: Minor > Labels: patch > Attachments: > 0002-websocket-PutWebSocket-processor-support-Penalty-dur.patch > > > PutWebSocket processor does not support 'Penalty duration' and 'Yield > duration' settings. > I'm assuming that calling content.yield() will also cover 'Penalty duration'. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4169) PutWebSocket processor with blank WebSocket session id attribute cannot transfer to failure queue
[ https://issues.apache.org/jira/browse/NIFI-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-4169: Fix Version/s: (was: 1.4.0) (was: 1.3.0) > PutWebSocket processor with blank WebSocket session id attribute cannot > transfer to failure queue > - > > Key: NIFI-4169 > URL: https://issues.apache.org/jira/browse/NIFI-4169 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.3.0 >Reporter: Y Wikander >Priority: Critical > Labels: patch > Attachments: > 0001-websocket-when-sendMessage-fails-under-blank-session.patch > > > If a PutWebSocket processor is setup with a blank WebSocket session id > attribute (see NIFI-3318; Send message from PutWebSocket to all connected > clients) and it is not connected to a websocket server it will log the > failure and mark the flowfile with Success (rather than Failure) -- and the > data is effectively lost. > If there are multiple connected clients, and some succeed and others fail, > routing Failure back into the PutWebSocket could result in duplicate data to > some clients. > Other NiFi processors seem to err on the side of "at least once". -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4169) PutWebSocket processor with blank WebSocket session id attribute cannot transfer to failure queue
[ https://issues.apache.org/jira/browse/NIFI-4169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083331#comment-16083331 ] Koji Kawamura commented on NIFI-4169: - With ConnectWebSocket, broadcasting messages would not be interesting at all, since NiFi is a WebSocket client and talking to a single WebSocket server. In this case, the size of the session wouldn't be grater than one. And there's no difference between broadcasting or not. The only benefit I can think of as a client is that, PutWebSocket doesn't have to know websocket session id to send a message to the connected WebSocket server from NiFi. Broadcasting is useful when NiFi acts as WebSocket server with ListenWebSocket processor. It can accepts multiple WebSocket clients to connect a NiFi. Then some use case requires push notification to all connected WebSocket clients from NiFi. For example, if a user want to show UI alert or notification when NiFi receives some important message from a MessagingQueue or topic, then they could write a Web App connecting to NiFi via WebSocket protocol to receive such notification. And if there're multiple users using the same Web App, then NiFi will have multiple WebSocket sessions and has to distribute the same notification to every connected client. In this context, even if NiFi couldn't send a message to a particular client, we'd expect it to continue sending to other clients. However, PutWebSocket doesn't care whether the session is for as a client or as a server. So, I think following implementation would be reasonable for PutWebSocket: - Transfer the incoming FlowFile to success if it's sent to at least one target - Continue looping through sessions even if one throws Exception - Catch Exception while in the loop, then generate FlowFiles to send to 'failure' relationship. Embed failed websocket session id to 'websocket.session.id' FlowFile attribute How do you think? > PutWebSocket processor with blank WebSocket session id attribute cannot > transfer to failure queue > - > > Key: NIFI-4169 > URL: https://issues.apache.org/jira/browse/NIFI-4169 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.3.0, 1.4.0 >Reporter: Y Wikander >Priority: Critical > Labels: patch > Fix For: 1.3.0, 1.4.0 > > Attachments: > 0001-websocket-when-sendMessage-fails-under-blank-session.patch > > > If a PutWebSocket processor is setup with a blank WebSocket session id > attribute (see NIFI-3318; Send message from PutWebSocket to all connected > clients) and it is not connected to a websocket server it will log the > failure and mark the flowfile with Success (rather than Failure) -- and the > data is effectively lost. > If there are multiple connected clients, and some succeed and others fail, > routing Failure back into the PutWebSocket could result in duplicate data to > some clients. > Other NiFi processors seem to err on the side of "at least once". -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083172#comment-16083172 ] ASF GitHub Bot commented on NIFI-1613: -- Github user ijokarumawak commented on a diff in the pull request: https://github.com/apache/nifi/pull/1976#discussion_r126833370 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml --- @@ -467,6 +467,7 @@ src/test/resources/TestUpdateRecord/output/person-with-firstname.json src/test/resources/TestUpdateRecord/output/person-with-firstname-lastname.json src/test/resources/TestUpdateRecord/output/person-with-capital-lastname.json + src/test/resources/TestUpdateRecord/output/person-with-bool.json --- End diff -- Thanks @mattyb149 , fixed this and confirmed RAT check passed locally. > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #1976: NIFI-1613: Make use of column type correctly at Con...
Github user ijokarumawak commented on a diff in the pull request: https://github.com/apache/nifi/pull/1976#discussion_r126833370 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml --- @@ -467,6 +467,7 @@ src/test/resources/TestUpdateRecord/output/person-with-firstname.json src/test/resources/TestUpdateRecord/output/person-with-firstname-lastname.json src/test/resources/TestUpdateRecord/output/person-with-capital-lastname.json + src/test/resources/TestUpdateRecord/output/person-with-bool.json --- End diff -- Thanks @mattyb149 , fixed this and confirmed RAT check passed locally. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4130) TransformXml - provide a way to define XSLT without external files
[ https://issues.apache.org/jira/browse/NIFI-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083010#comment-16083010 ] ASF GitHub Bot commented on NIFI-4130: -- Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1953 @pvillard31 It seems like we should just use LookupAttribute for this use case. Thoughts? > TransformXml - provide a way to define XSLT without external files > -- > > Key: NIFI-4130 > URL: https://issues.apache.org/jira/browse/NIFI-4130 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard > > In cluster deployments the need to reference external configuration files can > be annoying since it requires to access to all the NiFi nodes and to > correctly deploy the files. It would be interesting to leverage the lookup > controller services in TransformXml to provide a way to define XSLT directly > from the UI without external configuration files. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #1953: NIFI-4130 Add lookup controller service in TransformXML to...
Github user jfrazee commented on the issue: https://github.com/apache/nifi/pull/1953 @pvillard31 It seems like we should just use LookupAttribute for this use case. Thoughts? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFIREG-1) Create Initial Project Structure
[ https://issues.apache.org/jira/browse/NIFIREG-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082832#comment-16082832 ] ASF GitHub Bot commented on NIFIREG-1: -- Github user bbende commented on the issue: https://github.com/apache/nifi-registry/pull/1 This project structure is a work in progress to begin development and is expected to continue evolving as the project matures, but this should give enough for others to start collaborating. > Create Initial Project Structure > > > Key: NIFIREG-1 > URL: https://issues.apache.org/jira/browse/NIFIREG-1 > Project: NiFi Registry > Issue Type: New Feature >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Fix For: 0.0.1 > > > Create an initial project structure similar to NiFi with an assembly that > produces a binary artifact (tar/zip) to run the application. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFIREG-1) Create Initial Project Structure
[ https://issues.apache.org/jira/browse/NIFIREG-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082828#comment-16082828 ] ASF GitHub Bot commented on NIFIREG-1: -- GitHub user bbende opened a pull request: https://github.com/apache/nifi-registry/pull/1 NIFIREG-1 Initial project structure for NiFi Registry You can merge this pull request into a Git repository by running: $ git pull https://github.com/bbende/nifi-registry NIFIREG-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-registry/pull/1.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1 commit f5753966b58a4e1045eded8ffb36a3923fa0a31b Author: Bryan BendeDate: 2017-03-16T17:45:16Z NIFIREG-1 Initial project structure for NiFi Registry > Create Initial Project Structure > > > Key: NIFIREG-1 > URL: https://issues.apache.org/jira/browse/NIFIREG-1 > Project: NiFi Registry > Issue Type: New Feature >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Fix For: 0.0.1 > > > Create an initial project structure similar to NiFi with an assembly that > produces a binary artifact (tar/zip) to run the application. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFIREG-1) Create Initial Project Structure
[ https://issues.apache.org/jira/browse/NIFIREG-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082838#comment-16082838 ] ASF GitHub Bot commented on NIFIREG-1: -- Github user mcgilman commented on the issue: https://github.com/apache/nifi-registry/pull/1 Awesome thanks @bbende! Will review... > Create Initial Project Structure > > > Key: NIFIREG-1 > URL: https://issues.apache.org/jira/browse/NIFIREG-1 > Project: NiFi Registry > Issue Type: New Feature >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Fix For: 0.0.1 > > > Create an initial project structure similar to NiFi with an assembly that > produces a binary artifact (tar/zip) to run the application. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (NIFI-4119) Improve UX of canvas label configuration
[ https://issues.apache.org/jira/browse/NIFI-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto resolved NIFI-4119. - Resolution: Fixed > Improve UX of canvas label configuration > > > Key: NIFI-4119 > URL: https://issues.apache.org/jira/browse/NIFI-4119 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 1.3.0 >Reporter: Yuri >Priority: Trivial > Labels: newbie > > The single most used field of a canvas label's configuration dialog is it's > text value. Naturally, it should be in focus as soon as you open the dialog. > Current UX seems to be tedious when dealing with multiple labels. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-3680) Zookeeper ZNodes for NiFi LeaderElection shouldn't use spaces for ease of use
[ https://issues.apache.org/jira/browse/NIFI-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082601#comment-16082601 ] Joseph Niemiec commented on NIFI-3680: -- Just some more detail, it appears zkCli parses commands using a space as a split key. https://github.com/apache/zookeeper/blob/b9beabf5cfbb8e447f67a8050486ca2821132f61/src/java/main/org/apache/zookeeper/ZooKeeperMain.java#L156 > Zookeeper ZNodes for NiFi LeaderElection shouldn't use spaces for ease of use > - > > Key: NIFI-3680 > URL: https://issues.apache.org/jira/browse/NIFI-3680 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.1.1 > Environment: Ubuntu 14, Oracle JDK 1.8.0_77, 3 Node cluster, > Zookeeper 3.4.6 [2.5.3.0-37] >Reporter: Joseph Niemiec >Priority: Trivial > > When navigating the ZKCli for problems the current naming of ZNodes for > leader-election isn't easy to navigate. Noticed that the default cli > behaviour of a space is that is is stored as data inside the path of the > znode specified. I assume the ZK Java API is being alittle more 'powerful' > but it would be nice to enable operators a easier native way to check the > little things. > ``` > [zk: localhost:2181(CONNECTED) 7] ls /nifi/leaders/ > Primary Node Cluster Coordinator > [zk: localhost:2181(CONNECTED) 7] ls /nifi/leaders/Primary Node > Node does not exist: /nifi/leaders/Primary > [zk: localhost:2181(CONNECTED) 8] ls /nifi/leaders/Cluster Coordinator > Node does not exist: /nifi/leaders/Cluster > [zk: localhost:2181(CONNECTED) 1] create /test blah > Created /test > [zk: localhost:2181(CONNECTED) 2] get /test > blah > cZxid = 0xb038e > ctime = Thu Apr 06 21:24:10 EDT 2017 > mZxid = 0xb038e > mtime = Thu Apr 06 21:24:10 EDT 2017 > pZxid = 0xb038e > cversion = 0 > dataVersion = 0 > aclVersion = 0 > ephemeralOwner = 0x0 > dataLength = 4 > numChildren = 0 > ``` -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-3677) ExtracMediaMetadata should close TikaInputStream or use Tika TemporaryResources
[ https://issues.apache.org/jira/browse/NIFI-3677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3677: - Component/s: (was: Core Framework) Extensions > ExtracMediaMetadata should close TikaInputStream or use Tika > TemporaryResources > --- > > Key: NIFI-3677 > URL: https://issues.apache.org/jira/browse/NIFI-3677 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 0.8.0, 1.2.0 >Reporter: Joe Skora >Assignee: Joe Skora >Priority: Minor > Fix For: 1.4.0 > > > Per Tika docs\[1], {{ExtractMediaMetadata}} should cleanup the > {{TikaInputStream}} it creates or use a Tika's {{TemporaryResources}} class > for automatic cleanup. > \[1]https://tika.apache.org/1.8/api/org/apache/tika/io/TikaInputStream.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4175) Add Proxy Properties to SFTP Processors
[ https://issues.apache.org/jira/browse/NIFI-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Langlois updated NIFI-4175: - Summary: Add Proxy Properties to SFTP Processors (was: Add Proxy Properties to SFTP processors) > Add Proxy Properties to SFTP Processors > --- > > Key: NIFI-4175 > URL: https://issues.apache.org/jira/browse/NIFI-4175 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Grant Langlois >Priority: Minor > > Add proxy server configuration as properties to the Nifi SFTP components. > Specifically add properties for: > Proxy Type: JSCH supported proxies including SOCKS4, SOCKS5 and HTTP > Proxy Host > Proxy Port > Proxy Username > Proxy Password > This would allow these properties to be configured for each processor. These > properties would align with what is configurable for the JSCH session and > shouldn't require any additional dependencies. > This proposal is similar to what is already implemented for the FTP processors -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (NIFI-4176) IPLookupService should avoid using FILE_EXISTS_VALIDATOR
Mark Payne created NIFI-4176: Summary: IPLookupService should avoid using FILE_EXISTS_VALIDATOR Key: NIFI-4176 URL: https://issues.apache.org/jira/browse/NIFI-4176 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Mark Payne Currently, the IPLookupService uses a FILE_EXISTS_VALIDATOR in order to ensure that the given database file exists. Quite often, though, a flow is developed in such a way that the file is dynamically fetched via GetHTTP, InvokeHTTP, etc. and then written out so that IPLookupService is able to automatically reload the data. If a new flow is imported via template, for example, or copied from another NiFi instance, the file may not yet exist. As a result, the Controller Service is invalid and upon startup the service is not enabled. This then causes referencing processors not to start. This can be particularly problematic if running in a cluster where some nodes have the file and start the processor while other nodes do not. To avoid this, the IPLookupService (and Controller Services, in general) should not use the FILE_EXISTS_VALIDATOR but instead just fail in the @OnScheduled method. This way, the service will continually attempt to start and once the file does exist, everything will begin flowing as expected. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (NIFI-4175) Add Proxy Properties to SFTP processors
Grant Langlois created NIFI-4175: Summary: Add Proxy Properties to SFTP processors Key: NIFI-4175 URL: https://issues.apache.org/jira/browse/NIFI-4175 Project: Apache NiFi Issue Type: New Feature Reporter: Grant Langlois Priority: Minor Add proxy server configuration as properties to the Nifi SFTP components. Specifically add properties for: Proxy Type: JSCH supported proxies including SOCKS4, SOCKS5 and HTTP Proxy Host Proxy Port Proxy Username Proxy Password This would allow these properties to be configured for each processor. These properties would align with what is configurable for the JSCH session and shouldn't require any additional dependencies. This proposal is similar to what is already implemented for the FTP processors -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4127) Create a CompositeUserGroupProvider
[ https://issues.apache.org/jira/browse/NIFI-4127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-4127: - Resolution: Fixed Fix Version/s: 1.4.0 Status: Resolved (was: Patch Available) > Create a CompositeUserGroupProvider > --- > > Key: NIFI-4127 > URL: https://issues.apache.org/jira/browse/NIFI-4127 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.4.0 > > > Create a CompositeUserGroupProvider to support loading users/groups from > multiple sources. This composite implementation should support > {noformat} > 0-1 ConfigurableUserGroupProvider > 0-n UserGroupProviders > {noformat} > Only a single ConfigurableUserGroupProvider can be supplied to keep these > sources/implementation details hidden from the end users. The > CompositeUserGroupProvider must be configured with at least 1 underlying > provider. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-1763) Provide an integration with 'Schema registry for Kafka'
[ https://issues.apache.org/jira/browse/NIFI-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-1763: - Resolution: Fixed Fix Version/s: 1.4.0 Status: Resolved (was: Patch Available) > Provide an integration with 'Schema registry for Kafka' > --- > > Key: NIFI-1763 > URL: https://issues.apache.org/jira/browse/NIFI-1763 > Project: Apache NiFi > Issue Type: Wish > Components: Extensions >Reporter: Joseph Witt >Assignee: Mark Payne >Priority: Minor > Fix For: 1.4.0 > > > Reported on a mailing list question on 13 April 2016 > https://github.com/confluentinc/schema-registry > The registry itself is an ASLv2 licensed codebase. It offers a REST-based > Web API for interaction. It would be good to support integration with it for > users of Kafka so it would register schemas if needed when writing to Kafka > and understand how to parse data based on the indicated schema when reading > from Kafka. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFIREG-4) Add new logo to Registry web page
[ https://issues.apache.org/jira/browse/NIFIREG-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082450#comment-16082450 ] ASF GitHub Bot commented on NIFIREG-4: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-site/pull/20 > Add new logo to Registry web page > - > > Key: NIFIREG-4 > URL: https://issues.apache.org/jira/browse/NIFIREG-4 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Minor > > A new Registry logo is available > (https://issues.apache.org/jira/browse/NIFIREG-2) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4127) Create a CompositeUserGroupProvider
[ https://issues.apache.org/jira/browse/NIFI-4127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082446#comment-16082446 ] ASF GitHub Bot commented on NIFI-4127: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1978 > Create a CompositeUserGroupProvider > --- > > Key: NIFI-4127 > URL: https://issues.apache.org/jira/browse/NIFI-4127 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.4.0 > > > Create a CompositeUserGroupProvider to support loading users/groups from > multiple sources. This composite implementation should support > {noformat} > 0-1 ConfigurableUserGroupProvider > 0-n UserGroupProviders > {noformat} > Only a single ConfigurableUserGroupProvider can be supplied to keep these > sources/implementation details hidden from the end users. The > CompositeUserGroupProvider must be configured with at least 1 underlying > provider. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-site pull request #20: NIFIREG-4 Add/Update logos on Registry and MiNiF...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-site/pull/20 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4127) Create a CompositeUserGroupProvider
[ https://issues.apache.org/jira/browse/NIFI-4127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082445#comment-16082445 ] ASF subversion and git services commented on NIFI-4127: --- Commit eefad291674905b6e6df078f6bf07f7526ddc575 in nifi's branch refs/heads/master from [~mcgilman] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=eefad29 ] NIFI-4127: - Introducing composite ConfigurableUserGroupProvider and UserGroupProvider. - Adding appropriate unit tests. - Updating object model to support per resource (user/group/policy) configuration. - Updating UI to support per resource (user/group/policy) configuration. - Adding necessary documentation. - Updating documentation to clarify integrity checks. - Providing an example of configuring a composite implementation. Signed-off-by: Pierre VillardThis closes #1978. > Create a CompositeUserGroupProvider > --- > > Key: NIFI-4127 > URL: https://issues.apache.org/jira/browse/NIFI-4127 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.4.0 > > > Create a CompositeUserGroupProvider to support loading users/groups from > multiple sources. This composite implementation should support > {noformat} > 0-1 ConfigurableUserGroupProvider > 0-n UserGroupProviders > {noformat} > Only a single ConfigurableUserGroupProvider can be supplied to keep these > sources/implementation details hidden from the end users. The > CompositeUserGroupProvider must be configured with at least 1 underlying > provider. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #1978: NIFI-4127: Composite User Group Providers
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1978 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (NIFI-3677) ExtracMediaMetadata should close TikaInputStream or use Tika TemporaryResources
[ https://issues.apache.org/jira/browse/NIFI-3677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-3677. -- Resolution: Fixed Fix Version/s: 1.4.0 > ExtracMediaMetadata should close TikaInputStream or use Tika > TemporaryResources > --- > > Key: NIFI-3677 > URL: https://issues.apache.org/jira/browse/NIFI-3677 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 0.8.0, 1.2.0 >Reporter: Joe Skora >Assignee: Joe Skora >Priority: Minor > Fix For: 1.4.0 > > > Per Tika docs\[1], {{ExtractMediaMetadata}} should cleanup the > {{TikaInputStream}} it creates or use a Tika's {{TemporaryResources}} class > for automatic cleanup. > \[1]https://tika.apache.org/1.8/api/org/apache/tika/io/TikaInputStream.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4127) Create a CompositeUserGroupProvider
[ https://issues.apache.org/jira/browse/NIFI-4127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082440#comment-16082440 ] ASF GitHub Bot commented on NIFI-4127: -- Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/1978 +1, thanks @mcgilman, I'll squash and merge to master. > Create a CompositeUserGroupProvider > --- > > Key: NIFI-4127 > URL: https://issues.apache.org/jira/browse/NIFI-4127 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Gilman >Assignee: Matt Gilman > > Create a CompositeUserGroupProvider to support loading users/groups from > multiple sources. This composite implementation should support > {noformat} > 0-1 ConfigurableUserGroupProvider > 0-n UserGroupProviders > {noformat} > Only a single ConfigurableUserGroupProvider can be supplied to keep these > sources/implementation details hidden from the end users. The > CompositeUserGroupProvider must be configured with at least 1 underlying > provider. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #1978: NIFI-4127: Composite User Group Providers
Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/1978 +1, thanks @mcgilman, I'll squash and merge to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3677) ExtracMediaMetadata should close TikaInputStream or use Tika TemporaryResources
[ https://issues.apache.org/jira/browse/NIFI-3677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082435#comment-16082435 ] ASF subversion and git services commented on NIFI-3677: --- Commit 325fe53fa1bdb03b89a88a3c4c74a3623aaea6ec in nifi's branch refs/heads/master from [~jskora] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=325fe53 ] NIFI-3677 - ExtractMediaMetadata should close TikaInputStream * Added finally block to close TikaInputStream. Signed-off-by: Pierre VillardThis closes #1993. > ExtracMediaMetadata should close TikaInputStream or use Tika > TemporaryResources > --- > > Key: NIFI-3677 > URL: https://issues.apache.org/jira/browse/NIFI-3677 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.8.0, 1.2.0 >Reporter: Joe Skora >Assignee: Joe Skora >Priority: Minor > > Per Tika docs\[1], {{ExtractMediaMetadata}} should cleanup the > {{TikaInputStream}} it creates or use a Tika's {{TemporaryResources}} class > for automatic cleanup. > \[1]https://tika.apache.org/1.8/api/org/apache/tika/io/TikaInputStream.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-3677) ExtracMediaMetadata should close TikaInputStream or use Tika TemporaryResources
[ https://issues.apache.org/jira/browse/NIFI-3677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082436#comment-16082436 ] ASF GitHub Bot commented on NIFI-3677: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1993 > ExtracMediaMetadata should close TikaInputStream or use Tika > TemporaryResources > --- > > Key: NIFI-3677 > URL: https://issues.apache.org/jira/browse/NIFI-3677 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.8.0, 1.2.0 >Reporter: Joe Skora >Assignee: Joe Skora >Priority: Minor > > Per Tika docs\[1], {{ExtractMediaMetadata}} should cleanup the > {{TikaInputStream}} it creates or use a Tika's {{TemporaryResources}} class > for automatic cleanup. > \[1]https://tika.apache.org/1.8/api/org/apache/tika/io/TikaInputStream.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #1993: NIFI-3677 - ExtractMediaMetadata should close TikaI...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1993 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3677) ExtracMediaMetadata should close TikaInputStream or use Tika TemporaryResources
[ https://issues.apache.org/jira/browse/NIFI-3677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082434#comment-16082434 ] ASF GitHub Bot commented on NIFI-3677: -- Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/1993 +1, merging to master, thanks @jskora > ExtracMediaMetadata should close TikaInputStream or use Tika > TemporaryResources > --- > > Key: NIFI-3677 > URL: https://issues.apache.org/jira/browse/NIFI-3677 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.8.0, 1.2.0 >Reporter: Joe Skora >Assignee: Joe Skora >Priority: Minor > > Per Tika docs\[1], {{ExtractMediaMetadata}} should cleanup the > {{TikaInputStream}} it creates or use a Tika's {{TemporaryResources}} class > for automatic cleanup. > \[1]https://tika.apache.org/1.8/api/org/apache/tika/io/TikaInputStream.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #1993: NIFI-3677 - ExtractMediaMetadata should close TikaInputStr...
Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/1993 +1, merging to master, thanks @jskora --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4060) Create a MergeRecord Processor
[ https://issues.apache.org/jira/browse/NIFI-4060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082342#comment-16082342 ] ASF GitHub Bot commented on NIFI-4060: -- Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/1958 @mattyb149 I did realize that after setting mime.type we were overwriting it with the mime.type attribute of incoming FlowFiles. Sorry about that. I also found and fixed a threading issue in the RecordBinManager and improved the documentation around the Processor. I have pushed another commit that handles all of this. Please give it another review if you are able. > Create a MergeRecord Processor > -- > > Key: NIFI-4060 > URL: https://issues.apache.org/jira/browse/NIFI-4060 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.4.0 > > > When record-oriented data is received one record or a time or needs to be > split into small chunks for one reason or another, it will be helpful to be > able to combine those records into a single FlowFile that is made up of many > records for efficiency purposes, or to deliver to downstream systems as > larger batches. This processor should function similarly to MergeContent but > make use of Record Readers and Record Writer so that users don't have to deal > with headers, footers, demarcators, etc. > The Processor will also need to ensure that records only get merge into the > same FlowFile if they have compatible schemas. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #1958: NIFI-4060: Initial implementation of MergeRecord
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/1958 @mattyb149 I did realize that after setting mime.type we were overwriting it with the mime.type attribute of incoming FlowFiles. Sorry about that. I also found and fixed a threading issue in the RecordBinManager and improved the documentation around the Processor. I have pushed another commit that handles all of this. Please give it another review if you are able. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (NIFI-4174) GenerateTableFetch does not work with oracle on Nifi 1.2
Jorge Machado created NIFI-4174: --- Summary: GenerateTableFetch does not work with oracle on Nifi 1.2 Key: NIFI-4174 URL: https://issues.apache.org/jira/browse/NIFI-4174 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.2.0 Reporter: Jorge Machado Priority: Minor I'm trying to extract some data from a oracle DB. I'm getting : {code:java} 2017-07-11 16:19:29,612 WARN [StandardProcessScheduler Thread-7] o.a.n.controller.StandardProcessorNode Timed out while waiting for OnScheduled of 'GenerateTableFetch' processor to finish. An attempt is made to cancel the task via Thread.interrupt(). However it does not guarantee that the task will be canceled since the code inside current OnScheduled operation may have been written to ignore interrupts which may result in a runaway thread. This could lead to more issues, eventually requiring NiFi to be restarted. This is usually a bug in the target Processor 'GenerateTableFetch[id=f08a3acd-ac7e-17d7-598b-8f9720fd92d4]' that needs to be documented, reported and eventually fixed. 2017-07-11 16:19:29,612 ERROR [StandardProcessScheduler Thread-7] o.a.n.p.standard.GenerateTableFetch GenerateTableFetch[id=f08a3acd-ac7e-17d7-598b-8f9720fd92d4] GenerateTableFetch[id=f08a3acd-ac7e-17d7-598b-8f9720fd92d4] failed to invoke @OnScheduled method due to java.lang.RuntimeException: Timed out while executing one of processor's OnScheduled task.; processor will not be scheduled to run for 30 seconds: java.lang.RuntimeException: Timed out while executing one of processor's OnScheduled task. java.lang.RuntimeException: Timed out while executing one of processor's OnScheduled task. at org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1480) at org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:102) at org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1303) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: java.util.concurrent.TimeoutException: null at java.util.concurrent.FutureTask.get(FutureTask.java:205) at org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1465) ... 9 common frames omitted 2017-07-11 16:19:29,613 ERROR [StandardProcessScheduler Thread-7] o.a.n.controller.StandardProcessorNode Failed to invoke @OnScheduled method due to java.lang.RuntimeException: Timed out while executing one of processor's OnScheduled task. java.lang.RuntimeException: Timed out while executing one of processor's OnScheduled task. at org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1480) at org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:102) at org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1303) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: java.util.concurrent.TimeoutException: null at java.util.concurrent.FutureTask.get(FutureTask.java:205) at org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1465) ... 9 common frames omitted {code} Database Connection Pooling Service: jdbc:oracle:thin:@somehost:6779:someSID oracle.jdbc.OracleDriver /pathTo/ojdbc7.jar max wait time 500mil On the processor I have: Max Wait Time 10 seconds Partition Size 100 i tryed schedule 0s and 10 000s result is the same -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-1586) embedded zookeeper disk utilization grows unbounded
[ https://issues.apache.org/jira/browse/NIFI-1586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-1586: -- Resolution: Fixed Status: Resolved (was: Patch Available) > embedded zookeeper disk utilization grows unbounded > --- > > Key: NIFI-1586 > URL: https://issues.apache.org/jira/browse/NIFI-1586 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.3.0 > Environment: latest 0.5.1 release >Reporter: Matthew Clarke >Assignee: Jeff Storck > Fix For: 1.4.0 > > Attachments: ZK_Autopurge_Test.xml > > > Observed that embedded NiFi zookeeper disk utilization will grow unbounded. > Zookeeper will occasional create snapshots but at no time will it ever purge > any of those snapshots it creates. This behavior is documented here: > https://zookeeper.apache.org/doc/r3.1.2/zookeeperAdmin.html#sc_administering > It is the operators responsibility to purge old snapshot files. NiFi needs to > provide a configuration that will automate this pruge. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-1586) embedded zookeeper disk utilization grows unbounded
[ https://issues.apache.org/jira/browse/NIFI-1586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082243#comment-16082243 ] ASF subversion and git services commented on NIFI-1586: --- Commit 58fc5f24b8f8256846b0782f17b3b090c841f94b in nifi's branch refs/heads/master from [~jtstorck] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=58fc5f2 ] NIFI-1586 Removed check for distributed ZK quorum before starting the DatadirCleanupMananger to enable autopurge during standalone ZK server usage. This closes #2001 > embedded zookeeper disk utilization grows unbounded > --- > > Key: NIFI-1586 > URL: https://issues.apache.org/jira/browse/NIFI-1586 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.3.0 > Environment: latest 0.5.1 release >Reporter: Matthew Clarke >Assignee: Jeff Storck > Fix For: 1.4.0 > > Attachments: ZK_Autopurge_Test.xml > > > Observed that embedded NiFi zookeeper disk utilization will grow unbounded. > Zookeeper will occasional create snapshots but at no time will it ever purge > any of those snapshots it creates. This behavior is documented here: > https://zookeeper.apache.org/doc/r3.1.2/zookeeperAdmin.html#sc_administering > It is the operators responsibility to purge old snapshot files. NiFi needs to > provide a configuration that will automate this pruge. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-1586) embedded zookeeper disk utilization grows unbounded
[ https://issues.apache.org/jira/browse/NIFI-1586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082245#comment-16082245 ] ASF GitHub Bot commented on NIFI-1586: -- Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2001 Thanks @jtstorck! This has been merged to master. > embedded zookeeper disk utilization grows unbounded > --- > > Key: NIFI-1586 > URL: https://issues.apache.org/jira/browse/NIFI-1586 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.3.0 > Environment: latest 0.5.1 release >Reporter: Matthew Clarke >Assignee: Jeff Storck > Fix For: 1.4.0 > > Attachments: ZK_Autopurge_Test.xml > > > Observed that embedded NiFi zookeeper disk utilization will grow unbounded. > Zookeeper will occasional create snapshots but at no time will it ever purge > any of those snapshots it creates. This behavior is documented here: > https://zookeeper.apache.org/doc/r3.1.2/zookeeperAdmin.html#sc_administering > It is the operators responsibility to purge old snapshot files. NiFi needs to > provide a configuration that will automate this pruge. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-1586) embedded zookeeper disk utilization grows unbounded
[ https://issues.apache.org/jira/browse/NIFI-1586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082246#comment-16082246 ] ASF GitHub Bot commented on NIFI-1586: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2001 > embedded zookeeper disk utilization grows unbounded > --- > > Key: NIFI-1586 > URL: https://issues.apache.org/jira/browse/NIFI-1586 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.3.0 > Environment: latest 0.5.1 release >Reporter: Matthew Clarke >Assignee: Jeff Storck > Fix For: 1.4.0 > > Attachments: ZK_Autopurge_Test.xml > > > Observed that embedded NiFi zookeeper disk utilization will grow unbounded. > Zookeeper will occasional create snapshots but at no time will it ever purge > any of those snapshots it creates. This behavior is documented here: > https://zookeeper.apache.org/doc/r3.1.2/zookeeperAdmin.html#sc_administering > It is the operators responsibility to purge old snapshot files. NiFi needs to > provide a configuration that will automate this pruge. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2001: NIFI-1586 Removed check for distributed ZK quorum before s...
Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2001 Thanks @jtstorck! This has been merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #2001: NIFI-1586 Removed check for distributed ZK quorum b...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2001 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3218) MockProcessSession should prevent transferring new FlowFile to input queue
[ https://issues.apache.org/jira/browse/NIFI-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082224#comment-16082224 ] ASF subversion and git services commented on NIFI-3218: --- Commit 58f60b3cf0430e1ca5141c550595cd827b7e4b3e in nifi's branch refs/heads/master from m-hogue [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=58f60b3 ] NIFI-3218: throw exception in MockProcessSession when transferring file to self > MockProcessSession should prevent transferring new FlowFile to input queue > -- > > Key: NIFI-3218 > URL: https://issues.apache.org/jira/browse/NIFI-3218 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.1.0, 0.8.0 >Reporter: Joe Skora >Assignee: Michael Hogue > > StandardProcessSession.transfer() throws an exception if called with a newly > created FlowFile and no relationship. MockProcessionSession should behave > similarly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-3218) MockProcessSession should prevent transferring new FlowFile to input queue
[ https://issues.apache.org/jira/browse/NIFI-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082226#comment-16082226 ] ASF GitHub Bot commented on NIFI-3218: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1988 > MockProcessSession should prevent transferring new FlowFile to input queue > -- > > Key: NIFI-3218 > URL: https://issues.apache.org/jira/browse/NIFI-3218 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.1.0, 0.8.0 >Reporter: Joe Skora >Assignee: Michael Hogue > > StandardProcessSession.transfer() throws an exception if called with a newly > created FlowFile and no relationship. MockProcessionSession should behave > similarly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-3218) MockProcessSession should prevent transferring new FlowFile to input queue
[ https://issues.apache.org/jira/browse/NIFI-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082225#comment-16082225 ] ASF subversion and git services commented on NIFI-3218: --- Commit 07a6499dae93344b0e8a8e8cd890ba16cf667fe2 in nifi's branch refs/heads/master from m-hogue [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=07a6499 ] NIFI-3218: fixed incorrect assertion for self-routing flowfiles in MockProcessSession Signed-off-by: Joe SkoraThis closes #1988. > MockProcessSession should prevent transferring new FlowFile to input queue > -- > > Key: NIFI-3218 > URL: https://issues.apache.org/jira/browse/NIFI-3218 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.1.0, 0.8.0 >Reporter: Joe Skora >Assignee: Michael Hogue > > StandardProcessSession.transfer() throws an exception if called with a newly > created FlowFile and no relationship. MockProcessionSession should behave > similarly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #1988: NIFI-3218: throw exception in MockProcessSession wh...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1988 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (NIFI-4173) Processor to RemoveCache from DistributedMapCacheServer
Gabriel Queiroz created NIFI-4173: - Summary: Processor to RemoveCache from DistributedMapCacheServer Key: NIFI-4173 URL: https://issues.apache.org/jira/browse/NIFI-4173 Project: Apache NiFi Issue Type: Improvement Reporter: Gabriel Queiroz Attachments: Detect_Duplicate_V2_-_With_Remove_Cache.xml, RemoveCacheExample.xml I posted a question in hortonworks community asking for a way to remove a cached identifier from DistributedMapCacheServer (https://community.hortonworks.com/questions/110551/how-to-remove-a-cache-entry-identifier-from-distri.html) and the user *kkawamura* answered the question and asked me to open this issue requesting this funcionality. I attached two examples using the RemoveCache processor created by kkawamura using InvokeScriptedProcessor. Thank you very much! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082135#comment-16082135 ] ASF GitHub Bot commented on NIFI-1613: -- Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1976#discussion_r126673168 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml --- @@ -467,6 +467,7 @@ src/test/resources/TestUpdateRecord/output/person-with-firstname.json src/test/resources/TestUpdateRecord/output/person-with-firstname-lastname.json src/test/resources/TestUpdateRecord/output/person-with-capital-lastname.json + src/test/resources/TestUpdateRecord/output/person-with-bool.json --- End diff -- This should be in the TestConvertJSONToSQL section rather than TestUpdateRecord, it causes a RAT failure > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #1976: NIFI-1613: Make use of column type correctly at Con...
Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1976#discussion_r126673168 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml --- @@ -467,6 +467,7 @@ src/test/resources/TestUpdateRecord/output/person-with-firstname.json src/test/resources/TestUpdateRecord/output/person-with-firstname-lastname.json src/test/resources/TestUpdateRecord/output/person-with-capital-lastname.json + src/test/resources/TestUpdateRecord/output/person-with-bool.json --- End diff -- This should be in the TestConvertJSONToSQL section rather than TestUpdateRecord, it causes a RAT failure --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---