Re: Remote Processor Group error: Unable to communicate with remote NiFi cluster

2016-12-21 Thread Koji Kawamura
Hello Kevin, Sorry to hear that you got bitten by the issue, I think the 'Unable to find remote process group with id ...' is the same one I've encountered with NiFi 1.0.0. The issue has been solved since 1.1.0. https://issues.apache.org/jira/browse/NIFI-2687 So, if you're able to upgrade the clu

Remote Processor Group error: Unable to communicate with remote NiFi cluster

2016-12-21 Thread Kevin Verhoeven
I am running a cluster of twelve nodes on four VMs, each node runs on a different port. This configuration worked great on version 0.7. However, after an update from 0.7 to 1.0 we started seeing errors and flowfiles stopped moving into the Remote Processor Group. We are using the Remote Processo

Re: NiFi Cron scheduling

2016-12-21 Thread Joe Percivall
Totally forgot that UpdateAttribute with State was not in the previous release but is currently merged to master. You'd have to run 1.2.0-SNAPSHOT in order to do that workaround. Joe - - - - - -  Joseph Percivall linkedin.com/in/Percivall e: joeperciv...@yahoo.com On Wednesday, December 21, 2

Re: Flow seized, cannot list flow files for full queue

2016-12-21 Thread Bryan Bende
Thanks for the info. Since your Kafka broker is 0.10.1, I would be curious if you experience the same behavior switching to PublishKafka_0_10. The Kafka processors line up like this... GetKafka/PutKafka use the 0.8.x Kafka client ConsumeKafka/PublishKafka use the 0.9.x Kafka client ConsumeKafka_

Re: Flow seized, cannot list flow files for full queue

2016-12-21 Thread Nick Carenza
Hey Brian, Thanks for taking the time! - This is nifi 1.1.0. I had the same troubles on 1.0.0 and upgraded recently with the hope there was a fix for the issue. - Kafka is version 2.11-0.10.1.0 - I am using the PutKafka processor. - Nick On Wed, Dec 21, 2016 at 11:19 AM, Bryan Bende wrote: >

Re: Flow seized, cannot list flow files for full queue

2016-12-21 Thread Bryan Bende
Hey Nick, Sorry to hear about these troubles. A couple of questions... - What version of NiFi is this? - What version of Kafka are you using? - Which Kafka processor in NiFi are you using? It looks like PutKafka, but just confirming. Thanks, Bryan On Wed, Dec 21, 2016 at 2:00 PM, Nick Carenza

Re: NiFi Cron scheduling

2016-12-21 Thread Joe Percivall
I have created a JIRA here[1] for the issue. If you need a fix/workaround now (and are using the latest version), then you may want to utilize the newly added ability to use state with UpdateAttribute. You can set up a rule to remember the last time a FlowFile triggered the processor and if it

RE: Failing to Start NiFi 1.1.0, OverlappingFileLockException

2016-12-21 Thread Peter Wicks (pwicks)
Mark, We just got this working this morning. During a configuration file review late yesterday (we used a new config file and copied over settings) it was discovered that I hadn’t put in a Port number for Jetty to run on… nifi.web.https.port was left blank instead of providing a port number, an

Re: Executestream command - stream closed

2016-12-21 Thread Juan Sequeiros
ExecuteStreamCommand will "Executes an external command on the contents of a flow file" Looks like you want to format the output of your ExecuteSQL, instead I would suggest. ListDatabaseTables > ExecuteSQL > ReplaceText > {do something else with the flowfile} > PutHDFS / MergeContent / PutFile O

Re: NiFi Cron scheduling

2016-12-21 Thread Joe Percivall
Totally didn't realized I accidentally moved this off of the mailing list. Brining it back. -- I just realized as well that NiFi is only using Quartz for the CronExpression. I think this bug is due to the logic going on in this block[1]. One thing specifically that worries me is in the

Re: Failing to Start NiFi 1.1.0, OverlappingFileLockException

2016-12-21 Thread Mark Payne
Hey Peter, The FlowFile repository obtains a lock to ensure that no other process is using that directory. Getting an OverlappingFileLockException means that there is actually another process that has a lock. Can you verify that no other instance of NiFi is running on the node? If possible woul