Hey Brian, Thanks for taking the time!
- This is nifi 1.1.0. I had the same troubles on 1.0.0 and upgraded recently with the hope there was a fix for the issue. - Kafka is version 2.11-0.10.1.0 - I am using the PutKafka processor. - Nick On Wed, Dec 21, 2016 at 11:19 AM, Bryan Bende <[email protected]> wrote: > Hey Nick, > > Sorry to hear about these troubles. A couple of questions... > > - What version of NiFi is this? > - What version of Kafka are you using? > - Which Kafka processor in NiFi are you using? It looks like PutKafka, but > just confirming. > > Thanks, > > Bryan > > > On Wed, Dec 21, 2016 at 2:00 PM, Nick Carenza < > [email protected]> wrote: > >> I am running into an issue where a processor will stop receiving flow >> files from it's queue. >> >> flow: tcp --(100,000)--> evaljsonpath --(100,000)--> geoip >> --(100,000)--> putkafka >> >> This time, putkafka is the processor that has stopped receiving flowfiles​ >> >> I will try to list the queue and I'll get a message that says the queue >> has no flow files in it. I checked the http request and the response says >> there are 100,000 flow files in the queue but the flowFileSummaries array >> is empty. >> >> GET /nifi-api/flowfile-queues/1d72b81f-0159-1000-d09b-dc33e81b35 >>> c2/listing-requests/22754339-0159-1000-2dc9-07db09366132 HTTP/1.1 >>> { >>> "listingRequest": { >>> "id": "22754339-0159-1000-2dc9-07db09366132", >>> "uri": "http://ipaddress:8080/nifi-api/flowfile-queues/1d72b81f-015 >>> 9-1000-d09b-dc33e81b35c2/listing-requests/22754339-0159-1000 >>> -2dc9-07db09366132", >>> "submissionTime": "12/21/2016 17:37:07.385 UTC", >>> "lastUpdated": "17:37:07 UTC", >>> "percentCompleted": 100, >>> "finished": true, >>> "maxResults": 100, >>> "state": "Completed successfully", >>> "queueSize": { >>> "byteCount": 288609476, >>> "objectCount": 100000 >>> }, >>> "flowFileSummaries": [], >>> "sourceRunning": true, >>> "destinationRunning": true >>> } >>> } >> >> >> I tried stopping and starting all the processors, replacing the putkafka >> with a new duplicate putkafka processor and moving the queue over to it, >> restarting kafka itself. I ran a dump with all the processors "running". >> >> Since this is not running in a production environment, as a last resort I >> cleared the queue and then everything started flowing again. >> >> I have experienced this issue many times since I have begun evaluating >> Nifi. I have heard others having great success with it so I am convinced I >> have misconfigured something. I have tried to provide any relevant >> configuration information here: >> >> # nifi.properties >> nifi.version=1.1.0 >> nifi.flowcontroller.autoResumeState=true >> nifi.flowcontroller.graceful.shutdown.period=10 sec >> nifi.flowservice.writedelay.interval=500 ms >> nifi.administrative.yield.duration=30 sec >> nifi.bored.yield.duration=10 millis >> nifi.state.management.provider.local=local-provider >> nifi.swap.manager.implementation=org.apache.nifi.controller. >> FileSystemSwapManager >> nifi.queue.swap.threshold=1000 >> nifi.swap.in.period=5 sec >> nifi.swap.in.threads=1 >> nifi.swap.out.period=5 sec >> nifi.swap.out.threads=4 >> nifi.cluster.is.node=false >> nifi.build.tag=nifi-1.1.0-RC2 >> nifi.build.branch=NIFI-3100-rc2 >> nifi.build.revision=f61e42c >> nifi.build.timestamp=2016-11-26T04:39:37Z >> >> # JVM memory settings >> java.arg.2=-Xms28g >> java.arg.3=-Xmx28g >> java.arg.13=-XX:+UseG1GC >> >> controller settings: >> timer driven thread count: 10-20 (i have tried values from 10 to 20 and >> still experience the issue) >> event drive thread count: 5 (haven't touched) >> >> processors: >> concurrency: 1-20 (i have tried values from 1 to 20 and still experience >> the issue) >> scheduling: timer driven (run-schedule: 0 run-duration: 0) >> >> queues: >> backpressure flowfile count: 100,000 >> backpressure flowfile size: 1G >> >> machine: >> 128g ram >> 20 cpu >> disk: 3T >> >> --- >> >> Really I have 2 questions: >> >> 1. Why is this happening? >> 2. Once the flow is in this state, how can I get it flowing again without >> losing flowfiles? >> >> Thanks, >> Nick >> >> >> >
