Yes. As soon as I fire up zookeeper & PutKafka, I get zillions of these streaming by in the log:
[2016-06-12 10:24:52,267] INFO Closed socket connection for client /127.0.0.1:59958 (no session established for client) (org.apache.zookeeper.server.NIOServerCnxn) [2016-06-12 10:24:52,369] INFO Accepted socket connection from /127.0.0.1:59960 (org.apache.zookeeper.server.NIOServerCnxnFactory) [2016-06-12 10:24:52,369] WARN Exception causing close of session 0x0 due to java.io.IOException: Unreasonable length = 845873152 (org.apache.zookeeper.server.NIOServerCnxn) I will do as Juan Sequeiros (Hello Juan!) ... ugh... I'm hanging my head the way we programmers do when we've gone a day or more trying to figure out what is stopping us from continuing, and then find out it is not the cosmos that is against us, but out own naivete'/stupidite'... When I put my hostname in (saw that it worked better than localhost in archives), and changed my PutKafka port from 2181: localhost:2181 ---> wopr:9092 **Everything chirped up and gave me a raspberry...** My only defense is that I see port 2181 in all the examples... And that isn't a very good excuse... Thank you to everyone who pitched in and threw ideas out. Thanks! [pat](http://about.me/PatTrainor) ( ͡° ͜ʖ ͡°) "A wise man can learn more from a foolish _question _than a fool can learn from a wise _answer_". ~ Bruce Lee. On Jun 11 2016, at 5:17 pm, Oleg Zhurakousky <[email protected]> wrote: > Also, just looked ta the logs and what I see is that you have some type of connection problem to Kafka. > > > > 2016-06-11 15:23:06,193 WARN [kafka-producer-network-thread | SecondPutKafka2k] org.apache.kafka.common.network.Selector Error in I/O with localhost/127.0.0.1 > > java.io.EOFException: null > > at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.ja va:62) ~[kafka-clients-0.8.2.2.jar:na] > > at org.apache.kafka.common.network.Selector.poll(Selector.java:248) ~[kafka- clients-0.8.2.2.jar:na] > > at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) [kafka-clients-0.8.2.2.jar:na] > > at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191) [kafka-clients-0.8.2.2.jar:na] > > at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122) [kafka-clients-0.8.2.2.jar:na] > > > > The above is coming from Kafka (not NIFi). Are you sure that you have the correct IP:PORT in your producer config? One quick diagnostic you can do is to try to send message to Kafka from the command line on the same machine as NiFi. Basically this: > > > > $> bin/kafka-console-producer.sh --broker-list <host>:<port> \--topic <topic> > > > > Let me know > > Cheers > > Oleg > > > >> On Jun 11, 2016, at 5:03 PM, Oleg Zhurakousky <[[email protected]](mailto:[email protected])> wrote: >> >> >> >> Also, the NPE was a bug that was fixed a while back and will be available on O.7. >> >> Any chance you can build from 0.x branch? >> >> >> >> Cheers >> >> Oleg Sent from my iPhone >> >> On Jun 11, 2016, at 16:58, Juan Sequeiros <[[email protected]](mailto:[email protected])> wrote: >> >>> Hi Pat, >>> >>> >>> >>> Looks to me that on your PutKafka you are pointing it to your zookeeper instance port 2181 you should point it to your kafka broker. Port 9092 ( I think ) >>> >>> Not home to verify ... >>> >>> >>> >>> On Sat, Jun 11, 2016 at 3:31 PM, Pat Trainor <[[email protected]](mailto:[email protected])> wrote: >>> >>>> _OK... Prep'ing kafka:_ >>>> >>>> >>>> >>>> pat@wopr:/opt/kafka$ bin/kafka-server-stop.sh >>>> >>>> pat@wopr:/opt/kafka$ bin/zookeeper-server-stop.sh >>>> >>>> pat@wopr:/opt/kafka/logs$ rm log-cleaner.log controller.log server.log kafkaServer-gc.log zookeeper-gc.log >>>> >>>> >>>> >>>> _Prep'ing nifi:_ >>>> >>>> >>>> >>>> pat@wopr:/opt/nifi$ bin/nifi.sh stop >>>> >>>> pat@wopr:/opt/nifi$ rm logs/nifi-bootstrap.log logs/nifi-app.log logs /nifi-user.log >>>> >>>> >>>> >>>> _Starting kafka:_ >>>> >>>> >>>> >>>> pat@wopr:/opt/kafka$ bin/zookeeper-server-start.sh config/zookeeper.properties & >>>> >>>> pat@wopr:/opt/kafka$ bin/kafka-server-start.sh config/server.properties & >>>> >>>> >>>> >>>> bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication- factor 1 --partitions 1 --topic test **bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic fast-messages** bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic inNLP bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic outNLP >>>> >>>> >>>> >>>> The above topics were preserved. >>>> >>>> >>>> >>>> kafka.logs.tar.gz attached >>>> >>>> >>>> >>>> _Starting Nifi:_ >>>> >>>> >>>> >>>> pat@wopr:/opt/nifi$ bin/nifi.sh start >>>> >>>> >>>> >>>> nifi.logs.tar.gz attached. >>>> >>>> >>>> >>>> Now to log onto Nifi GUI and start the 2 processes (GenerateFlowfile & PutKafka ...to fast-messages topic): >>>> >>>> >>>> >>>> I really hope this helps... I can't believe I'm going to spend my entire weekend onthis... :-( >>>> >>>> >>>> >>>> Thanks! >>>> >>>> [pat](http://about.me/PatTrainor) >>>> >>>> ( ͡° ͜ʖ ͡°) >>>> >>>> >>>> >>>> "A wise man can learn more from a foolish _question _than a fool can learn from a wise _answer_". ~ Bruce Lee. >>>> >>>> >>>> >>>> On Jun 11 2016, at 1:38 pm, Joe Witt <[[email protected]](mailto:[email protected])> wrote: >>>> >>>>> Pat >>>>> >>>>> Could you please make sure we are seeing the full stacktrace from the app log? If we could get a full log that would he even better. >>>>> >>>>> Thanks Joe >>>>> >>>>> On Jun 11, 2016 1:17 PM, "Pat Trainor" <[[email protected]](mailto:[email protected])> wrote: >>>>> >>>>>> Andrew, >>>>>> >>>>>> >>>>>> >>>>>> Thanks for writing. >>>>>> >>>>>> >>>>>> >>>>>> _Nifi:_ >>>>>> >>>>>> nifi-0.6.1 >>>>>> >>>>>> _Kafka:_ >>>>>> >>>>>> kafka_2.11-0.10.0.0 >>>>>> >>>>>> >>>>>> >>>>>> startup with no changes to configs: >>>>>> >>>>>> >>>>>> >>>>>> # >>>>>> >>>>>> Kafka: >>>>>> >>>>>> bin/zookeeper-server-start.sh config/zookeeper.properties & bin/kafka-server-start.sh config/server.properties & # # create a topic # bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test >>>>>> >>>>>> [...] >>>>>> >>>>>> >>>>>> >>>>>> _Nifi's zookeeper.properties:_ >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> clientPort=2181 initLimit=10 autopurge.purgeInterval=24 syncLimit=5 tickTime=2000 dataDir=./state/zookeeper autopurge.snapRetainCount=30 # # server.1=nifi-node1-hostname:2888:3888 # server.2=nifi-node2-hostname:2888:3888 # server.3=nifi-node3-hostname:2888:3888 # **server.1= ** >>>>>> >>>>>> >>>>>> >>>>>> ^^^^^^ >>>>>> >>>>>> Even though I have: >>>>>> >>>>>> >>>>>> >>>>>> _nifi.state.management.embedded.zookeeper.start=**false** _ >>>>>> >>>>>> >>>>>> >>>>>> I filled in the above setting... changing it to: >>>>>> >>>>>> >>>>>> >>>>>> **server.1=localhost:2888:3888 ** >>>>>> >>>>>> >>>>>> >>>>>> ...which made no difference, of course... >>>>>> >>>>>> >>>>>> >>>>>> Any thoughts? >>>>>> >>>>>> >>>>>> >>>>>> Thanks! >>>>>> >>>>>> [pat](http://about.me/PatTrainor) >>>>>> >>>>>> ( ͡° ͜ʖ ͡°) >>>>>> >>>>>> On Jun 11 2016, at 12:17 pm, Andrew Grande <[[email protected]](mailto:[email protected])> wrote: >>>>>> >>>>>>> Pat, which NiFi version is that? Which Kafka broker version on the other end? >>>>>>> >>>>>>> >>>>>>> >>>>>>> Andrew >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Sat, Jun 11, 2016 at 12:15 PM Pat Trainor <[[email protected]](mailto:[email protected])> wrote: >>>>>>> >>>>>>>> Users, >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Has anyone come across PutKafka simply not wanting to send to Kafka? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> In this troubleshooting, I have replicated symptoms in the simplest way I can-with a GenerateFlowFile and a PutKafka: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>  >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> With a known good Kafka running (other /producers using the same Topic)... >>>>>>>> >>>>>>>> ...yields: >>>>>>>> >>>>>>>> Auto refresh started >>>>>>>> >>>>>>>> 11:17:19 EDT WARNING 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id= 2b76634c-6354-4c3c-b977-86f73d64efbc] Processor Administratively Yielded for 1 sec due to processing failure >>>>>>>> >>>>>>>> 11:18:00 EDT ERROR 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b 76634c-6354-4c3c-b977-86f73d64efbc] Failed while waiting for acks from Kafka >>>>>>>> >>>>>>>> 11:18:00 EDT ERROR 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b 76634c-6354-4c3c-b977-86f73d64efbc] PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] failed to process due to java.lang.NullPointerException; rolling back session: java.lang.NullPointerException >>>>>>>> >>>>>>>> 11:18:00 EDT ERROR 2b76634c-6354-4c3c-b977-86f73d64efbc >>>>>>>> >>>>>>>> PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] failed to process session due to java.lang.NullPointerException: java.lang.NullPointerException >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Everything is super-stock, out of the box, and not customized. In fact, I tested Kafka with the Apache test consumer/producer pair, and am trying to send to the same topic, same consumer, still running, after it worked for the test producer. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Funny thing is, I see this error in forums/emails lately, but nobody nails it down as to why it is occurring... >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ==> nifi-app.log <== 2016-06-11 12:04:24,172 WARN [kafka-producer-network-thread | SecondPutKafka2k] org.apache.kafka.common.network.Selector Error in I/O with localhost/[127.0.0.1](http://127.0.0.1/) java.io.EOFException: null at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java :62) ~[kafka-clients-0.8.2.2.jar:na] at org.apache.kafka.common.network.Selector.poll(Selector.java:248) ~[kafka- clients-0.8.2.2.jar:na] at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) [kafka- clients-0.8.2.2.jar:na] at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191) [kafka-clients-0.8.2.2.jar:na] at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122) [kafka-clients-0.8.2.2.jar:na] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> attached has screenshots... >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I've slowed down the 'schedules', messes with Message Delimiter, and what seems like all the obvious things... This makes me think it is obvious, so on day 2, I'm reaching out to see if any of you have seen/fixed this... >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>> [pat](http://about.me/PatTrainor) >>>>>>>> >>>>>>>> ( ͡° ͜ʖ ͡°) >>> >>> >>> >>> >>> >>> \-- >>> >>> Juan Carlos Sequeiros > >
