Hi Pat, Looks to me that on your PutKafka you are pointing it to your zookeeper instance port 2181 you should point it to your kafka broker. Port 9092 ( I think ) Not home to verify ...
On Sat, Jun 11, 2016 at 3:31 PM, Pat Trainor <[email protected]> wrote: > *OK... Prep'ing kafka:* > > pat@wopr:/opt/kafka$ bin/kafka-server-stop.sh > pat@wopr:/opt/kafka$ bin/zookeeper-server-stop.sh > pat@wopr:/opt/kafka/logs$ rm log-cleaner.log controller.log server.log > kafkaServer-gc.log zookeeper-gc.log > > *Prep'ing nifi:* > > pat@wopr:/opt/nifi$ bin/nifi.sh stop > pat@wopr:/opt/nifi$ rm logs/nifi-bootstrap.log logs/nifi-app.log > logs/nifi-user.log > > *Starting kafka:* > > pat@wopr:/opt/kafka$ bin/zookeeper-server-start.sh > config/zookeeper.properties & > pat@wopr:/opt/kafka$ bin/kafka-server-start.sh config/server.properties & > > bin/kafka-topics.sh --create --zookeeper localhost:2181 > --replication-factor 1 --partitions 1 --topic test > *bin/kafka-topics.sh --create --zookeeper localhost:2181 > --replication-factor 1 --partitions 1 --topic fast-messages* > bin/kafka-topics.sh --create --zookeeper localhost:2181 > --replication-factor 1 --partitions 1 --topic inNLP > bin/kafka-topics.sh --create --zookeeper localhost:2181 > --replication-factor 1 --partitions 1 --topic outNLP > > The above topics were preserved. > > kafka.logs.tar.gz attached > > *Starting Nifi:* > > pat@wopr:/opt/nifi$ bin/nifi.sh start > > nifi.logs.tar.gz attached. > > Now to log onto Nifi GUI and start the 2 processes (GenerateFlowfile & > PutKafka ...to fast-messages topic): > > I really hope this helps... I can't believe I'm going to spend my entire > weekend onthis... :-( > > Thanks! > > pat <http://about.me/PatTrainor> > ( ͡° ͜ʖ ͡°) > > > "A wise man can learn more from a foolish *question *than a fool can > learn from a wise *answer*". ~ Bruce Lee. > > On Jun 11 2016, at 1:38 pm, Joe Witt <[email protected]> wrote: > >> Pat >> >> Could you please make sure we are seeing the full stacktrace from the app >> log? If we could get a full log that would he even better. >> >> Thanks >> Joe >> On Jun 11, 2016 1:17 PM, "Pat Trainor" <[email protected]> wrote: >> >> Andrew, >> >> Thanks for writing. >> >> *Nifi:* >> nifi-0.6.1 >> *Kafka:* >> kafka_2.11-0.10.0.0 >> >> startup with no changes to configs: >> >> # >> Kafka: >> bin/zookeeper-server-start.sh config/zookeeper.properties & >> bin/kafka-server-start.sh config/server.properties & >> # >> # create a topic >> # >> bin/kafka-topics.sh --create --zookeeper localhost:2181 >> --replication-factor 1 --partitions 1 --topic test >> [...] >> >> *Nifi's zookeeper.properties:* >> >> >> clientPort=2181 >> initLimit=10 >> autopurge.purgeInterval=24 >> syncLimit=5 >> tickTime=2000 >> dataDir=./state/zookeeper >> autopurge.snapRetainCount=30 >> >> # >> # server.1=nifi-node1-hostname:2888:3888 >> # server.2=nifi-node2-hostname:2888:3888 >> # server.3=nifi-node3-hostname:2888:3888 >> # >> >> *server.1=* >> >> ^^^^^^ >> Even though I have: >> >> >> *nifi.state.management.embedded.zookeeper.start=false* >> >> I filled in the above setting... changing it to: >> >> >> *server.1=localhost:2888:3888* >> >> ...which made no difference, of course... >> >> Any thoughts? >> >> Thanks! >> >> pat <http://about.me/PatTrainor> >> ( ͡° ͜ʖ ͡°) >> >> On Jun 11 2016, at 12:17 pm, Andrew Grande <[email protected]> wrote: >> >> Pat, which NiFi version is that? Which Kafka broker version on the other >> end? >> >> Andrew >> >> On Sat, Jun 11, 2016 at 12:15 PM Pat Trainor <[email protected]> >> wrote: >> >> Users, >> >> Has anyone come across PutKafka simply not wanting to send to Kafka? >> >> In this troubleshooting, I have replicated symptoms in the simplest way I >> can-with a GenerateFlowFile and a PutKafka: >> >> >> With a known good Kafka running (other /producers using the same Topic)... >> >> ...yields: >> Auto refresh started >> 11:17:19 EDT WARNING >> 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] >> Processor Administratively Yielded for 1 sec due to processing failure >> 11:18:00 EDT ERROR >> 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] >> Failed while waiting for acks from Kafka >> 11:18:00 EDT ERROR >> 2b76634c-6354-4c3c-b977-86f73d64efbcPutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] >> PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] failed to process due to >> java.lang.NullPointerException; rolling back session: >> java.lang.NullPointerException >> 11:18:00 EDT ERROR 2b76634c-6354-4c3c-b977-86f73d64efbc >> >> PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] >> PutKafka[id=2b76634c-6354-4c3c-b977-86f73d64efbc] failed to process session >> due to java.lang.NullPointerException: java.lang.NullPointerException >> >> Everything is super-stock, out of the box, and not customized. In fact, I >> tested Kafka with the Apache test consumer/producer pair, and am trying to >> send to the same topic, same consumer, still running, after it worked for >> the test producer. >> >> Funny thing is, I see this error in forums/emails lately, but nobody >> nails it down as to why it is occurring... >> >> ==> nifi-app.log <== >> 2016-06-11 12:04:24,172 WARN [kafka-producer-network-thread | >> SecondPutKafka2k] org.apache.kafka.common.network.Selector Error in I/O >> with localhost/127.0.0.1 >> java.io.EOFException: null >> at >> org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:62) >> ~[kafka-clients-0.8.2.2.jar:na] >> at org.apache.kafka.common.network.Selector.poll(Selector.java:248) >> ~[kafka-clients-0.8.2.2.jar:na] >> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) >> [kafka-clients-0.8.2.2.jar:na] >> at >> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191) >> [kafka-clients-0.8.2.2.jar:na] >> at >> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122) >> [kafka-clients-0.8.2.2.jar:na] >> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] >> >> attached has screenshots... >> >> I've slowed down the 'schedules', messes with Message Delimiter, and what >> seems like all the obvious things... This makes me think it is obvious, so >> on day 2, I'm reaching out to see if any of you have seen/fixed this... >> >> Thanks! >> >> pat <http://about.me/PatTrainor> >> ( ͡° ͜ʖ ͡°) >> >> -- Juan Carlos Sequeiros
