[
https://issues.apache.org/jira/browse/NIFI-4682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16285948#comment-16285948
]
Joseph Witt commented on NIFI-4682:
-----------------------------------
[~clearlove] in looking at the provided stack trace all of the kafka clients
are sitting in their polling calls against the kafka broker. It is not clear
that they're actually stuck though. Are you looking at the lag indicators
showing that these clients are not keeping up or have lag? What command are
you running to verify that?
The warning you show when you stop these processors is just an indicator that
we told the kafka client to break out of the wait/poll cycle. This is expected
when a stop command is issued to the processor if it was in the middle of doing
that.
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-0-10-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/ConsumeKafka_0_10.java
There may be an issue with the client where it is actually stuck but we'll have
to keep looking for more detail to confirm that. The more you can do to
provide actual configuration details the better. Someone who will help dig
into this will need actual details on the Kafka broker and configuration, the
niFi server configuration, application logs, and ideally a flow template. I
appreciate you sharing the images but those often do not reveal enough data and
are not reproducible. Please share a flow template that replicates the issue
if you can.
> when i start consumerKafka processor more than eleven,all of the processors
> get into an indefinite stuck state
> --------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-4682
> URL: https://issues.apache.org/jira/browse/NIFI-4682
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.3.0
> Environment: Linux
> Reporter: clearlove
> Priority: Blocker
> Attachments: 2.png, nifi-bootstrap.log
>
>
> hello,
> When i start one consumeKafka_0_10 processor ,all of the processors are all
> right.
> but when i start consumeKafka_0_10 processor more than eleven,all of the
> processors get into an indefinite stuck state.
> !1.png|thumbnail!
> like in the picture ,look like everything is all right, but all
> processor(even processor is not consumeKafka_0_10 ) are stop work.All of the
> flowfile don't move.
> And when i stop them , some of consumeKafka_0_10 processor are warning
> {{Was interrupted while trying to communicate with Kafka with lease {}. Will
> roll back session and discard any partially received data.}}
> in the nifi-app.log
> {noformat}
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-4]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=099a83f6-7ddc-3df8-fe07-5b901799ade3] to run
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.SplitJson
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=fdd4850d-a619-3f17-6287-198cac3bb52a] to run
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-4]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ConsumeKafka_0_10[id=bed469b9-b52e-3178-f629-07f98ee6c07e] to run
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> SplitJson[id=f3f85270-6bb8-3f65-4347-6700426d6a2e] to run
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=6aacc9a9-c7e8-32b0-ed3c-3f89e72c30cf] to run
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,130 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=02aaae90-5ada-343e-6c15-3cfff7cfded0] to run
> 2017-12-05 10:12:02,130 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> PutSQL[id=278dc7a5-6d2e-336a-45c3-56cb7039dc7c] to run
> 2017-12-05 10:12:02,131 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=60fd28d8-b5f0-3a5b-e31a-e3ba7b73d7cb] to run
> 2017-12-05 10:12:02,131 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ConsumeKafka_0_10[id=efa69212-d0c9-3c83-0985-b22115b5e861] to run
> 2017-12-05 10:12:02,724 INFO [Flow Service Tasks Thread-1]
> o.a.nifi.controller.StandardFlowService Saved flow controller
> org.apache.nifi.controller.FlowController@a987d5e // Another save pending =
> false
> 2017-12-05 10:12:07,130 INFO [StandardProcessScheduler Thread-6]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=2fedc410-9641-3bda-cc40-59cf739a5400] to run
> 2017-12-05 10:12:07,130 WARN [Timer-Driven Process Thread-8]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=a2b469df-a2ef-355f-cc68-7b5dea6abb8b] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@2b5a2eed.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,130 INFO [StandardProcessScheduler Thread-6]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=d925dcdc-edef-3184-2185-47a0ebf3ea3b] to run
> 2017-12-05 10:12:07,131 INFO [StandardProcessScheduler Thread-6]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=8b6fef2d-3db8-3558-89a3-abce7b0e8797] to run
> 2017-12-05 10:12:07,131 INFO [StandardProcessScheduler Thread-7]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> PutSQL[id=e1af6798-96ba-31ec-c135-ff2f10d1306b] to run
> 2017-12-05 10:12:07,131 WARN [Timer-Driven Process Thread-7]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=d4b01bbd-62a3-3584-8391-371d52b83bce] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@7060d90d.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,132 INFO [StandardProcessScheduler Thread-8]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ConsumeKafka_0_10[id=b88105fe-cb1e-3c4e-f91d-22f5542cecd7] to run
> 2017-12-05 10:12:07,132 WARN [Timer-Driven Process Thread-2]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=4c067c00-fbc2-3571-5b7a-37b4957afb87] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@158837ab.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-7]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=182347e6-14ef-38c2-6873-5602f0405296] to run
> 2017-12-05 10:12:07,133 WARN [Timer-Driven Process Thread-3]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=4035760a-8b9e-3e34-51e9-e292c58e6ed7] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@4967b312.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-6]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ConsumeKafka_0_10[id=f7285790-6159-3726-662e-5717235a31dc] to run
> 2017-12-05 10:12:07,133 INFO [StandardProcessScheduler Thread-1]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ConsumeKafka_0_10[id=fae14360-943d-3a81-f3b1-675f5505e265] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=86b794b6-5875-3eac-9fdb-447743459b7d] to run
> 2017-12-05 10:12:07,133 WARN [Timer-Driven Process Thread-9]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=716f4415-e4ec-36da-8441-375bdddc690b] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@35eec0c6.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,134 WARN [Timer-Driven Process Thread-6]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=d2e3d4a1-3257-38a0-9e7b-2e0c5c7f7395] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@261dc487.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-7]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=2fb57630-0d77-3947-954c-dc3b1fd6174e] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=4cdd0444-8047-3447-9f6c-f8f536b8228a] to run
> 2017-12-05 10:12:07,134 WARN [Timer-Driven Process Thread-1]
> o.a.n.p.kafka.pubsub.ConsumeKafka_0_10
> ConsumeKafka_0_10[id=bed469b9-b52e-3178-f629-07f98ee6c07e] Was interrupted
> while trying to communicate with Kafka with lease
> org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@4f649784.
> Will roll back session and discard any partially received data.
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-4]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=d4372710-f550-3d31-10ae-8ae7e9300b44] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-7]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> PutSQL[id=df40050e-ebf1-368f-df61-43b5eb7bbdb6] to run
> 2017-12-05 10:12:07,134 INFO [StandardProcessScheduler Thread-5]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> PutSQL[id=8c979a96-03de-3beb-ec3a-1c91027162c9] to run
> {noformat}
> no Kerberos involved
> All configuration is default besides 'Kafka Brokers','Topic Name(s)','Group
> ID'
> The configuration file has been uploaded in the attachment 2.png
> the nifi-bootstrap.log is result of runing 'bin/nifi.sh dump'
> one consumeKafka processor -----------one topic-----------one partition
> --------------one replication-factor
> Every consumeKafka processor has a different topic.
> And i had tried out nifi 1.4.0,but it can not resolved the issues.
> thanks
> Sorry,my english is so bad.Please forgive me.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
