Hello Bryan, One more thing, I have just checked the *config/consumer.properties* where the consumer timeout is not set Does it have to do anything with this?
#consumer timeout #consumer.timeout.ms=5000 # timeout in ms for connecting to zookeeper zookeeper.connection.timeout.ms=6000 Thank you! ______________________ *Kind Regards,* *Anshuman Ghosh* *Contact - +49 179 9090964* On Tue, Mar 21, 2017 at 5:12 PM, Anshuman Ghosh < [email protected]> wrote: > Hello Bryan, > > Thank you so much for your reply! > > Just to answer your questions - > > 1. Back Pressure is set to the next processor but that isn't triggered > anytime (threshold limit is 1 GB) > 2. No other custom properties are set apart from the one I have shared > with you. > > Also it suggests to increase the *session timeout *value - *Where do we > do that*? > Settings for my Kafka cluster is* zookeeper.connection.timeout.ms > <http://zookeeper.connection.timeout.ms>=6000* > > > Thank you! > > ______________________ > > *Kind Regards,* > *Anshuman Ghosh* > *Contact - +49 179 9090964 <+49%20179%209090964>* > > > On Tue, Mar 21, 2017 at 3:36 PM, Bryan Bende <[email protected]> wrote: > >> Anshuman, >> >> The error message is saying that NiFi tried to use a consumer where >> the session already timed-out. So for example, if the session timeout >> was 10 seconds, and NiFi had a consumer that hadn't been used in 12 >> seconds and then tried to use it, you would get this error because the >> session for that consumer already exceeded the 10 seconds and expired. >> >> The typical reasons you would see that error message are the following... >> >> - The processor run schedule exceeds the duration of the >> session.timeout (i.e. processor scheduled to run every 30 seconds, >> session.timeout is 10 seconds) >> - The processor wasn't scheduled to run for some period of time due to >> back-pressure, so even though the run schedule might be every 1 >> second, it actually didn't execute for longer than the session.timeout >> - The amount of time to perform a single poll() from Kafka takes >> longer than the session.timeout, this is unlikely for NiFi but >> theoretically possible for other consumers depending how they are >> implemented >> >> From your screenshots it looks like ConsumeKafka_0_10 is scheduled to >> run every 2 seconds... >> >> Do you ever have back-pressure between ConsumeKafka and the next >> processors? >> Do you have any other custom properties set in ConsumeKafka? I can >> only see down to Max Uncommitted Time >> >> The good news is that newer versions of the Kafka 0.10.x client have >> some built in heart-beating such that even if a consumer hasn't been >> used in a while, it will continue to keep the session alive. >> >> Koji actually has a PR up to address this: >> https://github.com/apache/nifi/pull/1527 >> >> Thanks, >> >> Bryan >> >> >> On Tue, Mar 21, 2017 at 9:33 AM, Anshuman Ghosh >> <[email protected]> wrote: >> > Hello, >> > >> > Can anyone please help me with this error message "ConsumeKafka_0_10" >> > processor. >> > Earlier I was not getting these messages, now I am seeing this Error and >> > Waring message (attached screenshot) and sometimes getting duplicates :( >> > Processor is running though! >> > >> > I have attached the configuration parameters screenshots for both the >> > Publisher and Consumer processor. >> > >> > Thank >> > ing >> > you >> > in advance >> > ! >> > >> > ______________________ >> > >> > Kind Regards, >> > Anshuman Ghosh >> > Contact - +49 179 9090964 >> > >> > >
