[jira] [Commented] (NIFI-10353) ConsumeAzureEventHub does not stop even though output queue is backpressured
[ https://issues.apache.org/jira/browse/NIFI-10353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711708#comment-17711708 ] Peter Schmitzer commented on NIFI-10353: Update from our side: Azure eventhub supports connecting with the standard kafka protocol so standard kafka consumers can (and I believe should be) used for that purpose. We have removed this processor from all our flows and have no need to improve this. > ConsumeAzureEventHub does not stop even though output queue is backpressured > - > > Key: NIFI-10353 > URL: https://issues.apache.org/jira/browse/NIFI-10353 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.16.3 >Reporter: Peter Schmitzer >Priority: Major > > ConsumeAzureEventHub seems to not care about backpressure and continues to > send flowfiles to the output queue even though it is backpressured. This > endlessly growing queue will ultimately lead to nifi going into overload and > be unhealthy. > It was expected that the processor will stop putting further data in the > outgoing queue as soon as it is backpressured. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10353) ConsumeAzureEventHub does not stop even though output queue is backpressured
[ https://issues.apache.org/jira/browse/NIFI-10353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17636875#comment-17636875 ] David Handermann commented on NIFI-10353: - Thanks for the question [~peter.schmitzer]. The project receives a large number of bug reports and feature requests on a regular basis, and there is no automated prioritization mechanism. Unfortunately it looks like this behavior has been in place since the early versions of the Processor. NiFi 1.18.0 included some significant changes to refactor the Processor implementation based on the latest Azure SDK, but it did not change the core functionality. This issue has not been prioritized right now, but the Processor is now in a better position as far as the code to address the problem described. > ConsumeAzureEventHub does not stop even though output queue is backpressured > - > > Key: NIFI-10353 > URL: https://issues.apache.org/jira/browse/NIFI-10353 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.16.3 >Reporter: Peter Schmitzer >Priority: Major > > ConsumeAzureEventHub seems to not care about backpressure and continues to > send flowfiles to the output queue even though it is backpressured. This > endlessly growing queue will ultimately lead to nifi going into overload and > be unhealthy. > It was expected that the processor will stop putting further data in the > outgoing queue as soon as it is backpressured. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10353) ConsumeAzureEventHub does not stop even though output queue is backpressured
[ https://issues.apache.org/jira/browse/NIFI-10353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17636520#comment-17636520 ] Peter Schmitzer commented on NIFI-10353: Hi [~exceptionfactory] , is there any process we need to follow that this ticket is being looked at by someone? Thanks > ConsumeAzureEventHub does not stop even though output queue is backpressured > - > > Key: NIFI-10353 > URL: https://issues.apache.org/jira/browse/NIFI-10353 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.16.3 >Reporter: Peter Schmitzer >Priority: Major > > ConsumeAzureEventHub seems to not care about backpressure and continues to > send flowfiles to the output queue even though it is backpressured. This > endlessly growing queue will ultimately lead to nifi going into overload and > be unhealthy. > It was expected that the processor will stop putting further data in the > outgoing queue as soon as it is backpressured. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10353) ConsumeAzureEventHub does not stop even though output queue is backpressured
[ https://issues.apache.org/jira/browse/NIFI-10353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17579725#comment-17579725 ] Peter Schmitzer commented on NIFI-10353: Hi guys, we are using an azure eventhub as a central input source for our data ingestion and thus have the ConsumeAzureEventHub processor in place. Our idea was in case of outages of downstream systems that we can use the backpressure mechanisms of the nifi queues to have all queues backpressured up to the azure eventhub itself and then basically have nifi to "stop processing". All the new messages will wait in the eventhub for us. Unfortunately, that does not happen and it behaves as described above. > ConsumeAzureEventHub does not stop even though output queue is backpressured > - > > Key: NIFI-10353 > URL: https://issues.apache.org/jira/browse/NIFI-10353 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.16.3 >Reporter: Peter Schmitzer >Priority: Major > > ConsumeAzureEventHub seems to not care about backpressure and continues to > send flowfiles to the output queue even though it is backpressured. This > endlessly growing queue will ultimately lead to nifi going into overload and > be unhealthy. > It was expected that the processor will stop putting further data in the > outgoing queue as soon as it is backpressured. -- This message was sent by Atlassian Jira (v8.20.10#820010)
