[
https://issues.apache.org/jira/browse/DISPATCH-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekta updated DISPATCH-2353:
---------------------------
Description:
Hi There,
We have few issues with the qpid-dispatch router where we are seeing multiple
consumers are getting created randomly by our qpid dispatch routers. We are
unable to trace nor able to replicate this issue in our lower env's. Below is
the architecture diagram. Please provide any suggestions or feedback to help us
understand this issue better. Thanks
{*}Flow{*}:
Microservice (Kubernetes) ---> NLB (Load balancer) ---> *2* Qpid-Dispatch
Routers acting up as consumers in front of brokers ({*}1.16 version{*}) --->
*4* Broker pairs of activemq{*}(2.18 version){*} masters and slaves
({*}independent pairs{*})
*Problem statement # 1*
We are seeing more than the configured number of consumers count in our
activemq hawtio console causing messages to sit in delivering count which are
Un browsable since those messages are currently being delivered to its
consumers. Having only two qpid dispatch routers(acting as consumers) infront
of our activemq brokers, the count should always remain 2 but at times it is
going 3 sometimes 4. To resolve this issue, we are having to bounce the qpid’s
to release the stuck/bad consumer so that messages can be processed/consumed.
Restarting of the client(microservice) and broker does NOT help in this case.
Only component that helps is QPID.
*Problem statement # 2*
At times we see messages going to delivering count even when there are only two
configured consumers(2qpids) showing in activemq hawtio console. We don't know
the cause to why the messages get stuck in delivering count. To resolve this,
we tried restarting the consumer service but that did not help. Next up we
tried restarting the brokers, that did not help and noticed that all the stuck
delivering messages, the broker slowly replayed back and Eventually the stuck
messages came back to delivering count post broker restart. To resolve this
issue, we are having to bounce the qpid’s to release the messages and that
fixes the issue.
*Version of qpid-dispatch router*
qpid-dispatch-router-1.16.0-1.el7.x86_64.rpm
qpid-dispatch-tools-1.16.0-1.el7.noarch.rpm
As always any suggestions would be greatly appreciated. Thanks
was:
Hi There,
We have few issues with the qpid-dispatch router where we are seeing multiple
consumers are getting created randomly by our qpid dispatch routers. We are
unable to trace nor able to replicate this issue in our lower env's. Below is
the architecture diagram. Please provide any suggestions or feedback to help us
understand this issue better. Thanks
{*}Flow{*}:
Microservice (Kubernetes) ---> NLB (Load balancer) ---> *2* Qpid-Dispatch
Routers acting up as consumers in front of brokers ({*}1.16 version{*}) --->
*4* Broker pairs of activemq{*}(2.18 version){*} masters and slaves
({*}independent pairs{*})
*Problem statement # 1*
We are seeing more than the configured number of consumers count in our
activemq hawtio console causing messages to sit in delivering count which are
Un browsable since those messages are currently being delivered to its
consumers. Having only two qpid dispatch routers(acting as consumers) infront
of our activemq brokers, the count should always remain 2 but at times it is
going 3 sometimes 4. To resolve this issue, we are having to bounce the qpid’s
to release the stuck/bad consumer so that messages can be processed/consumed.
Restarting of the client(microservice) and broker does NOT help in this case.
Only component that helps is QPID.
*Problem statement # 2*
At times we see messages going to delivering count even when there are only two
configured consumers(2qpids) showing in activemq hawtio console. We don't know
the cause to why the messages get stuck in delivering count. To resolve this,
we tried restarting the consumer service but that did not help. Next up we
tried restarting the brokers, that did not help and noticed that all the stuck
delivering messages, the broker slowly replayed back and Eventually the stuck
messages came back to delivering count post broker restart. To resolve this
issue, we are having to bounce the qpid’s to release the messages and that
fixes the issue.
*Version of qpid-dispatch router*
qpid-dispatch-router-1.16.0-1.el7.x86_64.rpm
qpid-dispatch-tools-1.16.0-1.el7.noarch.rpm
> More than configured consumers count
> ------------------------------------
>
> Key: DISPATCH-2353
> URL: https://issues.apache.org/jira/browse/DISPATCH-2353
> Project: Qpid Dispatch
> Issue Type: Bug
> Components: Router Node, Routing Engine
> Affects Versions: 1.16.0
> Environment: PROD
> Reporter: Ekta
> Priority: Major
>
> Hi There,
>
> We have few issues with the qpid-dispatch router where we are seeing multiple
> consumers are getting created randomly by our qpid dispatch routers. We are
> unable to trace nor able to replicate this issue in our lower env's. Below is
> the architecture diagram. Please provide any suggestions or feedback to help
> us understand this issue better. Thanks
>
> {*}Flow{*}:
> Microservice (Kubernetes) ---> NLB (Load balancer) ---> *2* Qpid-Dispatch
> Routers acting up as consumers in front of brokers ({*}1.16 version{*}) --->
> *4* Broker pairs of activemq{*}(2.18 version){*} masters and slaves
> ({*}independent pairs{*})
>
> *Problem statement # 1*
> We are seeing more than the configured number of consumers count in our
> activemq hawtio console causing messages to sit in delivering count which are
> Un browsable since those messages are currently being delivered to its
> consumers. Having only two qpid dispatch routers(acting as consumers) infront
> of our activemq brokers, the count should always remain 2 but at times it is
> going 3 sometimes 4. To resolve this issue, we are having to bounce the
> qpid’s to release the stuck/bad consumer so that messages can be
> processed/consumed. Restarting of the client(microservice) and broker does
> NOT help in this case. Only component that helps is QPID.
>
> *Problem statement # 2*
> At times we see messages going to delivering count even when there are only
> two configured consumers(2qpids) showing in activemq hawtio console. We don't
> know the cause to why the messages get stuck in delivering count. To resolve
> this, we tried restarting the consumer service but that did not help. Next up
> we tried restarting the brokers, that did not help and noticed that all the
> stuck delivering messages, the broker slowly replayed back and Eventually the
> stuck messages came back to delivering count post broker restart. To resolve
> this issue, we are having to bounce the qpid’s to release the messages and
> that fixes the issue.
>
> *Version of qpid-dispatch router*
> qpid-dispatch-router-1.16.0-1.el7.x86_64.rpm
> qpid-dispatch-tools-1.16.0-1.el7.noarch.rpm
>
> As always any suggestions would be greatly appreciated. Thanks
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
