jpenning-tibco opened a new issue #10108:
URL: https://github.com/apache/pulsar/issues/10108
**Describe the bug**
Exclusive consumer creates a non-partitioned topic instead of failing with
ConsumerBusy.
**To Reproduce**
Steps to reproduce the behavior:
- Brokers are configured with `allowAutoTopicCreationType: "partitioned"`
(as a requirement to use Kafka-on-Pulsar).
- Run 4 consumers configured for exclusive subscriptions simultaneously:
```sub=foobar14 ; for i in $(seq 0 3); do golang/bin/pulsar_consumer -f
~/tcm-config.maas-dev -subtype exclusive -T $sub -S $sub -I $sub-$i & done```
- The source code for the consumer sample:
[pulsar_consumer.go.zip](https://github.com/apache/pulsar/files/6239673/pulsar_consumer.go.zip)
**Expected behavior**
- The expectation is that 1 consumer will be connected/created, and the
remaining 3 will fail with ConsumerBusy.
- We do not expect a non-partitioned topic to be created with
`allowAutoTopicCreationType: "partitioned"` set.
**Screenshots**
```
INFO[0000] [Connecting to broker]
remote_addr="<redacted>"
INFO[0000] [Connecting to broker]
remote_addr="<redacted>"
INFO[0000] [Connecting to broker]
remote_addr="<redacted>"
INFO[0000] [Connecting to broker]
remote_addr="<redacted>"
INFO[0000] [TCP connection established]
local_addr="10.98.48.127:57722" remote_addr="<redacted>"
INFO[0000] [TCP connection established]
local_addr="10.98.48.127:57726" remote_addr="<redacted>"
INFO[0000] [TCP connection established]
local_addr="10.98.48.127:57723" remote_addr="<redacted>"
INFO[0000] [TCP connection established]
local_addr="10.98.48.127:57729" remote_addr="<redacted>"
INFO[0000] [Connection is ready]
local_addr="10.98.48.127:57722" remote_addr="<redacted>"
INFO[0000] [Connection is ready]
local_addr="10.98.48.127:57726" remote_addr="<redacted>"
INFO[0000] [Connection is ready]
local_addr="10.98.48.127:57723" remote_addr="<redacted>"
INFO[0000] [Connection is ready]
local_addr="10.98.48.127:57729" remote_addr="<redacted>"
INFO[0000] [Connecting to broker]
remote_addr="<redacted>"
INFO[0000] [Connecting to broker]
remote_addr="<redacted>"
INFO[0000] [Connecting to broker]
remote_addr="<redacted>"
INFO[0000] [Connecting to broker]
remote_addr="<redacted>"
INFO[0001] [TCP connection established]
local_addr="10.98.48.127:57731" remote_addr="<redacted>"
INFO[0001] [TCP connection established]
local_addr="10.98.48.127:57730" remote_addr="<redacted>"
INFO[0001] [TCP connection established]
local_addr="10.98.48.127:57732" remote_addr="<redacted>"
INFO[0001] [TCP connection established]
local_addr="10.98.48.127:57733" remote_addr="<redacted>"
INFO[0001] [Connection is ready]
local_addr="10.98.48.127:57731" remote_addr="<redacted>"
INFO[0001] [Connection is ready]
local_addr="10.98.48.127:57730" remote_addr="<redacted>"
INFO[0001] [Connection is ready]
local_addr="10.98.48.127:57732" remote_addr="<redacted>"
INFO[0001] [Connection is ready]
local_addr="10.98.48.127:57733" remote_addr="<redacted>"
ERRO[0001] [Failed to create consumer] consumerID=1
error="server error: ConsumerBusy: Exclusive consumer is already connected"
name=foobar14-3 subscription=foobar14
topic="persistent://jpenning/channel/foobar14-partition-0"
ERRO[0001] [Failed to create consumer] consumerID=1
error="server error: ConsumerBusy: Exclusive consumer is already connected"
name=foobar14-3 subscription=foobar14
topic="persistent://jpenning/channel/foobar14-partition-0"
INFO[0001] [Connected consumer] consumerID=1
name=foobar14-2 subscription=foobar14
topic="persistent://jpenning/channel/foobar14-partition-0"
INFO[0001] [Created consumer] consumerID=1
name=foobar14-2 subscription=foobar14
topic="persistent://jpenning/channel/foobar14-partition-0"
2021/03/31 16:36:56 server error: ConsumerBusy: Exclusive consumer is
already connected
2021/03/31 16:36:56 Waiting for message(s)
ERRO[0001] [Failed to create consumer] consumerID=1
error="server error: ConsumerBusy: Exclusive consumer is already connected"
name=foobar14-1 subscription=foobar14
topic="persistent://jpenning/channel/foobar14-partition-0"
ERRO[0001] [Failed to create consumer] consumerID=1
error="server error: ConsumerBusy: Exclusive consumer is already connected"
name=foobar14-1 subscription=foobar14
topic="persistent://jpenning/channel/foobar14-partition-0"
2021/03/31 16:36:56 server error: ConsumerBusy: Exclusive consumer is
already connected
INFO[0001] [Connected consumer] consumerID=1
name=foobar14-0 subscription=foobar14
topic="persistent://jpenning/channel/foobar14"
INFO[0001] [Created consumer] consumerID=1
name=foobar14-0 subscription=foobar14
topic="persistent://jpenning/channel/foobar14"
2021/03/31 16:36:56 Waiting for message(s)
[11] Exit 1 golang/bin/pulsar_consumer -f
~/tcm-config.maas-dev -subtype exclusive -T $sub -S $sub -I $sub-$i
[13]+ Exit 1 golang/bin/pulsar_consumer -f
~/tcm-config.maas-dev -subtype exclusive -T $sub -S $sub -I $sub-$i
```
**Desktop (please complete the following information):**
- Clients are run on MacOS.
- Pulsar servers run behind a network load balancer in AWS on K8s using a
Ubuntu 20.04 image. The pulsar group consists of 3 bookies, 2 brokers, 2 proxy,
1 recovery, 1 toolset, and 5 zookeepers.
**Additional context**
- This issue has only been reproduced with new (auto) topics. I've never
reproduced the issue re-running the test case with the same $sub.
- This issue has only been reproduced using Go clients against the AWS
deployment described above. Smaller local deployments using DockerForDesktop
(with fewer components, e.g. 1 broker), and `pulsar standalone` work as
expected.
- Java and Python clients work as expected.
- When I run the test in the `pulsar standalone` context instead of
`ConsumerBusy` I see `ServiceNotReady: LookupError` for 3 of the 4 clients. So
the test behaves as expected, but the error message is different and in my mind
suggests what the problem might be.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
