thomas-turner opened a new issue, #20714: URL: https://github.com/apache/pulsar/issues/20714
### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Version Pulsar 3.0.0 Mac OS 13.4 (M1) & Windows 10 (Tested steps to reproduce on both) ### Minimal reproduce step Place the following `docker-compose.yml` in a folder and run `docker-compose up --build` in that folder. Exec into one of the Pulsar container (I used the Proxy one), run `bash` then `cd bin` then execute `./pulsar-client --url "pulsar://localhost:6650" consume "public/default/message-type-[a-z0-9]{1,8}" --subscription-name "serviceMessageTypeSubscription" --regex`. You should see the following: ``` 2023-07-04T12:07:35,209+0000 [pulsar-client-io-1-3] INFO org.apache.pulsar.client.impl.ConnectionPool - [[id: 0x99f4dfff, L:/127.0.0.1:39888 - R:localhost/127.0.0.1:6650]] Connected to server 2023-07-04T12:07:35,345+0000 [pulsar-client-io-1-3] INFO org.apache.pulsar.client.impl.TopicListWatcher - [null][Watcher(public/default/message-type-[a-z0-9]{1,8})] Creating topic list watcher on cnx [id: 0x99f4dfff, L:/127.0.0.1:39888 - R:localhost/127.0.0.1:6650], watcherId 0 2023-07-04T12:07:35,349+0000 [pulsar-client-io-1-3] WARN org.apache.pulsar.client.impl.TopicListWatcher - [null][Watcher(public/default/message-type-[a-z0-9]{1,8})] Failed to create topic list watcher on localhost/127.0.0.1:6650 ``` ```yml version: '3' networks: pulsar: driver: bridge services: zookeeper: image: apachepulsar/pulsar:3.0.0 container_name: zookeeper restart: on-failure networks: - pulsar ports: - "2181:2181" - "8010:8000" environment: - PULSAR_MEM=-Xms256m -Xmx256m -XX:MaxDirectMemorySize=256m command: > bash -c "bin/apply-config-from-env.py conf/zookeeper.conf && \ bin/generate-zookeeper-config.sh conf/zookeeper.conf && \ exec bin/pulsar zookeeper" healthcheck: test: ["CMD", "bin/pulsar-zookeeper-ruok.sh"] interval: 10s timeout: 5s retries: 30 cluster-init: container_name: cluster-init hostname: cluster-init image: apachepulsar/pulsar:3.0.0 networks: - pulsar command: > bin/pulsar initialize-cluster-metadata \ --cluster local \ --zookeeper zookeeper:2181 \ --configuration-store zookeeper:2181 \ --web-service-url http://broker:8080 \ --broker-service-url pulsar://broker:6650 depends_on: zookeeper: condition: service_healthy bookie: image: apachepulsar/pulsar:3.0.0 container_name: bookie restart: on-failure networks: - pulsar environment: - clusterName=local - zkServers=zookeeper:2181 - enableStatistics=true - advertisedAddress=bookie - extraServerComponents=org.apache.bookkeeper.stream.server.StreamStorageLifecycleComponent - BOOKIE_MEM=-Xms512m -Xmx512m -XX:MaxDirectMemorySize=256m depends_on: zookeeper: condition: service_healthy cluster-init: condition: service_completed_successfully ports: - "3181:3181" - "4181:4181" - "8020:8000" command: > bash -c " bin/apply-config-from-env.py conf/bookkeeper.conf && echo 'storageserver.grpc.port=4181' >> conf/bookkeeper.conf && echo 'dlog.bkcEnsembleSize=1' >> conf/bookkeeper.conf && echo 'dlog.bkcWriteQuorumSize=1' >> conf/bookkeeper.conf && echo 'dlog.bkcAckQuorumSize=1' >> conf/bookkeeper.conf && echo 'storage.range.store.dirs=data/bookkeeper/ranges' >> conf/bookkeeper.conf && echo 'storage.serve.readonly.tables=false' >> conf/bookkeeper.conf && echo 'storage.cluster.controller.schedule.interval.ms=30000' >> conf/bookkeeper.conf && exec bin/pulsar bookie " broker: image: apachepulsar/pulsar:3.0.0 container_name: broker hostname: broker restart: on-failure networks: - pulsar environment: - metadataStoreUrl=zookeeper:2181 - configurationMetadataStoreUrl=zookeeper:2181 - advertisedAddress=broker - clusterName=local - allowAutoTopicCreationType=partitioned - defaultNumPartitions=2 - subscriptionExpirationTimeMinutes=10 - subscriptionPatternMaxLength=65 - webSocketServiceEnabled=true - functionsWorkerEnabled=true - proxyRoles=proxy-admin - superUserRoles=broker-admin,proxy-admin,fw-admin,internal-admin - authenticateOriginalAuthData=false - authenticationEnabled=false - authorizationEnabled=false - managedLedgerDefaultEnsembleSize=1 - managedLedgerDefaultWriteQuorum=1 - managedLedgerDefaultAckQuorum=1 - PULSAR_MEM=-Xms512m -Xmx512m -XX:MaxDirectMemorySize=256m depends_on: zookeeper: condition: service_healthy bookie: condition: service_started ports: - "8030:8080" command: > bash -c " bin/apply-config-from-env.py conf/broker.conf && exec bin/pulsar broker " proxy: image: apachepulsar/pulsar:3.0.0 container_name: proxy hostname: proxy restart: on-failure networks: - pulsar environment: - metadataStoreUrl=zookeeper:2181 - configurationMetadataStoreUrl=zookeeper:2181 - forwardAuthorizationCredentials=true - authenticationEnabled=false - webSocketServiceEnabled=true - clusterName=local - PULSAR_MEM=-Xms256m -Xmx256m -XX:MaxDirectMemorySize=256m depends_on: zookeeper: condition: service_healthy broker: condition: service_started ports: - "6650:6650" - "8080:8080" command: bash -c "bin/apply-config-from-env.py conf/proxy.conf && exec bin/pulsar proxy" ``` ### What did you expect to see? I expected to see the client successfully create a topic watcher for my topic regex pattern. ### What did you see instead? The client fails to create a topic watcher for the regex pattern you can see in the steps to reproduce. When run from my Java microservice, we also get the following stack trace which seems to provide a little more information: ``` 2023-07-04 11:35:56.090 DEBUG 70935 --- [r-client-io-1-3] o.a.p.c.i.PatternMultiTopicsConsumerImpl : : Unable to create topic list watcher. Falling back to only polling for new topics java.util.concurrent.CompletionException: org.apache.pulsar.client.api.PulsarClientException$NotAllowedException: Broker does not allow broker side pattern evaluation. at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332) ~[?:?] at java.util.concurrent.CompletableFuture.uniAcceptNow(CompletableFuture.java:747) ~[?:?] at java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:735) ~[?:?] at java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2214) ~[?:?] at org.apache.pulsar.client.impl.TopicListWatcher.connectionOpened(TopicListWatcher.java:127) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.client.impl.ConnectionHandler.lambda$grabCnx$0(ConnectionHandler.java:84) ~[pulsar-client-3.0.0.jar!/:3.0.0] at java.util.concurrent.CompletableFuture.uniAcceptNow(CompletableFuture.java:757) ~[?:?] at java.util.concurrent.CompletableFuture.uniAcceptStage(CompletableFuture.java:735) ~[?:?] at java.util.concurrent.CompletableFuture.thenAccept(CompletableFuture.java:2214) ~[?:?] at org.apache.pulsar.client.impl.ConnectionHandler.grabCnx(ConnectionHandler.java:84) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.client.impl.TopicListWatcher.<init>(TopicListWatcher.java:81) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.client.impl.PatternMultiTopicsConsumerImpl.<init>(PatternMultiTopicsConsumerImpl.java:84) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.client.impl.PulsarClientImpl.lambda$patternTopicSubscribeAsync$8(PulsarClientImpl.java:588) ~[pulsar-client-3.0.0.jar!/:3.0.0] at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:718) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2179) ~[?:?] at org.apache.pulsar.client.impl.BinaryProtoLookupService.lambda$getTopicsUnderNamespace$11(BinaryProtoLookupService.java:315) ~[pulsar-client-3.0.0.jar!/:3.0.0] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2179) ~[?:?] at org.apache.pulsar.client.impl.ClientCnx.handleGetTopicsOfNamespaceSuccess(ClientCnx.java:899) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.common.protocol.PulsarDecoder.channelRead(PulsarDecoder.java:344) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.handler.flush.FlushConsolidationHandler.channelRead(FlushConsolidationHandler.java:152) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.shade.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[pulsar-client-3.0.0.jar!/:3.0.0] at java.lang.Thread.run(Thread.java:1583) ~[?:?] Caused by: org.apache.pulsar.client.api.PulsarClientException$NotAllowedException: Broker does not allow broker side pattern evaluation. at org.apache.pulsar.client.impl.ClientCnx.newWatchTopicList(ClientCnx.java:1131) ~[pulsar-client-3.0.0.jar!/:3.0.0] at org.apache.pulsar.client.impl.TopicListWatcher.connectionOpened(TopicListWatcher.java:125) ~[pulsar-client-3.0.0.jar!/:3.0.0] ... 43 more ``` In the microservice we are using the same Pulsar version and subscription pattern. We have confirmed that broker pattern evaluation is enabled in our conf. ### Anything else? We also tried subscribing against the broker directly and we **do not** see this issue. ### Are you willing to submit a PR? - [X] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
