fracasula edited a comment on pull request #366: URL: https://github.com/apache/pulsar-client-go/pull/366#issuecomment-696571267
@wolfstudy it may not be a client issue as I mentioned in my last post in [this other thread](https://github.com/apache/pulsar/issues/7682#issuecomment-694085248) and in our email correspondence. If you look [at the video I sent you](https://drive.google.com/file/d/1C3ZrnOaxqvNHEbCL88-XiYWkDa9l1hGs/view?usp=sharing) at 4:20 it shows that there are no consumers connected despite the connection being open. I can see the ping and pongs travelling back and forth with `tcpdump` though so I think we can assume that the client is connected, correct me if I'm wrong. ``` $ sudo tcpdump -i docker0 -nn -v -s0 -A port 6650 tcpdump: listening on docker0, link-type EN10MB (Ethernet), capture size 262144 bytes 10:00:08.514601 IP (tos 0x0, ttl 64, id 42719, offset 0, flags [DF], proto TCP (6), length 65) 172.17.0.1.32794 > 172.17.0.2.6650: Flags [P.], cksum 0x5859 (incorrect -> 0xc76b), seq 3118165760:3118165773, ack 962716458, win 502, options [nop,nop,TS val 823193679 ecr 1498243285], length 13 E..A..@.@.;...............o.9a.*....XY..... 1..OYM`.... ......... 10:00:08.514829 IP (tos 0x0, ttl 64, id 21439, offset 0, flags [DF], proto TCP (6), length 65) 172.17.0.2.6650 > 172.17.0.1.32794: Flags [P.], cksum 0x5859 (incorrect -> 0x4a29), seq 1:14, ack 13, win 509, options [nop,nop,TS val 1498273282 ecr 823193679], length 13 E..AS.@[email protected].*..o.....XY..... YM..1..O... ......... 10:00:08.514871 IP (tos 0x0, ttl 64, id 42720, offset 0, flags [DF], proto TCP (6), length 52) 172.17.0.1.32794 > 172.17.0.2.6650: Flags [.], cksum 0x584c (incorrect -> 0xec59), ack 14, win 502, options [nop,nop,TS val 823193680 ecr 1498273282], length 0 E..4..@.@.;...............o.9a.7....XL..... 1..PYM.. 10:00:08.516685 IP (tos 0x0, ttl 64, id 21440, offset 0, flags [DF], proto TCP (6), length 65) 172.17.0.2.6650 > 172.17.0.1.32794: Flags [P.], cksum 0x5859 (incorrect -> 0x521a), seq 14:27, ack 13, win 509, options [nop,nop,TS val 1498273284 ecr 823193680], length 13 E..AS.@[email protected]..... YM..1..P... ......... 10:00:08.516742 IP (tos 0x0, ttl 64, id 42721, offset 0, flags [DF], proto TCP (6), length 52) 172.17.0.1.32794 > 172.17.0.2.6650: Flags [.], cksum 0x584c (incorrect -> 0xec48), ack 27, win 502, options [nop,nop,TS val 823193682 ecr 1498273284], length 0 E..4..@.@.;...............o.9a.D....XL..... 1..RYM.. 10:00:08.517080 IP (tos 0x0, ttl 64, id 42722, offset 0, flags [DF], proto TCP (6), length 65) 172.17.0.1.32794 > 172.17.0.2.6650: Flags [P.], cksum 0x5859 (incorrect -> 0x4a11), seq 13:26, ack 27, win 502, options [nop,nop,TS val 823193682 ecr 1498273284], length 13 E..A..@.@.;...............o.9a.D....XY..... 1..RYM..... ......... 10:00:08.557481 IP (tos 0x0, ttl 64, id 21441, offset 0, flags [DF], proto TCP (6), length 52) 172.17.0.2.6650 > 172.17.0.1.32794: Flags [.], cksum 0x584c (incorrect -> 0xec0c), ack 26, win 509, options [nop,nop,TS val 1498273324 ecr 823193682], length 0 E..4S.@[email protected]..... YM.,1..R ``` The test also prints the following which is mostly from your library: ``` consumer_partition_integration_test.go:136: Waiting for group to finish time="2020-09-22T09:58:38+02:00" level=info msg="Connecting to broker" remote_addr="pulsar://localhost:6650" time="2020-09-22T09:58:38+02:00" level=info msg="TCP connection established" local_addr="[::1]:55076" remote_addr="pulsar://localhost:6650" time="2020-09-22T09:58:38+02:00" level=info msg="Connection is ready" local_addr="[::1]:55076" remote_addr="pulsar://localhost:6650" ``` Yet the consumer is stuck and doesn't proceed with message consumption. May it be the broker not sending the messages? As you can see from the video I'm using the latest image (i.e. `streamnative/pulsar-all:2.6.1-sn-3`) as per your suggestion. I also raised the max unacked messages per consumer and per subscription by doing `bin/pulsar-admin namespaces set-max-unacked-messages-per-consumer -c 99999999 public/default` and `bin/pulsar-admin namespaces set-max-unacked-messages-per-subscription -c 99999999 public/default`. In fact in the video we can see that `blockedSubscriptionOnUnackedMsgs` is `false`. ### stats ``` "subscription-name-2e970542c9706d02" : { "msgRateOut" : 0.0, "msgThroughputOut" : 0.0, "bytesOutCounter" : 0, "msgOutCounter" : 0, "msgRateRedeliver" : 0.0, "chuckedMessageRate" : 0, "msgBacklog" : 98316, "msgBacklogNoDelayed" : 98316, "blockedSubscriptionOnUnackedMsgs" : false, "msgDelayed" : 0, "unackedMessages" : 0, "type" : "Key_Shared", "msgRateExpired" : 0.0, "lastExpireTimestamp" : 0, "lastConsumedFlowTimestamp" : 1600761458710, "lastConsumedTimestamp" : 0, "lastAckedTimestamp" : 0, "consumers" : [ ], <-- NO CONSUMERS HERE "isDurable" : true, "isReplicated" : false }, ``` ### stats-internal ``` "subscription-name-2e970542c9706d02" : { "markDeletePosition" : "10:1614", "readPosition" : "10:1615", "waitingReadOp" : false, "pendingReadOps" : 0, "messagesConsumedCounter" : 1684, "cursorLedger" : 12, "cursorLedgerLastEntry" : 1, "individuallyDeletedMessages" : "[(10:1615..10:1683],(10:1687..10:1688]]", "lastLedgerSwitchTimestamp" : "2020-09-22T07:57:38.505Z", "state" : "Open", "numberOfEntriesSinceFirstNotAckedMessage" : 1, "totalNonContiguousDeletedMessagesRange" : 2, "properties" : { } }, ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
