BewareMyPower commented on PR #23062:
URL: https://github.com/apache/pulsar/pull/23062#issuecomment-2244764499
This improvement for the reader side can be verified by the following
example.
```java
public class LoadTopicTest extends ProducerConsumerBase {
@BeforeClass
@Override
protected void setup() throws Exception {
conf.setDefaultNumberOfNamespaceBundles(1);
conf.setNumTransactionReplayThreadPoolSize(1);
conf.setTransactionCoordinatorEnabled(true);
super.isTcpLookup = true;
super.internalSetup();
super.producerBaseSetup();
}
@AfterClass
@Override
protected void cleanup() throws Exception {
super.internalCleanup();
}
@Test
public void test() throws Exception {
final var topic = "test";
admin.topics().createPartitionedTopic(topic, 5);
@Cleanup final var producer = pulsarClient.newProducer().topic(topic
+ "-partition-0").create();
Thread.sleep(1000);
}
}
```
Save the outputs into `1.log` and run `grep -E "Subscribing on|Subscribed
to|Closed consumer" 1.log | grep snapshot`, then you can see:
```
2024-07-23T17:46:50,142 - INFO - [pulsar-io-9-4:ServerCnx] - [[id:
0x0134c04e, L:/127.0.0.1:57954 - R:/127.0.0.1:57960] [SR:127.0.0.1,
state:Connected]] Subscribing on topic
persistent://public/default/__transaction_buffer_snapshot / reader-8d442bc96f.
consumerId: 1
2024-07-23T17:46:50,145 - INFO - [pulsar-io-9-4:ServerCnx] - [[id:
0x0134c04e, L:/127.0.0.1:57954 - R:/127.0.0.1:57960] [SR:127.0.0.1,
state:Connected]] Subscribing on topic
persistent://public/default/__transaction_buffer_snapshot / reader-beb7e469f7.
consumerId: 3
2024-07-23T17:46:50,146 - INFO - [pulsar-io-9-4:ServerCnx] - [[id:
0x0134c04e, L:/127.0.0.1:57954 - R:/127.0.0.1:57960] [SR:127.0.0.1,
state:Connected]] Subscribing on topic
persistent://public/default/__transaction_buffer_snapshot / reader-e9f0ce9149.
consumerId: 2
2024-07-23T17:46:50,149 - INFO - [pulsar-io-9-4:ServerCnx] - [[id:
0x0134c04e, L:/127.0.0.1:57954 - R:/127.0.0.1:57960] [SR:127.0.0.1,
state:Connected]] Subscribing on topic
persistent://public/default/__transaction_buffer_snapshot / reader-d27782ccd4.
consumerId: 5
2024-07-23T17:46:50,150 - INFO - [pulsar-io-9-4:ServerCnx] - [[id:
0x0134c04e, L:/127.0.0.1:57954 - R:/127.0.0.1:57960] [SR:127.0.0.1,
state:Connected]] Subscribing on topic
persistent://public/default/__transaction_buffer_snapshot / reader-ff5b692579.
consumerId: 4
2024-07-23T17:46:50,174 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot][reader-ff5b692579]
Subscribed to topic on localhost/127.0.0.1:57954 -- consumer: 4
2024-07-23T17:46:50,177 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot] [reader-ff5b692579]
Closed consumer
2024-07-23T17:46:50,186 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot][reader-d27782ccd4]
Subscribed to topic on localhost/127.0.0.1:57954 -- consumer: 5
2024-07-23T17:46:50,187 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot][reader-e9f0ce9149]
Subscribed to topic on localhost/127.0.0.1:57954 -- consumer: 2
2024-07-23T17:46:50,187 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot] [reader-d27782ccd4]
Closed consumer
2024-07-23T17:46:50,188 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot] [reader-e9f0ce9149]
Closed consumer
2024-07-23T17:46:50,188 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot][reader-beb7e469f7]
Subscribed to topic on localhost/127.0.0.1:57954 -- consumer: 3
2024-07-23T17:46:50,190 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot] [reader-beb7e469f7]
Closed consumer
2024-07-23T17:46:50,190 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot][reader-8d442bc96f]
Subscribed to topic on localhost/127.0.0.1:57954 -- consumer: 1
2024-07-23T17:46:50,191 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot] [reader-8d442bc96f]
Closed consumer
```
Even though the `numTransactionReplayThreadPoolSize` is 1, there are 5
pending readers from the "Subscribing on" logs.
After this change, the logs become
```
2024-07-23T17:49:48,902 - INFO - [pulsar-io-9-1:ServerCnx] - [[id:
0x41f78fc2, L:/127.0.0.1:58007 - R:/127.0.0.1:58013] [SR:127.0.0.1,
state:Connected]] Subscribing on topic
persistent://public/default/__transaction_buffer_snapshot / reader-bef854f9d0.
consumerId: 1
2024-07-23T17:49:48,940 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot][reader-bef854f9d0]
Subscribed to topic on localhost/127.0.0.1:58007 -- consumer: 1
2024-07-23T17:49:48,943 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot] [reader-bef854f9d0]
Closed consumer
2024-07-23T17:49:48,947 - INFO - [pulsar-io-9-1:ServerCnx] - [[id:
0x41f78fc2, L:/127.0.0.1:58007 - R:/127.0.0.1:58013] [SR:127.0.0.1,
state:Connected]] Subscribing on topic
persistent://public/default/__transaction_buffer_snapshot / reader-bb423c5246.
consumerId: 2
2024-07-23T17:49:48,954 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot][reader-bb423c5246]
Subscribed to topic on localhost/127.0.0.1:58007 -- consumer: 2
2024-07-23T17:49:48,956 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot] [reader-bb423c5246]
Closed consumer
2024-07-23T17:49:48,961 - INFO - [pulsar-io-9-1:ServerCnx] - [[id:
0x41f78fc2, L:/127.0.0.1:58007 - R:/127.0.0.1:58013] [SR:127.0.0.1,
state:Connected]] Subscribing on topic
persistent://public/default/__transaction_buffer_snapshot / reader-11542281ee.
consumerId: 3
2024-07-23T17:49:48,964 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot][reader-11542281ee]
Subscribed to topic on localhost/127.0.0.1:58007 -- consumer: 3
2024-07-23T17:49:48,965 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot] [reader-11542281ee]
Closed consumer
2024-07-23T17:49:48,970 - INFO - [pulsar-io-9-1:ServerCnx] - [[id:
0x41f78fc2, L:/127.0.0.1:58007 - R:/127.0.0.1:58013] [SR:127.0.0.1,
state:Connected]] Subscribing on topic
persistent://public/default/__transaction_buffer_snapshot / reader-4826069b23.
consumerId: 4
2024-07-23T17:49:48,974 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot][reader-4826069b23]
Subscribed to topic on localhost/127.0.0.1:58007 -- consumer: 4
2024-07-23T17:49:48,975 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot] [reader-4826069b23]
Closed consumer
2024-07-23T17:49:48,980 - INFO - [pulsar-io-9-1:ServerCnx] - [[id:
0x41f78fc2, L:/127.0.0.1:58007 - R:/127.0.0.1:58013] [SR:127.0.0.1,
state:Connected]] Subscribing on topic
persistent://public/default/__transaction_buffer_snapshot / reader-5c857ecb26.
consumerId: 5
2024-07-23T17:49:48,984 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot][reader-5c857ecb26]
Subscribed to topic on localhost/127.0.0.1:58007 -- consumer: 5
2024-07-23T17:49:48,985 - INFO - [pulsar-io-9-3:ConsumerImpl] -
[persistent://public/default/__transaction_buffer_snapshot] [reader-5c857ecb26]
Closed consumer
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
