[GitHub] [pulsar] tuteng commented on issue #5417: Broker hangs and crashes when listing non-persistent topics
tuteng commented on issue #5417: Broker hangs and crashes when listing non-persistent topics URL: https://github.com/apache/pulsar/issues/5417#issuecomment-574421582 Can you give me a detailed description of your use of context information, such as whether to turn on authentication, the command used, the relevant configuration of the cluster, etc. @digi691 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5417: Broker hangs and crashes when listing non-persistent topics
tuteng commented on issue #5417: Broker hangs and crashes when listing non-persistent topics URL: https://github.com/apache/pulsar/issues/5417#issuecomment-562793789 I think this issue can be closed, if there are any problems, we can consider open it back on. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5417: Broker hangs and crashes when listing non-persistent topics
tuteng commented on issue #5417: Broker hangs and crashes when listing non-persistent topics URL: https://github.com/apache/pulsar/issues/5417#issuecomment-562549793 @digi691 You can try 2.4.2 https://github.com/apache/pulsar/blob/v2.4.2/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/NonPersistentTopics.java#L234 A blocking operation was used in version 2.4.1 version https://github.com/apache/pulsar/blob/v2.4.1/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/NonPersistentTopics.java#L184, and has been fixed in version 2.4.2 version https://github.com/apache/pulsar/blob/v2.4.2/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/NonPersistentTopics.java#L234. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5417: Broker hangs and crashes when listing non-persistent topics
tuteng commented on issue #5417: Broker hangs and crashes when listing
non-persistent topics
URL: https://github.com/apache/pulsar/issues/5417#issuecomment-562382897
I tested this problem. under a cluster (including three brokers and three
bookies), I used the `ab` tool to test. numHttpServerThreads defaults to 8.
when the number of query requests initiated is significantly higher than the
number that the thread can handle, it will indeed block the broker.
I think this is because `non-persisten topic` is stored in memory. when
there are multiple brokers, your request is sent to one of them, and the broker
completes the remaining work. after the broker receives the request, it will
forward the request to all brokers under the cluster. Look up all bundles of
all broker, then traverse all topics under these bundles, and finally return
topics. A large part of this operation is network delay. Therefore, when
concurrent requests are large, the above problems will occur. I have not found
a suitable way to solve this problem through fix code, but I have two
alternatives to query `non-persistent topic`.
1. Reasonably evaluate your query request and configure the appropriate
number of numHttpServerThreads, but this method cannot completely solve the
above problems.
2. The proposal is divided into the following three steps
a. Get all broker addresses
```
curl -v http://any-broker-ip:8080/admin/v2/brokers/cluster-name
```
b. Loop the following rest API to obtain bundle
```
curl -v http://broker-1:8080/admin/v2/non-persistent/test/test-namespace
curl -v http://broker-2:8080/admin/v2/non-persistent/test/test-namespace
```
The following results will be returned
```
"bundles" : {
"boundaries" : [ "0x", "0x4000", "0x8000", "0xc000",
"0x" ],
"numBundles" : 4
},
```
c. Get the topic under the bundle through the bundle.
```
...
http://broker-ip:8080/admin/v2/non-persistent/test/test-namespace/0x_0x4000
http://broker-ip:8080/admin/v2/non-persistent/test/test-namespace/0x4000_0x8000
...
```
I think the second method will solve the problem of blocking the broker due
to the high number of concurrent requests.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5417: Broker hangs and crashes when listing non-persistent topics
tuteng commented on issue #5417: Broker hangs and crashes when listing non-persistent topics URL: https://github.com/apache/pulsar/issues/5417#issuecomment-561446569 Thank you for your reply. I will test it under the cluster and try to fix this problem. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] tuteng commented on issue #5417: Broker hangs and crashes when listing non-persistent topics
tuteng commented on issue #5417: Broker hangs and crashes when listing non-persistent topics URL: https://github.com/apache/pulsar/issues/5417#issuecomment-560984383 I understand that it is possible to solve this problem by increasing the number of threads of the parameter numHttpServerThreads. If the amount of concurrency for HTTP requests is large, we should increase the value of this parameter. We may need to add documents to illustrate the HTTP request problem under the condition of high concurrency. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
