[GitHub] [pulsar] digi691 commented on issue #5417: Broker hangs and crashes when listing non-persistent topics
digi691 commented on issue #5417: URL: https://github.com/apache/pulsar/issues/5417#issuecomment-621328591 Know this issue is stale though we are still having this issue on our bare metal clusters. I recently setup Pulsar in Kubernetes and I'm not seeing this behavior. Is there any kind of miss-configuration of the brokers and zookeepers that could cause the broker just to swallow these type of requests, not respond, and not log any errors about it? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] digi691 commented on issue #5417: Broker hangs and crashes when listing non-persistent topics
digi691 commented on issue #5417: Broker hangs and crashes when listing non-persistent topics URL: https://github.com/apache/pulsar/issues/5417#issuecomment-575692466 @tuteng In my development environment I have two HA-Proxy servers that load balance between 2 - Pulsar proxy servers. I then have 3 brokers, these brokers then point to 4 bookkeepers. I also have 3 Zookeeper nodes. The difference between my development and production environments are that the development environment's configuration store is also the Zookeeper quorum. In my Production environment I have a separate quorum zookeeper cluster (3) and configuration store (3), and also have 8 Bookkeepers in production. Just to note, I am seeing this issue in our production instance as well. Here is the broker.conf I'm using on all three of the brokers and is pretty much the same in the production environment just different host names, cert names, and bucket names: https://gist.github.com/digi691/2a27c8a6055145e98450fc7efce8c0c4. FYI - I had to scrub the file for hostnames, Identifications in Cert Names, etc... As you'll notice: TLS is turned on though authentication is currently turned off. I timeout when going through the HA-Proxy and Pulsar Proxies as I would expect when hitting the admin api to list non-persistent volumes. When I point directly to the brokers and hit that that admin api, it just sits and waits forever as I explained above and never produces a response. I also cannot use the pulsar-admin tool sub command `topics list` as I believe it's trying to list non-persistent topics as well as persistent. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] digi691 commented on issue #5417: Broker hangs and crashes when listing non-persistent topics
digi691 commented on issue #5417: Broker hangs and crashes when listing non-persistent topics URL: https://github.com/apache/pulsar/issues/5417#issuecomment-574240631 @tuteng Let me know how I should proceed or offer further help into figuring out this issue. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] digi691 commented on issue #5417: Broker hangs and crashes when listing non-persistent topics
digi691 commented on issue #5417: Broker hangs and crashes when listing
non-persistent topics
URL: https://github.com/apache/pulsar/issues/5417#issuecomment-571677117
I was able to upgrade our dev/test Pulsar cluster to version 2.4.2. Now
hitting the /admin/v2/non-persistent/{tenant}/{namespace} API endpoint causes a
HTTP ERROR 504 whether the namspace has non-persistent topics or not. When
Pulsar is run in standalone mode, it produces an empty list.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [pulsar] digi691 commented on issue #5417: Broker hangs and crashes when listing non-persistent topics
digi691 commented on issue #5417: Broker hangs and crashes when listing non-persistent topics URL: https://github.com/apache/pulsar/issues/5417#issuecomment-563350359 @tuteng will talk to my team about upgrading our test environment. Hopefully I will be able to get to testing 2.4.2 within the next couple of weeks. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] digi691 commented on issue #5417: Broker hangs and crashes when listing non-persistent topics
digi691 commented on issue #5417: Broker hangs and crashes when listing non-persistent topics URL: https://github.com/apache/pulsar/issues/5417#issuecomment-561326130 What is the appropriate number? Like I said earlier, I currently have it set to 8. Which is the suggested default. What I don't understand is that if I make any other requests on the API, it finishes fine. It's only this endpoint that causes the whole broker to crash. I don't have proof of what is going on, though it seems like when I hit that endpoint it ties up a HttpServerThread indefinitely. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [pulsar] digi691 commented on issue #5417: Broker hangs and crashes when listing non-persistent topics
digi691 commented on issue #5417: Broker hangs and crashes when listing non-persistent topics URL: https://github.com/apache/pulsar/issues/5417#issuecomment-548826391 Currently `numHttpServerThreads=8` is set to what #3776 suggests. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
