maanders-tibco opened a new issue, #20532: URL: https://github.com/apache/pulsar/issues/20532
### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Motivation When disabling HTTP ports in the Pulsar broker, the [REST Client Producer](https://pulsar.apache.org/docs/3.0.x/client-libraries-rest/) fails to produce messages. The following is a description to reproduce the error: 1. Start secure single node bare metal cluster according to this doc: https://pulsar.apache.org/docs/3.0.x/deploy-bare-metal/ 2. Configure broker.conf with the following parameters: ``` brokerServicePort=6650 brokerServicePortTls=6651 webServicePort=8080 webServicePortTls=8081 ``` After starting the broker, this command: ``` [~/Desktop/pulsar/apache-pulsar-3.0.0]: curl -v -k --cert ./certs/client.cert.pem --key ./certs/client.key.pem --location --request POST 'https://localhost:8081/topics/persistent/public/default/test' --header 'Content-Type: application/json' --data-raw '{ "producerName": "rest-producer", "messages": [ { "payload":"{\"TestCURL\":\"TestCURL\"}" } ] }' ``` Results in an output of: ``` {"messagePublishResults":[{"messageId":"15:1:-1","errorCode":0,"schemaVersion":0}],"schemaVersion":0}% ``` (success) 3. Configure broker.conf without insecure ports defined: ``` brokerServicePort= brokerServicePortTls=6651 webServicePort= webServicePortTls=8081 ``` After starting the broker, this command: ``` [~/Desktop/pulsar/apache-pulsar-3.0.0]: curl -v -k --cert ./certs/client.cert.pem --key ./certs/client.key.pem --location --request POST 'https://localhost:8081/topics/persistent/public/default/test' --header 'Content-Type: application/json' --data-raw '{ "producerName": "rest-producer", "messages": [ { "payload":"{\"TestCURL\":\"TestCURL\"}" } ] }' ``` Results in an output of: ``` Maximum (50) redirects followed curl: (47) Maximum (50) redirects followed ``` (failure) (conclusion that even in the newest version of 3.0, the error persists) 4. Test with pulsar-client admin tool For added measure, I tested the pulsar-client tool, and found that even with the broker.conf, producing and consuming still works. (in one terminal) ``` bin/pulsar-client consume -s sub public/default/test -n 0 ``` (in another terminal) ``` bin/pulsar-client produce public/default/test -m "---------hello apache pulsar2-------" -n 10 ``` (conclusion that without any changes, the REST producer api has an issue, but the pulsar-client tool doesn’t have it with the exact same broker configuration). I also looked through the code and the public git history in the REST Client source code, but didn’t see anything fixed. In conclusion, my tests show that under this configuration, the REST Producer has a bug that has not been fixed in any newer version. ### Solution This error dues to the fact that the logic to find the broker from the topic name doesn't handle the case where `brokerServicePort=` or `webServicePort=`. It can be found here https://github.com/apache/pulsar/blob/60dba5d675f998a5108a35be65649edd57ce8596/pulsar-broker/src/main/java/org/apache/pulsar/broker/rest/TopicsBase.java#L436 To fix, I propose to change the following lines: ```java LookupResult result = optionalResult.get(); if (result.getLookupData().getHttpUrl().equals(pulsar().getWebServiceAddress())) { // Current broker owns the topic, add to owning topic. ``` To: ```java LookupResult result = optionalResult.get(); if (result.getLookupData().getHttpUrl().equals(pulsar().getWebServiceAddress()) || result.getLookupData().getHttpUrlTls().equals(pulsar().getWebServiceAddressTls())) { // Current broker owns the topic, add to owning topic. ``` I built Pulsar on my local machine, and this fixes the issue. ### Alternatives Using an alternative producer/consumer (pulsar-admin is the one I tested), would be the best workaround without a code change, but I would like to use this client. ### Anything else? It looks very similar to https://github.com/apache/pulsar/pull/9260, but inside a different component. ### Are you willing to submit a PR? - [X] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
