wangjialing218 opened a new issue #11695:
URL: https://github.com/apache/pulsar/issues/11695


   **To Reproduce**
   1. Start pulsar cluster with 3 brokers and `functionsWorkerEnabled=true`
   2. Stop the broker which is the leader of function worker and the owner of 
function worker coordinate topic.
   3. Another broker will become leader and then shutdown due to fail to get 
coordinate topic.
   
   **Expected behavior**
   Should not trigger shutdown when fail to get coordinate topic.
   
   **Additional context**
   When stop the broker which is the leader of function woker and the owner of 
coordinate topic. Another broker will become leader but coordinate topic did 
not shift to another broker in time.
   There is a schedule task runing only on leader and will get follow error:
   
   > ERROR org.apache.pulsar.functions.worker.ErrorNotifierShutdownServiceImpl 
- Encountered fatal error. Shutting down.
   > java.lang.RuntimeException: 
org.apache.pulsar.client.admin.PulsarAdminException: 
java.util.concurrent.CompletionException: 
org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector$RetryException: 
Could not complete the operation. Number of retries has been exhausted. Failed 
reason: Connection refused: /xxx.xxx.xxx.xxx:8080
   >    at 
org.apache.pulsar.functions.worker.MembershipManager.getCurrentMembership(MembershipManager.java:78)
 
   
   `getCurrentMembership` will throw a `RuntimeException` when fail to get 
coordinate topic. And because we are using `ErrorNotifierShutdownServiceImpl` 
the broker will shutdown when get exception.
   
   There may be two parts need to be improved:
   1. Add error handle when get coordinate topic.
   2. Do not use `ErrorNotifierShutdownServiceImpl`, it is not suitable to 
shutdown broker when function worker get any exception.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to