durdin85 opened a new issue, #14676:
URL: https://github.com/apache/cloudstack/issues/14676

   ### The required feature described as a wish
   
   I would like the RabbitMQ event bus to keep events that are published while 
the broker is unreachable and deliver them, in order, once the connection is 
back, within configurable limits (number of events and maximum
   age). 
   
   As we tested during the network outage, every event that is published while 
the broker is unreachable is lost. 
   
   At the moment it works like that events are sent with without publisher 
confirms, so when the connection is down, the event is discarded.
   The reconnect logic restores the connection only and nothing is replayed:
    - when `publish()` catches `AlreadyClosedException` it calls 
`closeConnection()`.
   That shutdown is application initiated, so `DisconnectHandler` ignores it 
and no
   `ReconnectionTask` is started. The next `publish()` then connects on the 
calling thread, which can block for up to the client's connection timeout if 
the host does not answer.
   
   Another observation - the RabbitMQ client reports any connection close it 
did not initiate as "shutdown by AMQP server", including closes by a load 
balancer idle timeout.
   
   Also it seems, the current code has no confirmation mechanism, so we never 
know the messages really reached the AMQP.
   
   #### Proposal
   
   - Keep events that cannot be sent in a bounded in-memory FIFO buffer and send
     them in order after the reconnect.
   - Two optional properties on the existing event bus bean: maximum number of
     buffered events (default 1000, oldest dropped first) and maximum age of a
     buffered event (default 5 minutes).
   - Buffer only after the broker was reachable at least once; before that keep
     today's behavior.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

  • [I] Feature: AMQP event bus -... durdin85 (via GitHub) via commits

Reply via email to