MrThreepwood opened a new issue #10092:
URL: https://github.com/apache/pulsar/issues/10092


   **Is your feature request related to a problem? Please describe.**
   We are currently using the Java client to work queues of data. This data 
needs to be processed in order. We create a thread  that then just constantly 
loops trying to grab new messages from this queue. Occasionally these queues 
stop being processed (perhaps an OOM error, but we aren't certain). Ideally we 
would like a health check to be able to confirm that the queues are still being 
worked by checking the age of the first message (for our purposes, it's also 
the oldest message in the queue), and restart a worker if this age gets beyond 
an acceptable value.
   
   **Describe the solution you'd like**
   Ideally there would be a way for a consumer to check the age of the oldest 
message without having to take the next message (therefore potentially 
disrupting processing order).
   
   **Describe alternatives you've considered**
   We could also potentially try managing a thread lock to prevent the health 
check from grabbing items while a worker is working, and prevent a worker from 
grabbing an item while a health check is checking health. However, this 
significantly complicates the code base and could lead to fairly long health 
checks. It also entangles the health check and worker more than we'd like as we 
don't currently understand what's causing the workers to stop processing the 
queue. If it were to grab a lock and not let go, we'd have our same problem all 
over again.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to