mariuskimmina opened a new issue, #14952:
URL: https://github.com/apache/druid/issues/14952

   Please provide a detailed title (e.g. "Broker crashes when using TopN query 
with Bound filter" instead of just "Broker crashes").
   
   ### Affected Version
   
   26.0.0
   
   ### Description
   
   We are encountering an issue where once we increase the number of middle 
managers to anything greater than 2 they are not picking up tasks effectively 
anymore. Only 1 middle manager is then using only 1 Slot and only 1 Task can be 
running and a time while all other tasks get stuck in a `Pending` state. See 
the image below for the tasks view.
   
   
![image](https://github.com/apache/druid/assets/38843153/20767778-a104-4fee-bf8b-f65f5bebb4c6)
   
   See the screenshot below showing only 1 slot being used
   
   
![image](https://github.com/apache/druid/assets/38843153/0ac8880a-5dee-4365-9afd-a5fd9b06bfe6)
   
   When we have only 2 middle managers, everything is fine, the slots are being 
used and the task get picked up as expected.
   The issue arises when using more than 2 middle managers, we tried it with 3, 
4 and 7 middle managers, the problem has always been the same.
   
   We are running on k8s deployed with druid-operator.
   The screenshot below gives an overview over the cluster.
   
   
![image](https://github.com/apache/druid/assets/38843153/78eac801-7543-4a6c-8b33-782abd48b56e)
   
   The problem first occurred sometime after we upgraded our MySQL database 
(rds) from 5.7 to 8.0.
   
   Any guidance in troubleshooting this issue would be appreciated.
   
   
   Middle manager logs: https://pastebin.com/xwjNHUJV
   Coordinator logs: https://pastebin.com/LLuE3SK5
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to