sachidananda007 opened a new issue, #15383: URL: https://github.com/apache/druid/issues/15383
I observed this issue in druid 25.0.0 Due to design changes, there is a increase in number of smaller ingestion tasks submitted to the druid cluster. The total number of files submitted are constant. the tasks submitted earlier had large number of files now the they have small number of files. However, the overlord stops responding once the number of pending tasks goes beyond ~5k. This is my observation not sure about the exact number and I can see taskQueue eating up all the resource. As show in below screenshot <img width="383" alt="Screenshot 2023-11-16 at 11 01 29 AM" src="https://github.com/apache/druid/assets/106651335/6b222bd3-4b94-4458-a703-a153f7f6fc93";> <img width="644" alt="Screenshot 2023-11-16 at 10 46 43 AM" src="https://github.com/apache/druid/assets/106651335/4c7a7c90-db47-45ed-b917-0f31575897f6";> I tried to increase instance type which didn't help. I needed to limit the[ queue size](https://github.com/apache/druid/issues/15380). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
