gianm commented on issue #17849: URL: https://github.com/apache/druid/issues/17849#issuecomment-2768872835
Recently work has been ongoing to improve Overlord scalability. It is possible that upgrading will help. An important item is batch segment allocation (`druid.indexer.tasklock.batchSegmentAllocation`) which I believe is on by default in your version 30.0.0. But it has been improved in 32.0.0 with #17496. And I expect that #17653, which will be included in 33.0.0, will help even more with a wider variety of operations. There are also recent improvements to lock granularity within the Overlord (breaking up large locks into smaller ones so more operations can proceed simultaneously), but I don't have the PR numbers handy for those. Besides that, it tends to help to ensure your metadata store is responding quickly, scaling it up if necessary. And it tends to help to ensure you don't have too many unused or pending segment records lying around. On one of your further questions: it definitely would be nice to have the `2m` timeout in `RequestBuilder` be configurable. If you submit a PR for that, it would be appreciated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
