suneet-s commented on PR #13842: URL: https://github.com/apache/druid/pull/13842#issuecomment-1444769560
> It appears that the policy is injected, which means the same policy for the entire cluster. Is there a need to allow the policy to be selected per task to better meed the needs per-datasource? We'd register the set of policies, and each task would name one of them, with some default. This is something I had thought about working towards. Today, there is a cluster wide configuration for max number of compaction task slots. So the policy is used as a tie breaker across datasources. In the future, it could make sense for a datasource to be able to say it prefers a search policy like "most fragmented intervals first" vs another datasource that cares about "newest interval first". But the cluster will need to decide how to choose which policy to apply when comparing intervals across these datasources. This sent my head spinning, so I figured I'd try to cross that bridge when I came to it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
