yupeng9 commented on issue #5627: URL: https://github.com/apache/incubator-pinot/issues/5627#issuecomment-652005534
Thanks for sharing the thoughts. I think it's a promising direction for users who query Pinot through Presto, as we can leverage the resource management layer in Presto. At the minimum, we could add policies to control the queue size, if we could add a policy for the ad-hoc users, which might solve the challenge #2. To further utilize the constraints on CPU and memory, I think we need an integration with Pinot server/broker for the statistics collection? I was also thinking of another direction like Query gateway to detect and block bad queries with some heuristics, e.g. SELECT with large time range scanning too many segments. > One quick hack: If we query Pinot via Presto then we can leverage Presto's resource groups (which is more of an admission control mechanism rather than a Yarn-queue like mechanism). Doing up front costing of the query shouldn't be too hard either if we can get apriori stats from Pinot, again leveraging Presto's pre existing CBO. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
