nishantmonu51 edited a comment on issue #8801: KubernetesTaskRunner for running druid tasks as kubernetes jobs URL: https://github.com/apache/incubator-druid/issues/8801#issuecomment-548560274 > Sounds good to me overall. > > > Instead of MM pushing the task logs on completion to log storage, Overlord will fetch the logs and push them to the log storage > > Could this potentially become a bottleneck when scaling? This will be done asynchronously by a background thread. However, if it becomes a bottleneck, log publishing can be turned off and any of kubernetes logging methods can be used to collect job logs - recommendations are described here - https://kubernetes.io/docs/concepts/cluster-administration/logging/
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
