xushiyan commented on issue #3344: URL: https://github.com/apache/hudi/issues/3344#issuecomment-1001120072
@Kavin88 Gonna close this as this is not a hudi-specific problem; it's a general orchestration decision problem for running large number of spark applications. You can look into apache livy that can help manage spark applications. if you run spark on kubernetes, then k8s helps with the orchestration -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
