holdenk commented on pull request #841: URL: https://github.com/apache/incubator-kyuubi/pull/841#issuecomment-884364030
I like the simplicity of this approach, but I think there are some pretty serious limitations we should think through. It would let us scale up and down executor pods but not the shuffle services and we'd probably end up doing a lot of remote fetches. Can you give me some more context around your plans here? Have you looked at the current work on introducing the new shuffle APIs for Kube in Spark? Is the goal just to support dynamic scaling? If so have you tried enabling migrations? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
