holdenk commented on issue #23347: allow building spark gpu docker images URL: https://github.com/apache/spark/pull/23347#issuecomment-449543693 Hi @RongGu there's some complications around this. Publishing to docker images as an ASF project is a bit complicated because of GPLd code in the image, I've been meaning to follow up on this but life has been kind of busy. The separate images do seem a bit silly I agree, but the overhead of having a combined image could be non-trivial (e.g. shipping Python to every worker when it's not required.) I agree with @felixcheung that adding more image types gives us more maintenance burden, if we could simplify this down (maybe have param inside of the other docker files) making it easier for end users to deploy GPU workloads does seem important. I know @ifilonenko has given a lot of thought about these issues so I'd really value his opinion here.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
