dongjoon-hyun edited a comment on issue #25641: [SPARK-28921][BUILD][K8S][2.4] Update kubernetes client to 4.4.2 URL: https://github.com/apache/spark/pull/25641#issuecomment-529271002 @Jeffwan . First fo all, could you tell me if EKS provides a fallback option (to launch the previous working EKS 1.13.x versions) for that outage? If not, I'm very sorry about that because I'm one of the EKS customers. For the outage you mentioned, I believe you know that the other K8s users (including AWS customers who are using their own K8s services) with old K8s versions are not suffering from this. There are many users with various environments unlikely managed EKS. Given that, the above production outage from EKS environment may be considered as a hidden pitfall of managed service providers due to lack of the downstream testing (or lack of consideration on customer use cases). More importantly, you should not assume that customers always live on the latest Spark releases. In the customer production environments, there are many used versions simultaneously; 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4 (the last EOL release), 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.4.4. So, new release cannot recover all production-down issues because old version' users have their reasons to stick to that. This is the reason why I think EKS had better provide a fallback option. You may recommend to upgrade Apache Spark versions, but what will happen if this kind of outages happens again and again frequently? We have many downstreams like EKS. As you know, technically, Apache Spark community didn't make a decision to break the customer production environments like this. Lastly, Apache Spark community is an independent community which has own release cycle and `2.4.4` is announced this Month and `2.4.3` was May 9th. We will release the next version when we have enough reasons to release.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
