A lot of developers may have already moved to 3.0.x, FYI 3.1.0 is just around the corner hopefully(in a few days) and has a lot of improvements to spark on K8s, including it will be transitioning from experimental to GA in this release.
See: https://issues.apache.org/jira/browse/SPARK-33005 Thanks, On Tue, Jan 5, 2021 at 12:41 AM Sachit Murarka <connectsac...@gmail.com> wrote: > Hi Users, > > Could you please tell which Spark version have you used in Production for > Kubernetes. > Which is a recommended version for Production provided that both Streaming > and core apis have to be used using Pyspark. > > Thanks ! > > Kind Regards, > Sachit Murarka >
