We need to strictly follow the backport and release policy. We can't merge such a new feature into a RC branch or a minor release (e.g., 2.4.1).
Cheers, Xiao Bolke de Bruin <bdbr...@gmail.com> 于2018年10月16日周二 下午12:48写道： > Chiming in here. We are in the same boat as Bloomberg. > > (But being a release manager often myself I understand the trade-off) > > B. > > Op di 16 okt. 2018 21:24 schreef Ilan Filonenko <i...@cornell.edu>: > >> On Erik's note, would SPARK-23257 be included in, say, 2.4.1? When would >> the next RC be? I would like to propose the inclusion of the Kerberos >> feature sooner rather than later as it would increase Spark-on-K8S adoption >> in production workloads while bringing greater feature parity with Yarn and >> Mesos. I would like to note that the feature itself is isolated from Core >> and isolated via the step-based architecture of the Kubernetes >> Driver/Executor builders. >> >> Furthermore, Spark users traditionally use HDFS for storage and in >> production use-cases these HDFS clusters would be kerberized. At Bloomberg, >> for example, all of the HDFS clusters are kerberized and for this reason, >> the only thing stopping our internal Data Science Platform from adopting >> Spark-on-K8S is this feature. >> >> On Tue, Oct 16, 2018 at 10:21 AM Erik Erlandson <eerla...@redhat.com> >> wrote: >> >>> >>> SPARK-23257 merged more recently than I realized. If that isn't on >>> branch-2.4 then the first question is how soon on the release sequence that >>> can be adopted >>> >>> On Tue, Oct 16, 2018 at 9:33 AM Reynold Xin <r...@databricks.com> wrote: >>> >>>> We shouldn’t merge new features into release branches anymore. >>>> >>>> On Tue, Oct 16, 2018 at 6:32 PM Rob Vesse <rve...@dotnetrdf.org> wrote: >>>> >>>>> Right now the Kerberos support for Spark on K8S is only on master >>>>> AFAICT i.e. the feature is not present on branch-2.4 >>>>> >>>>> >>>>> >>>>> Therefore I don’t see any point in adding the tests into branch-2.4 >>>>> unless the plan is to also merge the Kerberos support to branch-2.4 >>>>> >>>>> >>>>> >>>>> Rob >>>>> >>>>> >>>>> >>>>> *From: *Erik Erlandson <eerla...@redhat.com> >>>>> *Date: *Tuesday, 16 October 2018 at 16:47 >>>>> *To: *dev <dev@spark.apache.org> >>>>> *Subject: *[DISCUSS][K8S][TESTS] Include Kerberos integration tests >>>>> for Spark 2.4 >>>>> >>>>> >>>>> >>>>> I'd like to propose including integration testing for Kerberos on the >>>>> Spark 2.4 release: >>>>> >>>>> https://github.com/apache/spark/pull/22608 >>>>> >>>>> >>>>> >>>>> Arguments in favor: >>>>> >>>>> 1) it improves testing coverage on a feature important for integrating >>>>> with HDFS deployments >>>>> >>>>> 2) its intersection with existing code is small - it consists >>>>> primarily of new testing code, with a bit of refactoring into 'main' and >>>>> 'test' sub-trees. These new tests appear stable. >>>>> >>>>> 3) Spark 2.4 is still in RC, with outstanding correctness issues. >>>>> >>>>> >>>>> >>>>> The argument 'against' that I'm aware of would be the relatively large >>>>> size of the PR. I believe this is considered above, but am soliciting >>>>> community feedback before committing. >>>>> >>>>> Cheers, >>>>> >>>>> Erik >>>>> >>>>> >>>>> >>>>
