Re: [VOTE] Release Spark 3.1.0 (RC1)

2021-01-05 Thread Hyukjin Kwon
Seems like we have two PRs for both blockers, and one is already merged, nice. I will wait for a couple of days more before starting a new RC to make sure we catch more regressions before the new RC. Please keep testing this RC. I would appreciate it :-). 2021년 1월 6일 (수) 오후 2:28, Hyukjin Kwon 님이

Re: [VOTE] Release Spark 3.1.0 (RC1)

2021-01-05 Thread Hyukjin Kwon
Actually, I will mark https://issues.apache.org/jira/browse/SPARK-34021 as a blocker too. For CRAN submission, we should fix it. 2021년 1월 6일 (수) 오후 1:47, Jungtaek Lim 님이 작성: > There's an issue SPARK-33635 [1] reported due to performance regression on > Kafka read between Spark 2.4 vs 3.0, which

Re: [VOTE] Release Spark 3.1.0 (RC1)

2021-01-05 Thread Jungtaek Lim
There's an issue SPARK-33635 [1] reported due to performance regression on Kafka read between Spark 2.4 vs 3.0, which sounds like a blocker. I'll mark this as a blocker, unless anyone has different opinions. 1. https://issues.apache.org/jira/browse/SPARK-33635 On Wed, Jan 6, 2021 at 9:01 AM

[VOTE] Release Spark 3.1.0 (RC1)

2021-01-05 Thread Hyukjin Kwon
Please vote on releasing the following candidate as Apache Spark version 3.1.0. The vote is open until January 8th 4PM PST and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 3.1.0 [ ] -1 Do not release this package because

Spark History UI Authentication through Azure Active Directory

2021-01-05 Thread GSRIK
Hello Team I am implementing /javax.servlet.filter/ to enable Spark History UI authentication through Azure Active Directory. I have added the following configuration to SPARK_HISTORY_OPTS: /spark.ui.filters=com.company.project.spark.history.AADAuthentication/ where AADAuthentication is the