[GitHub] spark pull request #22977: [BUILD] Bump previousSparkVersion in MimaBuild.sc...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22977#discussion_r232456139
--- Diff: project/MimaExcludes.scala ---
@@ -84,7 +84,17 @@ object MimaExcludes {
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.streaming.api.java.JavaPairDStream.flatMapValues"),
// [SPARK-25680] SQL execution listener shouldn't happen on execution
thread
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.sql.util.ExecutionListenerManager.clone"),
-
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.util.ExecutionListenerManager.this")
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.util.ExecutionListenerManager.this"),
+// [SPARK-25862][SQL] Remove rangeBetween APIs introduced in
SPARK-21608
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.functions.unboundedFollowing"),
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.functions.unboundedPreceding"),
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.functions.currentRow"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.sql.expressions.Window.rangeBetween"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.sql.expressions.WindowSpec.rangeBetween"),
+// [SPARK-23781][CORE] Merge token renewer functionality into
HadoopDelegationTokenManager
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.deploy.SparkHadoopUtil.nextCredentialRenewalTime"),
+// [SPARK-25908][CORE][SQL] Remove old deprecated items in Spark 3
--- End diff --
@cloud-fan really these two lines should move to the top, because the first
long chunk of exclusions is for 25908.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22977: [BUILD] Bump previousSparkVersion in MimaBuild.sc...
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22977#discussion_r232005093
--- Diff: project/MimaBuild.scala ---
@@ -88,7 +88,7 @@ object MimaBuild {
def mimaSettings(sparkHome: File, projectRef: ProjectRef) = {
val organization = "org.apache.spark"
-val previousSparkVersion = "2.2.0"
+val previousSparkVersion = "2.4.0"
--- End diff --
Thanks, @cloud-fan and @srowen . +1 for another PR for `branch-2.4`.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22977: [BUILD] Bump previousSparkVersion in MimaBuild.sc...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22977#discussion_r231935977
--- Diff: project/MimaBuild.scala ---
@@ -88,7 +88,7 @@ object MimaBuild {
def mimaSettings(sparkHome: File, projectRef: ProjectRef) = {
val organization = "org.apache.spark"
-val previousSparkVersion = "2.2.0"
+val previousSparkVersion = "2.4.0"
--- End diff --
I believe @cloud-fan is right here. If master is 3.0, the previous version
is 2.4.0 in master. Analogously for branch 2.4. Yes sounds like this should be
a step in the release process somewhere.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22977: [BUILD] Bump previousSparkVersion in MimaBuild.sc...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22977#discussion_r231936644
--- Diff: project/MimaExcludes.scala ---
@@ -84,7 +84,17 @@ object MimaExcludes {
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.streaming.api.java.JavaPairDStream.flatMapValues"),
// [SPARK-25680] SQL execution listener shouldn't happen on execution
thread
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.sql.util.ExecutionListenerManager.clone"),
-
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.util.ExecutionListenerManager.this")
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.util.ExecutionListenerManager.this"),
+// [SPARK-25862][SQL] Remove rangeBetween APIs introduced in
SPARK-21608
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.functions.unboundedFollowing"),
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.functions.unboundedPreceding"),
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.functions.currentRow"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.sql.expressions.Window.rangeBetween"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.sql.expressions.WindowSpec.rangeBetween"),
+// [SPARK-23781][CORE] Merge token renewer functionality into
HadoopDelegationTokenManager
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.deploy.SparkHadoopUtil.nextCredentialRenewalTime"),
--- End diff --
It was a `private[spark]` method which has to be implemented as a public
method in the Java bytecode. MiMa doesn't know the difference. Yes it can be
ignore for sure.
So these failures are 'new' because they were introduced after Spark 2.2
(hence not missing since 2.2) then removed, so bumping the previous version
causes them to fail now? OK.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22977: [BUILD] Bump previousSparkVersion in MimaBuild.sc...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22977#discussion_r231872904
--- Diff: project/MimaExcludes.scala ---
@@ -84,7 +84,17 @@ object MimaExcludes {
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.streaming.api.java.JavaPairDStream.flatMapValues"),
// [SPARK-25680] SQL execution listener shouldn't happen on execution
thread
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.sql.util.ExecutionListenerManager.clone"),
-
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.util.ExecutionListenerManager.this")
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.util.ExecutionListenerManager.this"),
+// [SPARK-25862][SQL] Remove rangeBetween APIs introduced in
SPARK-21608
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.functions.unboundedFollowing"),
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.functions.unboundedPreceding"),
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.functions.currentRow"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.sql.expressions.Window.rangeBetween"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.sql.expressions.WindowSpec.rangeBetween"),
+// [SPARK-23781][CORE] Merge token renewer functionality into
HadoopDelegationTokenManager
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.deploy.SparkHadoopUtil.nextCredentialRenewalTime"),
--- End diff --
This is actually a private method, I'm not sure why mima tracks it.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22977: [BUILD] Bump previousSparkVersion in MimaBuild.sc...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22977#discussion_r231866958
--- Diff: project/MimaBuild.scala ---
@@ -88,7 +88,7 @@ object MimaBuild {
def mimaSettings(sparkHome: File, projectRef: ProjectRef) = {
val organization = "org.apache.spark"
-val previousSparkVersion = "2.2.0"
+val previousSparkVersion = "2.4.0"
--- End diff --
up to my understanding, we should have changed it to 2.3.0 when 2.3.0 was
released.
Maybe we should send a another PR to branch-2.4 and change it to 2.3.0
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22977: [BUILD] Bump previousSparkVersion in MimaBuild.sc...
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22977#discussion_r231860289
--- Diff: project/MimaBuild.scala ---
@@ -88,7 +88,7 @@ object MimaBuild {
def mimaSettings(sparkHome: File, projectRef: ProjectRef) = {
val organization = "org.apache.spark"
-val previousSparkVersion = "2.2.0"
+val previousSparkVersion = "2.4.0"
--- End diff --
According to https://issues.apache.org/jira/browse/SPARK-23070, `2.3.0`
seems to be the correct version in this PR.
cc @gatorsmile .
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22977: [BUILD] Bump previousSparkVersion in MimaBuild.sc...
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/22977 [BUILD] Bump previousSparkVersion in MimaBuild.scala to be 2.4.0 ## What changes were proposed in this pull request? Since Spark 2.4.0 is already in maven repo, we can Bump previousSparkVersion in MimaBuild.scala to be 2.4.0. Note that, seems we forgot to do it for 2.4.0, so this PR also updates MimaExcludes.scala ## How was this patch tested? N/A You can merge this pull request into a Git repository by running: $ git pull https://github.com/cloud-fan/spark mima Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/22977.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #22977 commit a0357782e75967d309dc5229b3c36a0c295f2956 Author: Wenchen Fan Date: 2018-11-08T11:34:31Z Bump previousSparkVersion in MimaBuild.scala to be 2.2.0 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
