Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
dongjoon-hyun commented on PR #45838: URL: https://github.com/apache/spark/pull/45838#issuecomment-2046598213 Thank you, @pan3793 and all. Merged to master for Apache Spark 4.0.0. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
dongjoon-hyun closed pull request #45838: [SPARK-47706][BUILD] Bump json4s 4.0.7 URL: https://github.com/apache/spark/pull/45838 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
pan3793 commented on PR #45838: URL: https://github.com/apache/spark/pull/45838#issuecomment-2046557968 @LuciferYang @dongjoon-hyun The failed test is known flaky case, I re-triggered CI, thanks for your approval and help with Mima -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
dongjoon-hyun commented on PR #45838: URL: https://github.com/apache/spark/pull/45838#issuecomment-2046536383 Could you re-trigger the failed test pipeline, @pan3793 ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
pan3793 commented on PR #45838: URL: https://github.com/apache/spark/pull/45838#issuecomment-2046420830 rebased on master -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
dongjoon-hyun commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1557856537
##
sql/api/pom.xml:
##
@@ -56,13 +56,6 @@
org.json4s
json4s-jackson_${scala.binary.version}
-3.7.0-M11
Review Comment:
Thanks. I merged it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
pan3793 commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1556905110
##
sql/api/pom.xml:
##
@@ -56,13 +56,6 @@
org.json4s
json4s-jackson_${scala.binary.version}
-3.7.0-M11
Review Comment:
opened https://github.com/apache/spark/pull/45943
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
dongjoon-hyun commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1556899643
##
sql/api/pom.xml:
##
@@ -56,13 +56,6 @@
org.json4s
json4s-jackson_${scala.binary.version}
-3.7.0-M11
Review Comment:
Could you spin off this file change as a new PR, please, @pan3793 ? We can
merge that before this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
pan3793 commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1556900847
##
sql/api/pom.xml:
##
@@ -56,13 +56,6 @@
org.json4s
json4s-jackson_${scala.binary.version}
-3.7.0-M11
Review Comment:
Okay
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
dongjoon-hyun commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1556899643
##
sql/api/pom.xml:
##
@@ -56,13 +56,6 @@
org.json4s
json4s-jackson_${scala.binary.version}
-3.7.0-M11
Review Comment:
Could you spin off this as a new PR, please, @pan3793 ? We can merge that
before this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
dongjoon-hyun commented on PR #45838: URL: https://github.com/apache/spark/pull/45838#issuecomment-2044149193 Thank you for rebasing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
pan3793 commented on PR #45838: URL: https://github.com/apache/spark/pull/45838#issuecomment-2044102847 @dongjoon-hyun @cloud-fan @gengliangwang @LuciferYang I updated MimaExcludes, now it looks good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
pan3793 commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1556835313
##
project/MimaExcludes.scala:
##
@@ -90,7 +90,21 @@ object MimaExcludes {
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLQueryBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLBuilder.this"),
-
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this")
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this"),
+// SPARK-47706: Bump json4s from 3.7.0-M11 to 4.0.7
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.sql.types.DataType#JSortedObject.unapplySeq"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.sql.expressions.MutableAggregationBuffer.jsonValue"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.params"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.defaultParams"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.metadata"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.getParamValue"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$5"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$6"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$7"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.this"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.apply"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.mllib.tree.model.TreeEnsembleModel#SaveLoadV1_0.readMetadata")
Review Comment:
this should not be an issue
```
private[tree] object TreeEnsembleModel extends Logging {
object SaveLoadV1_0 {
...
}
...
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
pan3793 commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1556834466
##
project/MimaExcludes.scala:
##
@@ -90,7 +90,11 @@ object MimaExcludes {
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLQueryBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLBuilder.this"),
-
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this")
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this"),
+// SPARK-47706: Bump json4s from 3.7.0-M11 to 4.0.7
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.sql.expressions.MutableAggregationBuffer.jsonValue"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.sql.types.DataType#JSortedObject.unapplySeq"),
Review Comment:
this should not be a real issue
```
object DataType {
private object JSortedObject {
...
}
...
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
pan3793 commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1556833527
##
project/MimaExcludes.scala:
##
@@ -90,7 +90,11 @@ object MimaExcludes {
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLQueryBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLBuilder.this"),
-
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this")
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this"),
+// SPARK-47706: Bump json4s from 3.7.0-M11 to 4.0.7
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.sql.expressions.MutableAggregationBuffer.jsonValue"),
Review Comment:
it's weird
```
trait Row {
...
private[sql] def jsonValue: JValue
...
}
abstract class MutableAggregationBuffer extends Row {
...
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
dongjoon-hyun commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1556656706
##
project/MimaExcludes.scala:
##
@@ -90,7 +90,21 @@ object MimaExcludes {
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLQueryBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLBuilder.this"),
-
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this")
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this"),
+// SPARK-47706: Bump json4s from 3.7.0-M11 to 4.0.7
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.sql.types.DataType#JSortedObject.unapplySeq"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.sql.expressions.MutableAggregationBuffer.jsonValue"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.params"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.defaultParams"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.metadata"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.getParamValue"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$5"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$6"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$7"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.this"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.apply"),
Review Comment:
I filed SPARK-47770.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
dongjoon-hyun commented on PR #45838: URL: https://github.com/apache/spark/pull/45838#issuecomment-2043927068 To @pan3793 , for MIMA issue, I made the following PR to fix it. - #45938 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
dongjoon-hyun commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1556539422
##
project/MimaExcludes.scala:
##
@@ -90,7 +90,21 @@ object MimaExcludes {
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLQueryBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLBuilder.this"),
-
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this")
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this"),
+// SPARK-47706: Bump json4s from 3.7.0-M11 to 4.0.7
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.sql.types.DataType#JSortedObject.unapplySeq"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.sql.expressions.MutableAggregationBuffer.jsonValue"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.params"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.defaultParams"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.metadata"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.getParamValue"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$5"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$6"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$7"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.this"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.apply"),
Review Comment:
I took a look at that briefly and found that `dev/mima` shows error for this
class already.
```
$ dev/mima | grep org.apache.spark.ml.util.DefaultParamsReader
Using SPARK_LOCAL_IP=localhost
Using SPARK_LOCAL_IP=localhost
Error instrumenting
class:org.apache.spark.ml.util.DefaultParamsReader$Metadata$
Error instrumenting
class:org.apache.spark.ml.util.DefaultParamsReader$Metadata
Using SPARK_LOCAL_IP=localhost
spark/connect/relations.proto:26:1: warning: Import
spark/connect/common.proto is unused.
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
dongjoon-hyun commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1556539422
##
project/MimaExcludes.scala:
##
@@ -90,7 +90,21 @@ object MimaExcludes {
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLQueryBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLBuilder.this"),
-
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this")
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this"),
+// SPARK-47706: Bump json4s from 3.7.0-M11 to 4.0.7
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.sql.types.DataType#JSortedObject.unapplySeq"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.sql.expressions.MutableAggregationBuffer.jsonValue"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.params"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.defaultParams"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.metadata"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.getParamValue"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$5"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$6"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$7"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.this"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.apply"),
Review Comment:
I took a look at that briefly and found that `dev/mima` shows error for this
class already.
```
$ dev/mima | grep org.apache.spark.ml.util.DefaultParamsReader
Using SPARK_LOCAL_IP=localhost
Using SPARK_LOCAL_IP=localhost
Error instrumenting
class:org.apache.spark.ml.util.DefaultParamsReader$Metadata$
Error instrumenting
class:org.apache.spark.ml.util.DefaultParamsReader$Metadata
Using SPARK_LOCAL_IP=localhost
spark/connect/relations.proto:26:1: warning: Import
spark/connect/common.proto is unused.
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
dongjoon-hyun commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1556151836
##
project/MimaExcludes.scala:
##
@@ -90,7 +90,21 @@ object MimaExcludes {
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLQueryBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLBuilder.this"),
-
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this")
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this"),
+// SPARK-47706: Bump json4s from 3.7.0-M11 to 4.0.7
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.sql.types.DataType#JSortedObject.unapplySeq"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.sql.expressions.MutableAggregationBuffer.jsonValue"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.params"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.defaultParams"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.metadata"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.getParamValue"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$5"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$6"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$7"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.this"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.apply"),
Review Comment:
Could you file a JIRA issue for this, @pan3793 ? It seems that we had better
fix that first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
dongjoon-hyun commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1556151836
##
project/MimaExcludes.scala:
##
@@ -90,7 +90,21 @@ object MimaExcludes {
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLQueryBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLBuilder.this"),
-
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this")
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this"),
+// SPARK-47706: Bump json4s from 3.7.0-M11 to 4.0.7
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.sql.types.DataType#JSortedObject.unapplySeq"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.sql.expressions.MutableAggregationBuffer.jsonValue"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.params"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.defaultParams"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.metadata"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.getParamValue"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$5"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$6"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$7"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.this"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.apply"),
Review Comment:
Could you file a JIRA issue for this, @pan3793 ? We had better fix that
first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
pan3793 commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1555230740
##
project/MimaExcludes.scala:
##
@@ -90,7 +90,21 @@ object MimaExcludes {
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLQueryBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLBuilder.this"),
-
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this")
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this"),
+// SPARK-47706: Bump json4s from 3.7.0-M11 to 4.0.7
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.sql.types.DataType#JSortedObject.unapplySeq"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.sql.expressions.MutableAggregationBuffer.jsonValue"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.params"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.defaultParams"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.metadata"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.getParamValue"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$5"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$6"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$7"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.this"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.apply"),
Review Comment:
I noticed that `DefaultParamsReader`'s scope is `private[ml]`, but Mima
still complains, is it expected? cc @LuciferYang
```
private[ml] class DefaultParamsReader
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
pan3793 commented on PR #45838: URL: https://github.com/apache/spark/pull/45838#issuecomment-2041867141 @cloud-fan seems json4s does not have release note. the API binary incompatible does not affect JSON serialize/deserialize, the case of parsing event log generated by the previous Spark version is already covered by Spark UT. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
cloud-fan commented on PR #45838: URL: https://github.com/apache/spark/pull/45838#issuecomment-2041836659 does json4s release notes mention anything about backward compatibility? We have persistent event logs in JSON format that may need to be consumed by new version of Spark history server. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
gengliangwang commented on PR #45838: URL: https://github.com/apache/spark/pull/45838#issuecomment-2041235111 Since there are many connectors/libraries that depend on Spark, I am +0 on this one. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
pan3793 commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1549972129
##
connector/protobuf/src/test/scala/org/apache/spark/sql/protobuf/ProtobufFunctionsSuite.scala:
##
@@ -22,7 +22,6 @@ import java.time.Duration
import scala.jdk.CollectionConverters._
import com.google.protobuf.{Any => AnyProto, BoolValue, ByteString,
BytesValue, DoubleValue, DynamicMessage, FloatValue, Int32Value, Int64Value,
StringValue, UInt32Value, UInt64Value}
-import org.json4s.StringInput
Review Comment:
No, it's there. But method `JsonMethods#parse` signature changed. As this is
the test code, I chose the simplest way to change it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
pan3793 commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1549948290
##
project/MimaExcludes.scala:
##
@@ -90,7 +90,21 @@ object MimaExcludes {
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLQueryBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLBuilder.this"),
-
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this")
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this"),
+// SPARK-47706: Bump json4s from 3.7.0-M11 to 4.0.7
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.sql.types.DataType#JSortedObject.unapplySeq"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.sql.expressions.MutableAggregationBuffer.jsonValue"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.params"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.defaultParams"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.metadata"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.getParamValue"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$5"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$6"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$7"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.this"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.apply"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.mllib.tree.model.TreeEnsembleModel#SaveLoadV1_0.readMetadata")
Review Comment:
This is because the `JValue` package is changed, it does not mean can not
read the model saved by the old Spark
```
def readMetadata(metadata: JValue): Metadata
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
dongjoon-hyun commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1549942965
##
connector/protobuf/src/test/scala/org/apache/spark/sql/protobuf/ProtobufFunctionsSuite.scala:
##
@@ -22,7 +22,6 @@ import java.time.Duration
import scala.jdk.CollectionConverters._
import com.google.protobuf.{Any => AnyProto, BoolValue, ByteString,
BytesValue, DoubleValue, DynamicMessage, FloatValue, Int32Value, Int64Value,
StringValue, UInt32Value, UInt64Value}
-import org.json4s.StringInput
Review Comment:
Is this removed or deprecated?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
dongjoon-hyun commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1549937536
##
project/MimaExcludes.scala:
##
@@ -90,7 +90,21 @@ object MimaExcludes {
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.MySQLDialect#MySQLSQLQueryBuilder.this"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLBuilder.this"),
-
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this")
+
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.jdbc.OracleDialect#OracleSQLQueryBuilder.this"),
+// SPARK-47706: Bump json4s from 3.7.0-M11 to 4.0.7
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.sql.types.DataType#JSortedObject.unapplySeq"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.sql.expressions.MutableAggregationBuffer.jsonValue"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.params"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.defaultParams"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.metadata"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.getParamValue"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$5"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$6"),
+
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$7"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.this"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.apply"),
+
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.mllib.tree.model.TreeEnsembleModel#SaveLoadV1_0.readMetadata")
Review Comment:
Wow.. Everything is inevitable?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
pan3793 commented on PR #45838: URL: https://github.com/apache/spark/pull/45838#issuecomment-2034852723 @LuciferYang `MimaExcludes` is updated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
pan3793 commented on PR #45838: URL: https://github.com/apache/spark/pull/45838#issuecomment-2034104674 @LuciferYang json4s classes leaked to public(maybe? at least in the Mima audit list) API, and json4s 4.x changed those classes, this requires the Spark community to evaluate if we should do such an upgrading -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
LuciferYang commented on PR #45838:
URL: https://github.com/apache/spark/pull/45838#issuecomment-2034076896
I get errors when I manually run the mima check. Could you please update the
`MimaExcludes`?
```
[error] spark-sql-api: Failed binary compatibility check against
org.apache.spark:spark-sql-api_2.13:3.5.0! Found 1 potential problems (filtered
170)
[error] * method unapplySeq(org.json4s.JsonAST#JValue)scala.Option in
object org.apache.spark.sql.types.DataType#JSortedObject's type is different in
current version, where it is (org.json4s.JValue)scala.Option instead of
(org.json4s.JsonAST#JValue)scala.Option
[error]filter with:
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.sql.types.DataType#JSortedObject.unapplySeq")
[error] spark-sql: Failed binary compatibility check against
org.apache.spark:spark-sql_2.13:3.5.0! Found 1 potential problems (filtered 861)
[error] * method jsonValue()org.json4s.JsonAST#JValue in class
org.apache.spark.sql.expressions.MutableAggregationBuffer has a different
result type in current version, where it is org.json4s.JValue rather than
org.json4s.JsonAST#JValue
[error]filter with:
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.sql.expressions.MutableAggregationBuffer.jsonValue")
[error] spark-mllib: Failed binary compatibility check against
org.apache.spark:spark-mllib_2.13:3.5.0! Found 11 potential problems (filtered
477)
[error] * method params()org.json4s.JsonAST#JValue in class
org.apache.spark.ml.util.DefaultParamsReader#Metadata has a different result
type in current version, where it is org.json4s.JValue rather than
org.json4s.JsonAST#JValue
[error]filter with:
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.params")
[error] * method defaultParams()org.json4s.JsonAST#JValue in class
org.apache.spark.ml.util.DefaultParamsReader#Metadata has a different result
type in current version, where it is org.json4s.JValue rather than
org.json4s.JsonAST#JValue
[error]filter with:
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.defaultParams")
[error] * method metadata()org.json4s.JsonAST#JValue in class
org.apache.spark.ml.util.DefaultParamsReader#Metadata has a different result
type in current version, where it is org.json4s.JValue rather than
org.json4s.JsonAST#JValue
[error]filter with:
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.metadata")
[error] * method getParamValue(java.lang.String)org.json4s.JsonAST#JValue
in class org.apache.spark.ml.util.DefaultParamsReader#Metadata has a different
result type in current version, where it is org.json4s.JValue rather than
org.json4s.JsonAST#JValue
[error]filter with:
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.getParamValue")
[error] * method
copy(java.lang.String,java.lang.String,Long,java.lang.String,org.json4s.JsonAST#JValue,org.json4s.JsonAST#JValue,org.json4s.JsonAST#JValue,java.lang.String)org.apache.spark.ml.util.DefaultParamsReader#Metadata
in class org.apache.spark.ml.util.DefaultParamsReader#Metadata's type is
different in current version, where it is
(java.lang.String,java.lang.String,Long,java.lang.String,org.json4s.JValue,org.json4s.JValue,org.json4s.JValue,java.lang.String)org.apache.spark.ml.util.DefaultParamsReader#Metadata
instead of
(java.lang.String,java.lang.String,Long,java.lang.String,org.json4s.JsonAST#JValue,org.json4s.JsonAST#JValue,org.json4s.JsonAST#JValue,java.lang.String)org.apache.spark.ml.util.DefaultParamsReader#Metadata
[error]filter with:
ProblemFilters.exclude[IncompatibleMethTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy")
[error] * synthetic method copy$default$5()org.json4s.JsonAST#JValue in
class org.apache.spark.ml.util.DefaultParamsReader#Metadata has a different
result type in current version, where it is org.json4s.JValue rather than
org.json4s.JsonAST#JValue
[error]filter with:
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$5")
[error] * synthetic method copy$default$6()org.json4s.JsonAST#JValue in
class org.apache.spark.ml.util.DefaultParamsReader#Metadata has a different
result type in current version, where it is org.json4s.JValue rather than
org.json4s.JsonAST#JValue
[error]filter with:
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.util.DefaultParamsReader#Metadata.copy$default$6")
[error] * synthetic method copy$default$7()org.json4s.JsonAST#JValue in
class org.apache.spark.ml.util.DefaultParamsReader#Metadata has a different
result type in current version, where it is org.json4s.JValue
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
LuciferYang commented on PR #45838: URL: https://github.com/apache/spark/pull/45838#issuecomment-2034039323 Previously, there were two unsuccessful upgrade attempts: - https://github.com/apache/spark/pull/32636 - https://github.com/apache/spark/pull/33630 cc @dongjoon-hyun @sarutak @JoshRosen FYI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
Re: [PR] [SPARK-47706][BUILD] Bump json4s 4.0.7 [spark]
pan3793 commented on code in PR #45838:
URL: https://github.com/apache/spark/pull/45838#discussion_r1549070164
##
sql/api/pom.xml:
##
@@ -56,13 +56,6 @@
org.json4s
json4s-jackson_${scala.binary.version}
-3.7.0-M11
Review Comment:
redundant, it's same as the one defined in root `pom.xml`
`dependencyManagement`.
##
sql/api/pom.xml:
##
@@ -56,13 +56,6 @@
org.json4s
json4s-jackson_${scala.binary.version}
-3.7.0-M11
Review Comment:
redundant, it same as the one defined in root `pom.xml`
`dependencyManagement`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
