[GitHub] spark issue #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson version ...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/22372 Btw, I don't think we can run current Spark with Hadoop 3.1 without any change. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson version ...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/22372 Do we officially support hadoop3 in branch 2.4? If branch 2.4 doesn't target to support Hadoop3 and this fix is only for Hadoop3, then I don't think it is meaningful to have this fix. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson version ...
Github user wangyum commented on the issue:
https://github.com/apache/spark/pull/22372
I did a simple test for 2.9.6. It works well. But that pr for 3.0. It means
that a simple test on branch 2.4 will fail:
```scala
scala> spark.range(10).write.parquet("/tmp/spark/parquet")
com.fasterxml.jackson.databind.JsonMappingException: Incompatible Jackson
version: 2.7.8
at
com.fasterxml.jackson.module.scala.JacksonModule$class.setupModule(JacksonModule.scala:64)
at
com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:19)
at
com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:730)
at
org.apache.spark.rdd.RDDOperationScope$.(RDDOperationScope.scala:82)
at
org.apache.spark.rdd.RDDOperationScope$.(RDDOperationScope.scala)
```
How about merge this pr to branch-2.4 only?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson version ...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/22372 Jackson version below 2.9.5 has CVE issues, I would suggest to upgrade to 2.9.6 as #21596 did. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson version ...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22372 Also, I think we should fix https://github.com/apache/spark/pull/21588 first. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson version ...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22372 Also, IIRC, https://github.com/apache/spark/pull/21596 change is needed for Jackson upgrade. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson version ...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22372 @wangyum, are you doubly sure if still SBT uses the Jackson? I roughly tried this a bit of while ago and found SBT doesn't pick up Maven's. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson version ...
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22372 We do not have jenkins tests for 3.1 profile: https://github.com/apache/spark/blob/395860a986987886df6d60fd9b26afd818b2cb39/dev/run-tests.py#L307-L310 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson version ...
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22372 do we have jenkins tests for 3.1 profile? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson version ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22372 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson version ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22372 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95846/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson version ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22372 **[Test build #95846 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95846/testReport)** for PR 22372 at commit [`f68083a`](https://github.com/apache/spark/commit/f68083ab07df6fedf8a30c94e706a74e0c620694). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson version ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22372 **[Test build #95846 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95846/testReport)** for PR 22372 at commit [`f68083a`](https://github.com/apache/spark/commit/f68083ab07df6fedf8a30c94e706a74e0c620694). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson version ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22372 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22372: [SPARK-25385][BUILD] Upgrade Hadoop 3.1 jackson version ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22372 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2955/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
