[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99588/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21588 **[Test build #99588 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99588/testReport)** for PR 21588 at commit [`a011e50`](https://github.com/apache/spark/commit/a011e50e57537589b23099b4c7b6e2e893c86f9e). * This patch **fails to generate documentation**. * This patch **does not merge cleanly**. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21588 **[Test build #99588 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99588/testReport)** for PR 21588 at commit [`a011e50`](https://github.com/apache/spark/commit/a011e50e57537589b23099b4c7b6e2e893c86f9e). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21588 If supporting Hadoop 3 means updating to Hive 2, more or less (not dropping old Hive metastore support), then yes that seems pretty important. I didn't hear objections to updating Hive, right? and it gets us off the fork? were the main issue with that PR that it altered ORC support @dongjoon-hyun ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 To all, so how about we start the fix @wangyum tried before? If we are generally agreed upon the direction itself, upgrading Hive to 2.3 (or 3), I would like to encourage him to continue #20659. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 The test failure itself doesn't look caused by this change. The tests will fail anyway with a different error message. If the goal is really just to check if the tests pass or not, you should use `com.github.hyukjinkwon` artifact instead of `org.spark-project.hive` in Spark which I released under my personal domain to test this, as I did above. We should fix the hive issue we're discussing right now first so that we can safely merge this PR and verify that Hadoop 3 profile works fine. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user LiehuoChen commented on the issue: https://github.com/apache/spark/pull/21588 Hi HyukjinKwon, Thanks for all the works to try to make the Jenkin test pass. I patched this PR to spark 2.4, and anything works fine but failed in org.apache.spark.deploy.yarn.YarnClusterSuite for following four unit tests: 1). run Spark in yarn-cluster mode 2). run Spark in yarn-cluster mode with different configurations, ensuring redaction 3). run Spark in yarn-client mode 4). run Spark in yarn-client mode with different configurations, ensuring redaction 1), 2), failed everytime with really few useful error Msg, like: `FAILED did not equal FINISHED Exception in thread "main" org.apache.spark.SparkException: Application application_1542090777201_0002 finished with failed status [info] at org.apache.spark.deploy.yarn.Client.run(Client.scala:1149) .. [info] at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) (BaseYarnClusterSuite.scala:201)` 2).4). succeed most of time, but sometimes failed on ` Exception in thread "main" java.io.IOException: Server returned HTTP response code: 500 for URL: http://user-c02wq03ghtdg.corp.uber.com:61313/node/containerlogs/container_1541809642345_0002_01_02/lhc/stdout?start=-4096` and `Fail to invoke HBaseConfiguration [info] java.lang.ClassNotFoundException: org.apache.hadoop.hbase.HBaseConfiguration` Any you ever see the similars errors before? do you did any other fixes besides this PR to make the all test pass. Thanks for your time. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 Yes, that was what I was thinking at worst case. For clarification, @wangyum made a try and all tests were passed at least - https://github.com/apache/spark/pull/20659. Given this try, I think it looks pretty much feasible to upgrade without breaking compatibility. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21588 Can you clarify what you mean by drop builtin metastore support? Are you just saying users must always provide jars to use it or something more? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 @dongjoon-hyun and @wangyum, please fix my comment if I am wrong at any point - I believe you guys took a look for this part more then I did. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 > Does this upgrade Hive for execution or also for metastore? Spark supports virtually all Hive metastore versions out there, and a lot of deployments do run different versions of Spark against the same old Hive metastore, and it'd be bad to break connectivity to old Hive metastores. > The execution part is a different story and we can upgrade them easily. The upgrade basically targets to upgrade Hive for execution (let me know if I am mistaken). For metastore compatibility, I believe we are able to provide metastore jars and support other Hive versions via explicitly configuring the JARs via isolated classloader. I believe we have basic tests for different Hive versions. I would cautiously like to raise an option - drop the builtin metastore support at 3.0 by default if the upgrade makes to keep builtin metastore support hard enough. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user tooptoop4 commented on the issue: https://github.com/apache/spark/pull/21588 ship it! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 > Hive 2.3 works with Hadoop 2.x (Hive 3.x works with Hadoop 3.x). This is essentially what we need for Hadoop 3 support [release-2.3.2|https://github.com/apache/hive/blob/rel/release-2.3.2/shims/common/src/main/java/org/apache/hadoop/hive/shims/ShimLoader.java#L152-L154]. See https://issues.apache.org/jira/browse/HIVE-16081. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21588 Does this upgrade Hive for execution or also for metastore? Spark supports virtually all Hive metastore versions out there, and a lot of deployments do run different versions of Spark against the same old Hive metastore, and it'd be bad to break connectivity to old Hive metastores. The execution part is a different story and we can upgrade them easily. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user functicons commented on the issue: https://github.com/apache/spark/pull/21588 Do we really want to switch to Hive 2.3? From this page https://hive.apache.org/downloads.html, Hive 2.3 works with Hadoop 2.x (Hive 3.x works with Hadoop 3.x). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21588 Sounds like we should try this then --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21588 So, let's say we decide to only support Hive 2.3.x+, as a precursor to this. We could already eliminate a lot of the Hive tests, right? that might be useful in its own right as they take time and are a little flaky. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 Yup, it supports Hadoop 3, and other fixes what @wangyum mentioned. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21588 does Apache Hive 2.3.2 have all the fixes we need? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 @rxin and @gatorsmile, WDYT? I already had to argue about Hadoop 3 support here and there (for instance see [SPARK-18112|https://issues.apache.org/jira/browse/SPARK-18112] and [SPARK-18673|https://issues.apache.org/jira/browse/SPARK-18673]), and explain what's going on. Looks ideally we should go ahead 2. (https://github.com/apache/spark/pull/21588#issuecomment-429272279) if I am not mistaken. If there are some more concerns we should address before going ahead, definitely I am willing to help investigating as well. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21588 Thanks @HyukjinKwon Upgrade Hive to 2.3.2 can fix [SPARK-12014](https://issues.apache.org/jira/browse/SPARK-12014), [SPARK-18673](https://issues.apache.org/jira/browse/SPARK-18673), [SPARK-24766](https://issues.apache.org/jira/browse/SPARK-24766) and [SPARK-25193](https://issues.apache.org/jira/browse/SPARK-25193). Also, can improve the performance of the [SPARK-18107](https://issues.apache.org/jira/browse/SPARK-18107). Seems it doesnât break backward compatibility. I have verified it in our production environment (Hive 1.2.1). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 ping @wangyum, if you're willing to make a progress about this, please provide some input here and/or in the JIRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 Yes it solves anything. We could consider to upgrade to Hive 3 but I am unsure on this since any try (as far as I know) wasn't made yet. But for Hive 2.3.2, @wangyum made a try here (https://github.com/apache/spark/pull/20659) where at least the tests were passed - looks feasible. Some people worry about the difficulties. So for clarification, @wangyum, do you mind if I ask to list up the potential advantage and disadvantages (for instance breaking backward compatibility), and some existing difficulties? I think this is only concerns left if I am not mistaken. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21588 I know this is probably just reviving an old thread elsewhere, but, we don't know how to update our 1.2.1 Hive fork anyway, it seems? if so, and the fork is undesirable, seems like time to drop it. If it's hard to get on mainstream Hive 1.x, then, how is 2.x? Certainly its reasonable to drop 1.x support in Spark 3.0. Does that solve anything? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 Since Hadoop 3 support is also being discussed for Spark 3 (http://apache-spark-developers-list.1001551.n3.nabble.com/time-for-Apache-Spark-3-0-td23755.html). We should at least: 1. Land a fix for our Hive fork (1.2.x with some fixes) 2. Spark replace its Hive fork to Apache Hive, for instance Apache Hive 2.3.2 3. Apache Hive releases 1.2.x So far, 1 or 2 is a desirable option. 2 could ideally only happen at major version bump-up - Spark 3 looks a good target to go, logically. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 This is currently blocked by [SPARK-20202](https://issues.apache.org/jira/browse/SPARK-20202) per https://github.com/apache/spark/pull/21588#issuecomment-399292229. Please provide some input there. Looks I am blocked because there are no input there. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user tooptoop4 commented on the issue: https://github.com/apache/spark/pull/21588 @jerryshao @vanzin can this be merged to master? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96685/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21588 **[Test build #96685 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96685/testReport)** for PR 21588 at commit [`a011e50`](https://github.com/apache/spark/commit/a011e50e57537589b23099b4c7b6e2e893c86f9e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user elgalu commented on the issue: https://github.com/apache/spark/pull/21588 ð works! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3533/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21588 **[Test build #96685 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96685/testReport)** for PR 21588 at commit [`a011e50`](https://github.com/apache/spark/commit/a011e50e57537589b23099b4c7b6e2e893c86f9e). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user tooptoop4 commented on the issue: https://github.com/apache/spark/pull/21588 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96664/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21588 **[Test build #96664 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96664/testReport)** for PR 21588 at commit [`a011e50`](https://github.com/apache/spark/commit/a011e50e57537589b23099b4c7b6e2e893c86f9e). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3515/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21588 **[Test build #96664 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96664/testReport)** for PR 21588 at commit [`a011e50`](https://github.com/apache/spark/commit/a011e50e57537589b23099b4c7b6e2e893c86f9e). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92573/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21588 **[Test build #92573 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92573/testReport)** for PR 21588 at commit [`b69e29a`](https://github.com/apache/spark/commit/b69e29a39a0c3244f2decc4cb85b14353d4aaa11). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21588 **[Test build #92573 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92573/testReport)** for PR 21588 at commit [`b69e29a`](https://github.com/apache/spark/commit/b69e29a39a0c3244f2decc4cb85b14353d4aaa11). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/644/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 Yea, that's all true. I admit what you and @jerryshao did makes sense in a way. If we failed to replace the Hive fork to 2.3.x and keep the current fork, I got it that's the last resort that might make sense. It should have never been done like this with a fork. I personally see it's what we Spark made the mistake about and what I \*personally\* thought was we should do our own cleanup things, and Spark should mainly be responsible for it. I tried hard to understand the contexts and find the most reasonable way (to me) to get through this smoothly and reasonably. I left a comment in [SPARK-20202|https://issues.apache.org/jira/browse/SPARK-20202]. Should be the most appropriate place to talk about this. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/21588 There's a technical issue: trivial change to the case statement and a ASF process one: the only ASF project which can release hive artifacts is the hive team; it's that way due to ASF release indemnity stuff (only ASF official releases come with that, and only a project may release its own artifacts). There's not AFAIK any reason the Hive team can't/won't do this, its just a matter of pushing them hard enough to get signoff. @jerryshao has the bigger patch though it's not been getting the attention it deserves. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 > is it basically true that Hadoop 3 will work with only minor patches to the Hive fork in Spark? Up to my knowledge so far, yes, it basically works. At least, the regression tests we wrote so far work for what we currently cover. > Is the blocker that we can't get a change into the Hive fork? Up to my knowledge so far, yes, there is one line fix (HIVE-16081) that the fork needs. > who owns it? As far as I know, @JoshRosen owns it. Please correct me if I am wrong. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21588 Also weighing in here to ask: is it basically true that Hadoop 3 will work with only minor patches to the Hive fork in Spark? then that seems worth the hacking. Is the blocker that we can't get a change into the Hive fork? who owns it? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 Yup, will fix the hive fork thing and be back. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21588 @HyukjinKwon , I'm in favor of @vanzin 's comment, we should fix things first and then back to this one. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 Will try to fix it then. We can just enable it back. If we want to support those Hive versions in Hadoop 3, we could simply enable them back with some fixes at that time. Adding the support sounds an incremental improvement. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21588 > The tests were passed in this PR builder Against your private build of the Hive stuff. Again, fix that and this will become a lot easier to discuss. I'm also against disabling these tests without a proper discussion of what that means, and I've said multiple times. If we want to support those Hive versions in Hadoop 3, then this is the wrong change. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 The tests were passed in this PR builder. The only hack I used is that I landed a one liner fix to an artifact to use it in this PR, which is already in Hive, and is proposed in Hive's fork which is blocked by non-techinical reason. I am working on this to get through. Okay, if you think it should be blocked, let me get through this first. I am not dropping it. Isn't it what we already cover? I believe this is the most minimised and conservative fix to make Hadoop 3 working within Spark since we already added it. FWIW, we didn't document Hadoop 3 profile yet, so my impression is that it's in progress yet. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21588 I already explained my view of why I don't think this should get in, in its current form. Passing tests in someone's private environment, for me, is not a worthy goal. You say the fix is needed, but I'm not even sure this is the right fix. You're dropping support for a bunch of Hive versions, effectively. Is that what we want? If it is, you need to properly document that, and fix places where you need a proper error message so users are not confused. If it's not, you need to find a solution to that problem. And for that it would be easier if you could actually test your change here. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 I at least checked if this passed with that fix to fork manually. It fixes everything else that can be fixed in Spark. I wonder why this should be blocked to be honest yet. It can't be ran via Jenkins, which I accept that thiis change should be blocked but this fix is needed anyway and can be unblocked. If something is needed, I just review and merge. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21588 Let me phrase the question a different way: your title says "Make jenkins tests passed" [sic]. If you check this in, and we enable a jenkins job for hadoop 3, will it pass? I'm 100% sure the answer is no. So fix the Hive fork, then update this PR, and let's have it actually run through jenkins. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 > The main thing is that this change is changing test coverage based on the Hadoop version > The Hive 2.1 suite you're disabling is also pretty important to keep working, since it tests behavior that changed from Spark's built in version of Hive, and most probably is similar in newer versions. > We should be looking at what it means to support Hadoop 3, and answer that before we go hacking and disabling things just to get tests to pass. I believe that's what we currently cover with Hadoop 3 profile ... maybe we should fix it back when other decisions are made or newer changes are landed. In my point of view, we don't hack things here. It just explicitly makes the tests passed as we cover with Hadoop 3 profile. The point of this PR and changes are to make Spark to the working state ... > The error message you saw seems familiar to you but I'm pretty sure it would be very cryptic to someone who's not familiar with this problem. (Why is Hive complaining about a Hadoop version if I'm running Spark?) Yea, I agree with fixing the error message, sure. I was thinking of doing this separately since this PR targets to make the tests passed and make Spark into working state with Hadoop 3 at least (of course except that the external (one liner) fix into Hive fork). For me, I don't see the reasons to block fix everything else for Hadoop 3 within Spark side by one liner fix within the Hive fork. We should really make a progress in parallel if we can ... --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21588 The main thing is that this change is changing test coverage based on the Hadoop version. So that means that we're effectively changing supported versions of Hive here, and we should do all the necessary changes to let people know of that. That includes deciding whether disabling those tests is the right thing, or whether we should make them work. The error message you saw seems familiar to you but I'm pretty sure it would be very cryptic to someone who's not familiar with this problem. (Why is Hive complaining about a Hadoop version if I'm running Spark?) The Hive 2.1 suite you're disabling is also pretty important to keep working, since it tests behavior that changed from Spark's built in version of Hive, and most probably is similar in newer versions. We should be looking at what it means to support Hadoop 3, and answer that before we go hacking and disabling things just to get tests to pass. I also really don't see the point of this before we fix the Hive fork... --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21588
I saw:
```
java.lang.IllegalArgumentException: Unrecognized Hadoop major version
number: 3.1.0
```
Other error messages are fixed by the current change. Yup, we could make a
decision and change into Spark side. However, I would like to ask to do this
separately ...
I missed an answer to this question:
> Do things work if you add hadoop-3.1 to sbt_maven_hadoop_profiles in
run-tests.py and change the PR title accordingly? Don't remember what's the
right incantation to trigger running with a different profile
("[test-hadoop-x.y]" maybe?).
BTW, I plan to fix everything else about doc and Jenkins together when we
are ready (which should be few line changes all together I think). I will try
and fix these too when we are ready.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21588
I saw:
```
Otherwise, we are not even able to start Spark shell. Currently, Hadoop 3
profile in Apache Spark doesn't work at all. It will face an error message such
as:
java.lang.IllegalArgumentException: Unrecognized Hadoop major version
number: 3.1.0
```
Other error messages are fixed by the current change. Yup, we could make a
decision and change into Spark side. However, I would like to ask to do this
separately ...
I missed an answer to this question:
> Do things work if you add hadoop-3.1 to sbt_maven_hadoop_profiles in
run-tests.py and change the PR title accordingly? Don't remember what's the
right incantation to trigger running with a different profile
("[test-hadoop-x.y]" maybe?).
BTW, I plan to fix everything else about doc and Jenkins together when we
are ready (which should be few line changes all together I think). I will try
and fix these too when we are ready.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21588 > but the error message was pretty readable to me What is the error message you see? I didn't see any changes in the Spark code that handles that (`IsolatedClientLoader.hiveVersion`), so whatever error would probably come from Hive. We should have an error that says "Spark does not support connecting to Hive blah when using Hadoop 3", if we really cannot support those versions. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 Yup, it will still fail but it fixes everything else to make it working with Hadoop 3 within Spark. I think the current change is minimised as the current status as is and I meant to target make the tests going through without other changes. > If, when build with Hadoop 3, Spark will not support older versions of Hive, that needs to be reflected in the code, not just in the tests. Yup, but the error message was pretty readable to me, at least. > And I'm also referring to the "org.spark-project" fork of Hive. That hasn't been updated, right? Which means that if you run this PR here, it will still fail? It will fail but it fixes everything else that we need for Spark & Hadoop 3 within Spark. I thought it should be tested so I republished it under my personal domain. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21588 I'm talking about the `VersionsSuite` stuff. I think it needs to be a more conscious decision about what happens. If, when build with Hadoop 3, Spark will not support older versions of Hive, that needs to be reflected in the code, not just in the tests. If, on the other hand, we want to support those, there may be ways. e.g. by forcing `sharesHadoopClasses` to false when using those versions with Hadoop 3. But I think just disabling the tests is the wrong approach. And I'm also referring to the "org.spark-project" fork of Hive. That hasn't been updated, right? Which means that if you run this PR here, it will still fail? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 @vanzin, which tests do you mean? Two types of tests are skipped. One is by external Hive's limit which we can't control and the other one (two tests) looks by a JDK bug which I think we usually leave just skipped. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/21588 If jenkins is happy, this is good. * Be interesting to see what happens in a build with the hadoop-cloud-storage module, if it adds new dependencies * regarding commons-config, know that todd lipcon has just filed [HADOOP-15549](https://issues.apache.org/jira/browse/HADOOP-15549) over a perf regression. Not sure what will happen there: rollback vs try to upgrade --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92134/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21588 **[Test build #92134 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92134/testReport)** for PR 21588 at commit [`3579017`](https://github.com/apache/spark/commit/3579017da9bf77ab3874640be600b02c1787ded6). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92135/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21588 **[Test build #92135 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92135/testReport)** for PR 21588 at commit [`d13e71a`](https://github.com/apache/spark/commit/d13e71acb1d7274ca303608942817873351de91f). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21588 **[Test build #92135 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92135/testReport)** for PR 21588 at commit [`d13e71a`](https://github.com/apache/spark/commit/d13e71acb1d7274ca303608942817873351de91f). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/355/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21588 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/4251/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 +@steveloughran --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 @srowen, @vanzin, @jerryshao and @gatorsmile, I believe this is ready for a look. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
