[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16990 I merged this to master, but the script gave an error from git. I had experienced some intermittent Github errors, but I also have a new environment. The commit looks correct but hasn't synced immediately to Github. Not sure what's happened, but will monitor it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16990 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73511/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16990 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16990 **[Test build #73511 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73511/testReport)** for PR 16990 at commit [`21956db`](https://github.com/apache/spark/commit/21956db9ac1908807bbe7761c815980309c35ac8). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16990 **[Test build #73511 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73511/testReport)** for PR 16990 at commit [`21956db`](https://github.com/apache/spark/commit/21956db9ac1908807bbe7761c815980309c35ac8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/16990 OK. I have reverted `set hive.mapreduce.job.reduces.speculative.execution=false` to `set hive.mapred.reduce.tasks.speculative.execution=false`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/16990 LGTM, though you'd have to go do the full coverage to verify that there's not a typo in any of the strings. This is why although Spark has adopted the more readable inline strings, I'm more of a fan of the "refer to the constant" for both spelling and the ability to locate use though the IDE That said in [HDFS-9301](https://issues.apache.org/jira/browse/HDFS-9301) and [HDFS-6418](https://issues.apache.org/jira/browse/HDFS-6418) I have expressed my concerns about HDFS constants, and invariably encountered resistance to fixing regressions. core and YARN are stable, and I'll happily revert anything there if people complain. Note also that deprecation warnings go to a special log, `"org.apache.hadoop.conf.Configuration.deprecation"`. Set that to WARN and you don't get told off any more. Finally, I still have no idea why HDFS-531 changed fs.default.name to a new mixed case string. It does, well, nothing. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/16990 @srowen @felixcheung The SQL query is related to the file name, see: https://github.com/apache/spark/blob/v2.1.0/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveComparisonTest.scala#L314 e.g; `set mapred.reduce.tasks=31`'s MD5 is `83c59d378571a6e487aa20217bd87817`, `set mapreduce.job.reduces=31`'s MD5 is `be2c0b32a02a1154bfdee1a52530f387`. So I change the following file names: ``` mv input12_hadoop20-0-db1cd54a4cb36de2087605f32e41824f input12_hadoop20-0-2b9ccaa793eae0e73bf76335d3d6880 mv join14_hadoop20-1-db1cd54a4cb36de2087605f32e41824f join14_hadoop20-1-2b9ccaa793eae0e73bf76335d3d6880 mv auto_join14_hadoop20-2-db1cd54a4cb36de2087605f32e41824f auto_join14_hadoop20-2-2b9ccaa793eae0e73bf76335d3d6880 mv groupby4_noskew-2-83c59d378571a6e487aa20217bd87817 groupby4_noskew-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby4_map-2-83c59d378571a6e487aa20217bd87817 groupby4_map-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby4_map_skew-2-83c59d378571a6e487aa20217bd87817 groupby4_map_skew-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby7_map-3-83c59d378571a6e487aa20217bd87817 groupby7_map-3-be2c0b32a02a1154bfdee1a52530f387 mv groupby2_limit-0-83c59d378571a6e487aa20217bd87817 groupby2_limit-0-be2c0b32a02a1154bfdee1a52530f387 mv groupby6_map_skew-2-83c59d378571a6e487aa20217bd87817 groupby6_map_skew-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby5_map_skew-2-83c59d378571a6e487aa20217bd87817 groupby5_map_skew-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby5_noskew-2-83c59d378571a6e487aa20217bd87817 groupby5_noskew-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby2_map-2-83c59d378571a6e487aa20217bd87817 groupby2_map-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby7_noskew-3-83c59d378571a6e487aa20217bd87817 groupby7_noskew-3-be2c0b32a02a1154bfdee1a52530f387 mv groupby1_map_skew-2-83c59d378571a6e487aa20217bd87817 groupby1_map_skew-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby8_map-2-83c59d378571a6e487aa20217bd87817 groupby8_map-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby6_noskew-2-83c59d378571a6e487aa20217bd87817 groupby6_noskew-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby7_map_skew-2-83c59d378571a6e487aa20217bd87817 groupby7_map_skew-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby7_noskew_multi_single_reducer-2-83c59d378571a6e487aa20217bd87817 groupby7_noskew_multi_single_reducer-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby_map_ppr-2-83c59d378571a6e487aa20217bd87817 groupby_map_ppr-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby8_map_skew-2-83c59d378571a6e487aa20217bd87817 groupby8_map_skew-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby6_map-2-83c59d378571a6e487aa20217bd87817 groupby6_map-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby1_noskew-2-83c59d378571a6e487aa20217bd87817 groupby1_noskew-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby1_limit-0-83c59d378571a6e487aa20217bd87817 groupby1_limit-0-be2c0b32a02a1154bfdee1a52530f387 mv groupby5_map-2-83c59d378571a6e487aa20217bd87817 groupby5_map-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby7_map_multi_single_reducer-2-83c59d378571a6e487aa20217bd87817 groupby7_map_multi_single_reducer-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby1_map-2-83c59d378571a6e487aa20217bd87817 groupby1_map-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby2_noskew-2-83c59d378571a6e487aa20217bd87817 groupby2_noskew-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby8_noskew-2-83c59d378571a6e487aa20217bd87817 groupby8_noskew-2-be2c0b32a02a1154bfdee1a52530f387 mv groupby2_map_skew-2-83c59d378571a6e487aa20217bd87817 groupby2_map_skew-2-be2c0b32a02a1154bfdee1a52530f387 mv
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16990 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16990 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73207/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16990 **[Test build #73207 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73207/testReport)** for PR 16990 at commit [`97734c5`](https://github.com/apache/spark/commit/97734c5af3df4e6525e8015459af16ab193dfc24). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16990 **[Test build #73207 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73207/testReport)** for PR 16990 at commit [`97734c5`](https://github.com/apache/spark/commit/97734c5af3df4e6525e8015459af16ab193dfc24). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16990 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73205/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16990 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16990 **[Test build #73205 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73205/testReport)** for PR 16990 at commit [`d8e5862`](https://github.com/apache/spark/commit/d8e58627e866f114fd9df6bdb953947b47aaea95). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/16990 I'm working on the tests fail. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16990 ``` [info] *** 34 TESTS FAILED *** [error] Failed: Total 2311, Failed 34, Errors 0, Passed 2277, Ignored 597 [error] Failed tests: [error] org.apache.spark.sql.hive.execution.HiveCompatibilitySuite [error] (hive/test:test) sbt.TestsFailedException: Tests unsuccessful ``` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16990 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16990 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73128/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16990 **[Test build #73128 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73128/testReport)** for PR 16990 at commit [`39b3778`](https://github.com/apache/spark/commit/39b37780ad827fb92c3fb9e6398a675ef189ebef). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16990 **[Test build #73128 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73128/testReport)** for PR 16990 at commit [`39b3778`](https://github.com/apache/spark/commit/39b37780ad827fb92c3fb9e6398a675ef189ebef). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16990 **[Test build #73127 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73127/testReport)** for PR 16990 at commit [`b13ca22`](https://github.com/apache/spark/commit/b13ca22953209a5866cbf93da8a95c9af677b846). * This patch **fails Python style tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16990 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73127/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16990 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #16990: [SPARK-19660][CORE][SQL] Replace the configuration prope...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16990 **[Test build #73127 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73127/testReport)** for PR 16990 at commit [`b13ca22`](https://github.com/apache/spark/commit/b13ca22953209a5866cbf93da8a95c9af677b846). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
