[GitHub] spark issue #19830: [SPARK-22520][SQL][followup] remove outer if for case wh...

2017-11-28 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19830 LGTM, thanks for the fix! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19827: [SPARK-22617][SQL] make splitExpressions extract ...

2017-11-28 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19827#discussion_r153452590 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -351,24 +351,25 @@ case class

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19222 **[Test build #84254 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84254/testReport)** for PR 19222 at commit

[GitHub] spark issue #19827: [SPARK-22617][SQL] make splitExpressions extract current...

2017-11-28 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19827 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19601: [SPARK-22383][SQL] Generate code to directly get value o...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19601 **[Test build #84253 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84253/testReport)** for PR 19601 at commit

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-11-28 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19222 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19823: [SPARK-22601][SQL] Data load is getting displayed succes...

2017-11-28 Thread sujith71955
Github user sujith71955 commented on the issue: https://github.com/apache/spark/pull/19823 @gatorsmile @HyukjinKwon @srowen Please review as i modified the code as per provided comments. thanks --- - To

[GitHub] spark pull request #19832: [SPARK-22628][CORE]Some situations, the assignm...

2017-11-28 Thread 10110346
GitHub user 10110346 opened a pull request: https://github.com/apache/spark/pull/19832 [SPARK-22628][CORE]Some situations, the assignment of executors on workers is not what we expected when `spark.deploy.spreadOut=true`. ## What changes were proposed in this pull request?

[GitHub] spark issue #19831: [SPARK-22489][SQL] Wrong Hive table statistics may trigg...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19831 **[Test build #84255 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84255/testReport)** for PR 19831 at commit

[GitHub] spark pull request #19831: [SPARK-22489][SQL] Wrong Hive table statistics ma...

2017-11-28 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/19831 [SPARK-22489][SQL] Wrong Hive table statistics may trigger OOM if enables join reorder in CBO ## What changes were proposed in this pull request? How to reproduce: ```basg

[GitHub] spark issue #19830: [SPARK-22520][SQL][followup] remove outer if for case wh...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19830 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-11-28 Thread zzl1787
Github user zzl1787 commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r153467044 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. - As a

[GitHub] spark issue #19831: [SPARK-22626][SQL] Wrong Hive table statistics may trigg...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19831 **[Test build #84255 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84255/testReport)** for PR 19831 at commit

[GitHub] spark issue #19831: [SPARK-22626][SQL] Wrong Hive table statistics may trigg...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19831 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19828: [SPARK-22614] Dataset API: repartitionByRange(...)

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19828 **[Test build #84258 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84258/testReport)** for PR 19828 at commit

[GitHub] spark issue #19830: [SPARK-22520][SQL][followup] remove outer if for case wh...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19830 **[Test build #84249 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84249/testReport)** for PR 19830 at commit

[GitHub] spark issue #19830: [SPARK-22520][SQL][followup] remove outer if for case wh...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19830 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84249/ Test PASSed. ---

[GitHub] spark issue #19832: [SPARK-22628][CORE]Some situations, the assignment of ...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19832 **[Test build #84257 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84257/testReport)** for PR 19832 at commit

[GitHub] spark issue #19828: [SPARK-22614] Dataset API: repartitionByRange(...)

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19828 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19827: [SPARK-22617][SQL] make splitExpressions extract current...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19827 **[Test build #84251 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84251/testReport)** for PR 19827 at commit

[GitHub] spark issue #11994: [SPARK-14151] Expose metrics Source and Sink interface

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11994 **[Test build #84250 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84250/testReport)** for PR 11994 at commit

[GitHub] spark issue #11994: [SPARK-14151] Expose metrics Source and Sink interface

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11994 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84250/ Test PASSed. ---

[GitHub] spark issue #11994: [SPARK-14151] Expose metrics Source and Sink interface

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11994 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19390: [SPARK-18935][MESOS] Fix dynamic reservations on mesos

2017-11-28 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/19390 @vanzin gentle ping. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19714: [SPARK-22489][SQL] Shouldn't change broadcast join build...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19714 **[Test build #84248 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84248/testReport)** for PR 19714 at commit

[GitHub] spark issue #19714: [SPARK-22489][SQL] Shouldn't change broadcast join build...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19714 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19714: [SPARK-22489][SQL] Shouldn't change broadcast join build...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19714 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84248/ Test PASSed. ---

[GitHub] spark issue #19743: [SPARK-22515] [SQL] Estimation relation size based on nu...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19743 **[Test build #84256 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84256/testReport)** for PR 19743 at commit

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-11-28 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19468 I think the other cases, mesos, yarn/docker are quite different as they are one of the possible environment/setup, whereas in this case there is no other alternative - docker image must be

[GitHub] spark issue #19827: [SPARK-22617][SQL] make splitExpressions extract current...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19827 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84251/ Test PASSed. ---

[GitHub] spark issue #19827: [SPARK-22617][SQL] make splitExpressions extract current...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19827 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19827: [SPARK-22617][SQL] make splitExpressions extract ...

2017-11-28 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19827#discussion_r153453043 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala --- @@ -258,31 +264,28 @@ object

[GitHub] spark issue #19828: [SPARK-22614] Dataset API: repartitionByRange(...)

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19828 **[Test build #84252 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84252/testReport)** for PR 19828 at commit

[GitHub] spark issue #19828: [SPARK-22614] Dataset API: repartitionByRange(...)

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19828 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84252/ Test PASSed. ---

[GitHub] spark issue #19827: [SPARK-22617][SQL] make splitExpressions extract current...

2017-11-28 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19827 LGTM, only very minor comments. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19831: [SPARK-22626][SQL] Wrong Hive table statistics may trigg...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19831 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84255/ Test FAILed. ---

[GitHub] spark issue #19831: [SPARK-22626][SQL] Wrong Hive table statistics may trigg...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19831 **[Test build #84259 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84259/testReport)** for PR 19831 at commit

[GitHub] spark pull request #19717: [SPARK-18278] [Submission] Spark on Kubernetes - ...

2017-11-28 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r153552488 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -0,0 +1,160 @@ +/* + * Licensed to

[GitHub] spark issue #19717: [SPARK-18278] [Submission] Spark on Kubernetes - basic s...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19717 **[Test build #84268 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84268/testReport)** for PR 19717 at commit

[GitHub] spark issue #19810: [SPARK-22599][SQL] In-Memory Table Pruning without Extra...

2017-11-28 Thread CodingCat
Github user CodingCat commented on the issue: https://github.com/apache/spark/pull/19810 reading less data is a observation from the input metrics in Spark UI which includes both of local/remote read in BlockManagers, and also the overhead in BlockManager layer itself (especially

[GitHub] spark issue #19835: [SPARK-21866][ML][PYTHON] Few cleanups and fix test case...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19835 **[Test build #84267 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84267/testReport)** for PR 19835 at commit

[GitHub] spark issue #19835: [SPARK-21866][ML][PYTHON] Few cleanups and fix test case...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19835 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19747: [Spark-22431][SQL] Ensure that the datatype in the schem...

2017-11-28 Thread skambha
Github user skambha commented on the issue: https://github.com/apache/spark/pull/19747 Thanks @gatorsmile. I have addressed your comments in the latest commit. Please take a look. Thanks. --- - To

[GitHub] spark issue #19835: [SPARK-21866][ML][PYTHON][FOLLOWUP] Few cleanups and fix...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19835 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84270/ Test PASSed. ---

[GitHub] spark issue #19835: [SPARK-21866][ML][PYTHON][FOLLOWUP] Few cleanups and fix...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19835 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19834: [SPARK-22585][Core] Path in addJar is not url encoded

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19834 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19834: [SPARK-22585][Core] Path in addJar is not url encoded

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19834 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84262/ Test PASSed. ---

[GitHub] spark issue #19743: [SPARK-22515] [SQL] Estimation relation size based on nu...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19743 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19717: [SPARK-18278] [Submission] Spark on Kubernetes - ...

2017-11-28 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r153548705 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/Client.scala --- @@ -0,0 +1,236 @@ +/* + *

[GitHub] spark pull request #19835: [SPARK-21866][ML][PYTHON] Few cleanups and fix te...

2017-11-28 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19835#discussion_r153552951 --- Diff: python/pyspark/ml/image.py --- @@ -146,7 +163,11 @@ def toImage(self, array, origin=""): mode = ocvTypes["CV_8UC4"]

[GitHub] spark issue #19835: [SPARK-21866][ML][PYTHON] Few cleanups and fix test case...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19835 **[Test build #84267 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84267/testReport)** for PR 19835 at commit

[GitHub] spark issue #19835: [SPARK-21866][ML][PYTHON] Few cleanups and fix test case...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19835 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84267/ Test PASSed. ---

[GitHub] spark pull request #19835: [SPARK-21866][ML][PYTHON] Few cleanups and fix te...

2017-11-28 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19835#discussion_r153562522 --- Diff: python/pyspark/ml/image.py --- @@ -146,7 +163,12 @@ def toImage(self, array, origin=""): mode = ocvTypes["CV_8UC4"]

[GitHub] spark issue #19568: SPARK-22345: Fix sort-merge joins with conditions and co...

2017-11-28 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/19568 @gatorsmile, I think it would be better to fix codegen than to prevent it from happening with an assertion. If `CodegenFallback` can produce fallback code, why not allow it to when necessary? ---

[GitHub] spark pull request #19568: SPARK-22345: Fix sort-merge joins with conditions...

2017-11-28 Thread rdblue
Github user rdblue closed the pull request at: https://github.com/apache/spark/pull/19568 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19568: SPARK-22345: Fix sort-merge joins with conditions...

2017-11-28 Thread rdblue
GitHub user rdblue reopened a pull request: https://github.com/apache/spark/pull/19568 SPARK-22345: Fix sort-merge joins with conditions and codegen. ## What changes were proposed in this pull request? This adds a joined row to sort-merge join codegen. That joined row is

[GitHub] spark issue #19813: [WIP][SPARK-22600][SQL] Fix 64kb limit for deeply nested...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19813 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84260/ Test FAILed. ---

[GitHub] spark issue #19813: [WIP][SPARK-22600][SQL] Fix 64kb limit for deeply nested...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19813 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19792: [SPARK-22566][PYTHON] Better error message for `_merge_t...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19792 **[Test build #84272 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84272/testReport)** for PR 19792 at commit

[GitHub] spark issue #19835: [SPARK-21866][ML][PYTHON][FOLLOWUP] Few cleanups and fix...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19835 **[Test build #84270 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84270/testReport)** for PR 19835 at commit

[GitHub] spark issue #19834: [SPARK-22585][Core] Path in addJar is not url encoded

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19834 **[Test build #84262 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84262/testReport)** for PR 19834 at commit

[GitHub] spark pull request #19714: [SPARK-22489][SQL] Shouldn't change broadcast joi...

2017-11-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19714#discussion_r153583900 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -153,6 +151,27 @@ abstract class SparkStrategies extends

[GitHub] spark pull request #19717: [SPARK-18278] [Submission] Spark on Kubernetes - ...

2017-11-28 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r153548760 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark-base/Dockerfile --- @@ -0,0 +1,43 @@ +# +# Licensed to the Apache

[GitHub] spark pull request #19717: [SPARK-18278] [Submission] Spark on Kubernetes - ...

2017-11-28 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r153548770 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark-base/entrypoint.sh --- @@ -0,0 +1,37 @@ +#!/bin/bash +# +#

[GitHub] spark pull request #19717: [SPARK-18278] [Submission] Spark on Kubernetes - ...

2017-11-28 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r153548579 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -590,6 +600,11 @@ private[deploy] class

[GitHub] spark pull request #19717: [SPARK-18278] [Submission] Spark on Kubernetes - ...

2017-11-28 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r153548599 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -0,0 +1,160 @@ +/* + * Licensed to

[GitHub] spark pull request #19717: [SPARK-18278] [Submission] Spark on Kubernetes - ...

2017-11-28 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r153548666 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/Client.scala --- @@ -0,0 +1,236 @@ +/* + *

[GitHub] spark pull request #19717: [SPARK-18278] [Submission] Spark on Kubernetes - ...

2017-11-28 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r153548556 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -702,6 +715,19 @@ object SparkSubmit extends CommandLineUtils with

[GitHub] spark pull request #19717: [SPARK-18278] [Submission] Spark on Kubernetes - ...

2017-11-28 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r153548730 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/DriverServiceBootstrapStep.scala --- @@ -0,0 +1,103

[GitHub] spark pull request #19835: [SPARK-21866][ML][PYTHON] Few cleanups and fix te...

2017-11-28 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/19835 [SPARK-21866][ML][PYTHON] Few cleanups and fix test case for Python 3.6.0 / NumPy 1.13.3 ## What changes were proposed in this pull request? Image test seems failed in Python 3.6.0 /

[GitHub] spark issue #19833: [SPARK-22605][SQL] SQL write job should also set Spark t...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19833 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84261/ Test FAILed. ---

[GitHub] spark issue #19823: [SPARK-22601][SQL] Data load is getting displayed succes...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19823 **[Test build #3996 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3996/testReport)** for PR 19823 at commit

[GitHub] spark issue #19833: [SPARK-22605][SQL] SQL write job should also set Spark t...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19833 **[Test build #84261 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84261/testReport)** for PR 19833 at commit

[GitHub] spark issue #19833: [SPARK-22605][SQL] SQL write job should also set Spark t...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19833 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19129: [SPARK-13656][SQL] Delete spark.sql.parquet.cache...

2017-11-28 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19129#discussion_r153565297 --- Diff: docs/sql-programming-guide.md --- @@ -1587,6 +1580,10 @@ options. Note that this is different from the Hive behavior. -

[GitHub] spark issue #19835: [SPARK-21866][ML][PYTHON] Few cleanups and fix test case...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19835 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84269/ Test PASSed. ---

[GitHub] spark issue #19835: [SPARK-21866][ML][PYTHON] Few cleanups and fix test case...

2017-11-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19835 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19835: [SPARK-21866][ML][PYTHON] Few cleanups and fix test case...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19835 **[Test build #84270 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84270/testReport)** for PR 19835 at commit

[GitHub] spark pull request #19717: [SPARK-18278] [Submission] Spark on Kubernetes - ...

2017-11-28 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r153581863 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/BaseDriverConfigurationStep.scala --- @@ -0,0 +1,162

[GitHub] spark pull request #19714: [SPARK-22489][SQL] Shouldn't change broadcast joi...

2017-11-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19714#discussion_r153583413 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -153,6 +151,27 @@ abstract class SparkStrategies extends

[GitHub] spark pull request #19714: [SPARK-22489][SQL] Shouldn't change broadcast joi...

2017-11-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19714#discussion_r153583308 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -153,6 +151,27 @@ abstract class SparkStrategies extends

[GitHub] spark issue #19823: [SPARK-22601][SQL] Data load is getting displayed succes...

2017-11-28 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/19823 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19743: [SPARK-22515] [SQL] Estimation relation size based on nu...

2017-11-28 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/19743 ping @cloud-fan Could you review this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19743: [SPARK-22515] [SQL] Estimation relation size base...

2017-11-28 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19743#discussion_r153514962 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala --- @@ -41,7 +41,35 @@ import org.apache.spark.sql.types._

[GitHub] spark issue #19833: [SPARK-22605][SQL] SQL write job should also set Spark t...

2017-11-28 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19833 cc @rxin @hvanhovell @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19826: [SPARK-22428][DOC] Add spark application garbage collect...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19826 **[Test build #3995 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3995/testReport)** for PR 19826 at commit

[GitHub] spark pull request #19834: [SPARK-22585][Core] Path in addJar is not url enc...

2017-11-28 Thread james64
Github user james64 commented on a diff in the pull request: https://github.com/apache/spark/pull/19834#discussion_r153524308 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1846,7 +1846,16 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark issue #19834: [SPARK-22585][Core] Path in addJar is not url encoded

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19834 **[Test build #84266 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84266/testReport)** for PR 19834 at commit

[GitHub] spark issue #19835: [SPARK-21866][ML][PYTHON] Few cleanups and fix test case...

2017-11-28 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19835 cc @jkbradley and @imatiach-msft, could you maybe take a look and see if it makes sense? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19823: [SPARK-22601][SQL] Data load is getting displayed succes...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19823 **[Test build #3996 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3996/testReport)** for PR 19823 at commit

[GitHub] spark issue #19813: [WIP][SPARK-22600][SQL] Fix 64kb limit for deeply nested...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19813 **[Test build #84260 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84260/testReport)** for PR 19813 at commit

[GitHub] spark pull request #19792: [SPARK-22566][PYTHON] Better error message for `_...

2017-11-28 Thread gberger
Github user gberger commented on a diff in the pull request: https://github.com/apache/spark/pull/19792#discussion_r153579071 --- Diff: python/pyspark/sql/types.py --- @@ -1108,19 +1109,23 @@ def _has_nulltype(dt): return isinstance(dt, NullType) -def

[GitHub] spark issue #19714: [SPARK-22489][SQL] Shouldn't change broadcast join build...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19714 **[Test build #84264 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84264/testReport)** for PR 19714 at commit

[GitHub] spark pull request #19714: [SPARK-22489][SQL] Shouldn't change broadcast joi...

2017-11-28 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19714#discussion_r153584009 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -153,6 +151,27 @@ abstract class SparkStrategies extends

[GitHub] spark issue #19743: [SPARK-22515] [SQL] Estimation relation size based on nu...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19743 **[Test build #84263 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84263/testReport)** for PR 19743 at commit

[GitHub] spark pull request #19717: [SPARK-18278] [Submission] Spark on Kubernetes - ...

2017-11-28 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19717#discussion_r153548743 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/driver/Dockerfile --- @@ -0,0 +1,31 @@ +# +# Licensed to the Apache Software

[GitHub] spark issue #19835: [SPARK-21866][ML][PYTHON] Few cleanups and fix test case...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19835 **[Test build #84269 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84269/testReport)** for PR 19835 at commit

[GitHub] spark issue #19835: [SPARK-21866][ML][PYTHON] Few cleanups and fix test case...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19835 **[Test build #84269 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84269/testReport)** for PR 19835 at commit

[GitHub] spark issue #19747: [Spark-22431][SQL] Ensure that the datatype in the schem...

2017-11-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19747 **[Test build #84271 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84271/testReport)** for PR 19747 at commit

[GitHub] spark pull request #19792: [SPARK-22566][PYTHON] Better error message for `_...

2017-11-28 Thread gberger
Github user gberger commented on a diff in the pull request: https://github.com/apache/spark/pull/19792#discussion_r153578612 --- Diff: python/pyspark/sql/tests.py --- @@ -1722,6 +1723,83 @@ def test_infer_long_type(self): self.assertEqual(_infer_type(2**61),

  1   2   3   4   >