[GitHub] spark pull request #20362: [Spark-22886][ML][TESTS] ML test for structured s...

2018-02-22 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/20362#discussion_r170072392 --- Diff: mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala --- @@ -586,6 +586,68 @@ class ALSSuite allModelParamSetti

[GitHub] spark pull request #20362: [Spark-22886][ML][TESTS] ML test for structured s...

2018-02-22 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/20362#discussion_r170072578 --- Diff: mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala --- @@ -413,34 +411,36 @@ class ALSSuite .setSeed(0)

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20653 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1012/ Tes

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20653 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20653 **[Test build #87622 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87622/testReport)** for PR 20653 at commit [`d6ac338`](https://github.com/apache/spark/commit/d6

[GitHub] spark pull request #20362: [Spark-22886][ML][TESTS] ML test for structured s...

2018-02-22 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/20362#discussion_r170074167 --- Diff: mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala --- @@ -586,6 +586,68 @@ class ALSSuite allModelParamSetti

[GitHub] spark issue #20652: [SPARK-23476][CORE] Generate secret in local mode when a...

2018-02-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20652 Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h..

[GitHub] spark pull request #20652: [SPARK-23476][CORE] Generate secret in local mode...

2018-02-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20652 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20640: [SPARK-19755][Mesos] Blacklist is always active for Meso...

2018-02-22 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/20640 @squito @IgorBerman a few points: a) This should be configurable. SPARK-16630 is not closed, it seems also kubernetes needs to support checking for blacklisted nodes when launching tasks. From wh

[GitHub] spark pull request #20661: epoch ID in writer

2018-02-22 Thread jose-torres
GitHub user jose-torres opened a pull request: https://github.com/apache/spark/pull/20661 epoch ID in writer ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how

[GitHub] spark pull request #20661: epoch ID in writer

2018-02-22 Thread jose-torres
Github user jose-torres closed the pull request at: https://github.com/apache/spark/pull/20661 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.or

[GitHub] spark pull request #20624: [SPARK-23445] ColumnStat refactoring

2018-02-22 Thread juliuszsompolski
Github user juliuszsompolski commented on a diff in the pull request: https://github.com/apache/spark/pull/20624#discussion_r170086812 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -387,6 +390,101 @@ case class CatalogStatistics

[GitHub] spark issue #20382: [SPARK-23097][SQL][SS] Migrate text socket source to V2

2018-02-22 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20382 @jerryshao ping on this. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170087573 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -107,17 +106,24 @@ case class DataSourceV2

[GitHub] spark pull request #20622: [SPARK-23491][SS] Remove explicit job cancellatio...

2018-02-22 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20622#discussion_r170088438 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -266,6 +265,10 @@ class ContinuousExecu

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170088798 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -35,15 +35,14 @@ case class DataSourceV2Re

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-22 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20650 All the failures above can be attributed to other flakiness unrelated to the flakiness this PR trying to address. --- - To unsubscr

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170089368 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -107,17 +106,24 @@ case class DataSourceV2

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170089465 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala --- @@ -23,11 +23,11 @@ import org.apache.spark.s

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170089913 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/PushDownOperatorsToDataSource.scala --- @@ -26,7 +26,7 @@ object PushDownOpe

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170090323 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -77,31 +79,32 @@ class MicroBatchExecution(

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170090758 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -415,12 +418,14 @@ class MicroBatchExecution(

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #87620 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87620/testReport)** for PR 20622 at commit [`0e5e52f`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20622 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87620/ Test FAILed. ---

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20622 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #87623 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87623/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/ae

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170091954 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -415,12 +418,14 @@ class MicroBatchExecution(

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170092091 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala --- @@ -492,16 +492,16 @@ class StreamSuite extends StreamTest {

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170092263 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala --- @@ -17,15 +17,12 @@ package org.apache.spar

[GitHub] spark issue #20640: [SPARK-19755][Mesos] Blacklist is always active for Meso...

2018-02-22 Thread susanxhuynh
Github user susanxhuynh commented on the issue: https://github.com/apache/spark/pull/20640 @skonto We should not remove the logging. The logging [here](https://github.com/apache/spark/blob/f41c0a93fd3913ad93e55ddbfd875229872ecc97/core/src/main/scala/org/apache/spark/scheduler/Blacklist

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170095169 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -415,12 +418,14 @@ class MicroBatchExecutio

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4127 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4127/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4128 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4128/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4129 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4129/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4131 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4131/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4130 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4130/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4132 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4132/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4133 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4133/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4134 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4134/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4135 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4135/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/a

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170097795 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -415,12 +418,14 @@ class MicroBatchExecution(

[GitHub] spark issue #20640: [SPARK-19755][Mesos] Blacklist is always active for Meso...

2018-02-22 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/20640 @susanxhuynh I agree in case it is not enabled we can log failures as usual, but not for blacklisting as it is disabled it wouldnt make sense. User should have this option not to care. > In

[GitHub] spark pull request #20624: [SPARK-23445] ColumnStat refactoring

2018-02-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20624#discussion_r170102064 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -387,6 +390,101 @@ case class CatalogStatistics(

[GitHub] spark pull request #20604: [SPARK-23365][CORE] Do not adjust num executors w...

2018-02-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20604#discussion_r170102582 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -334,6 +336,11 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #20604: [SPARK-23365][CORE] Do not adjust num executors w...

2018-02-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20604#discussion_r170102363 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationClient.scala --- @@ -55,18 +55,18 @@ private[spark] trait ExecutorAllocationClient {

[GitHub] spark pull request #20604: [SPARK-23365][CORE] Do not adjust num executors w...

2018-02-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20604#discussion_r170103841 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationClient.scala --- @@ -55,18 +55,18 @@ private[spark] trait ExecutorAllocationClient {

[GitHub] spark pull request #20604: [SPARK-23365][CORE] Do not adjust num executors w...

2018-02-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20604#discussion_r170102063 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -334,6 +336,11 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170106104 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -415,12 +418,14 @@ class MicroBatchExecutio

[GitHub] spark issue #20645: SPARK-23472: Add defaultJavaOptions for drivers and exec...

2018-02-22 Thread rednaxelafx
Github user rednaxelafx commented on the issue: https://github.com/apache/spark/pull/20645 A quick question: after this change, `extraJavaOptions` is still able to cleanly override whatever's set in `defaultJavaOptions`, is that right? ^^ Just making sure I understood the inte

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170115965 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -415,12 +418,14 @@ class MicroBatchExecution(

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20653 **[Test build #87622 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87622/testReport)** for PR 20653 at commit [`d6ac338`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #20657: [SPARK-23361][yarn] Allow AM to restart after initial to...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20657 **[Test build #87621 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87621/testReport)** for PR 20657 at commit [`2c3448d`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20653 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87622/ Test PASSed. ---

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20653 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20657: [SPARK-23361][yarn] Allow AM to restart after initial to...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20657 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87621/ Test PASSed. ---

[GitHub] spark issue #20657: [SPARK-23361][yarn] Allow AM to restart after initial to...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20657 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20645: SPARK-23472: Add defaultJavaOptions for drivers and exec...

2018-02-22 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20645 > A quick question: after this change, extraJavaOptions is still able to cleanly override whatever's set in defaultJavaOptions, is that right? No, the intent is for both sets of options to be

[GitHub] spark issue #20650: [SPARK-23408][SS] Synchronize successive AddData actions...

2018-02-22 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20650 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.a

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4128 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4128/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4135 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4135/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/

[GitHub] spark issue #20660: [SPARK-23490][SQL]Check storage.locationUri with existin...

2018-02-22 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/20660 @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #87623 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87623/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20622 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87623/ Test PASSed. ---

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20622 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4127 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4127/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4130 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4130/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4133 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4133/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4131 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4131/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4132 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4132/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4134 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4134/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/

[GitHub] spark issue #20553: [SPARK-23285][K8S] Add a config property for specifying ...

2018-02-22 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/20553 Any comment or concern on this? Is this good to merge? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4129 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4129/testReport)** for PR 20622 at commit [`ae2d853`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #20553: [SPARK-23285][K8S] Add a config property for spec...

2018-02-22 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/20553#discussion_r170138218 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodFactory.scala --- @@ -144,7 +149,7 @@ private[s

[GitHub] spark pull request #20553: [SPARK-23285][K8S] Add a config property for spec...

2018-02-22 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/20553#discussion_r170137631 --- Diff: docs/running-on-kubernetes.md --- @@ -576,14 +576,21 @@ specific to Spark on Kubernetes. spark.kubernetes.driver.limit.cores (none)

[GitHub] spark pull request #20553: [SPARK-23285][K8S] Add a config property for spec...

2018-02-22 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/20553#discussion_r170138424 --- Diff: docs/running-on-kubernetes.md --- @@ -576,14 +576,21 @@ specific to Spark on Kubernetes. spark.kubernetes.driver.limit.cores (none)

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-22 Thread asolimando
Github user asolimando commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r170139957 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -402,20 +405,40 @@ class RandomForestSuite extends SparkFunSu

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-22 Thread asolimando
Github user asolimando commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r170140280 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/tree/DecisionTreeSuite.scala --- @@ -359,29 +339,6 @@ class DecisionTreeSuite extends SparkFunSuite

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-22 Thread asolimando
Github user asolimando commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r170140738 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -362,10 +365,10 @@ class RandomForestSuite extends SparkFunSu

[GitHub] spark pull request #20519: [Spark-23240][python] Don't let python site custo...

2018-02-22 Thread bersprockets
Github user bersprockets closed the pull request at: https://github.com/apache/spark/pull/20519 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.o

[GitHub] spark pull request #20658: [SPARK-23488][python] Add missing catalog methods...

2018-02-22 Thread drboyer
Github user drboyer commented on a diff in the pull request: https://github.com/apache/spark/pull/20658#discussion_r170158422 --- Diff: python/pyspark/sql/catalog.py --- @@ -28,7 +28,7 @@ Database = namedtuple("Database", "name description locationUri") Table = namedtuple(

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170159543 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -107,17 +106,24 @@ case class DataSourc

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170160050 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -77,31 +79,32 @@ class MicroBatchExecution(

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170162261 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -35,15 +35,14 @@ case class DataSourceV

[GitHub] spark issue #20648: [SPARK-23448][SQL] JSON parser should return partial row...

2018-02-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20648 Yes, thanks @HyukjinKwon for checking the behavior. If we look at the codes of JSON parser, we will find many places indicating the expectation of availability of partial results. For exampl

[GitHub] spark issue #20647: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20647 **[Test build #87624 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87624/testReport)** for PR 20647 at commit [`dbee281`](https://github.com/apache/spark/commit/db

[GitHub] spark issue #20647: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20647 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1013/ Tes

[GitHub] spark issue #20647: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20647 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20628: [SPARK-23449][K8S] Preserve extraJavaOptions ordering

2018-02-22 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20628 also cc @vanzin to verify this change. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comm

[GitHub] spark pull request #20553: [SPARK-23285][K8S] Add a config property for spec...

2018-02-22 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20553#discussion_r170168480 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodFactory.scala --- @@ -144,7 +149,7 @@ priva

[GitHub] spark pull request #20553: [SPARK-23285][K8S] Add a config property for spec...

2018-02-22 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20553#discussion_r170168795 --- Diff: docs/running-on-kubernetes.md --- @@ -576,14 +576,21 @@ specific to Spark on Kubernetes. spark.kubernetes.driver.limit.cores (none

[GitHub] spark issue #20553: [SPARK-23285][K8S] Add a config property for specifying ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20553 **[Test build #87625 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87625/testReport)** for PR 20553 at commit [`11f94e2`](https://github.com/apache/spark/commit/11

[GitHub] spark pull request #20553: [SPARK-23285][K8S] Add a config property for spec...

2018-02-22 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20553#discussion_r170169357 --- Diff: docs/running-on-kubernetes.md --- @@ -576,14 +576,21 @@ specific to Spark on Kubernetes. spark.kubernetes.driver.limit.cores (none

[GitHub] spark issue #20553: [SPARK-23285][K8S] Add a config property for specifying ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20553 **[Test build #87626 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87626/testReport)** for PR 20553 at commit [`8f457ce`](https://github.com/apache/spark/commit/8f

[GitHub] spark issue #20553: [SPARK-23285][K8S] Add a config property for specifying ...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20553 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20553: [SPARK-23285][K8S] Add a config property for specifying ...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20553 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1014/ Tes

[GitHub] spark issue #20553: [SPARK-23285][K8S] Add a config property for specifying ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20553 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1008/ --- --

[GitHub] spark issue #20553: [SPARK-23285][K8S] Add a config property for specifying ...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20553 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87625/ Test PASSed. ---

[GitHub] spark issue #20553: [SPARK-23285][K8S] Add a config property for specifying ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20553 **[Test build #87625 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87625/testReport)** for PR 20553 at commit [`11f94e2`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #20553: [SPARK-23285][K8S] Add a config property for specifying ...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20553 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20553: [SPARK-23285][K8S] Add a config property for specifying ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20553 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1008/ ---

<    1   2   3   >