[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2018-02-22 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/19788 @cloud-fan if encryption is enabled `blockManager.serializerManager().encryptionEnabled() == true`, shall we disable this feature also? ---

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19788 **[Test build #87610 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87610/testReport)** for PR 19788 at commit

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20653 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87612/ Test FAILed. ---

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19788 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87610/ Test FAILed. ---

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20653 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19788 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20653 **[Test build #87612 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87612/testReport)** for PR 20653 at commit

[GitHub] spark pull request #20649: [SPARK-23462][SQL] improve missing field error me...

2018-02-22 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20649#discussion_r169882987 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/StructTypeSuite.scala --- @@ -0,0 +1,43 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2018-02-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r169898393 --- Diff: project/MimaExcludes.scala --- @@ -1129,6 +1129,12 @@ object MimaExcludes {

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2018-02-22 Thread yucai
Github user yucai commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r169901683 --- Diff: project/MimaExcludes.scala --- @@ -1129,6 +1129,12 @@ object MimaExcludes {

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2018-02-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r169902931 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -295,8 +307,8 @@ private[spark] abstract class MapOutputTracker(conf:

[GitHub] spark issue #20659: [DNM] Try to update Hive to 2.3.2

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20659 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20659: [DNM] Try to update Hive to 2.3.2

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20659 **[Test build #87613 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87613/testReport)** for PR 20659 at commit

[GitHub] spark issue #20659: [DNM] Try to update Hive to 2.3.2

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20659 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87613/ Test FAILed. ---

[GitHub] spark issue #20659: [DNM] Try to update Hive to 2.3.2

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20659 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20659: [DNM] Try to update Hive to 2.3.2

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20659 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20659: [DNM] Try to update Hive to 2.3.2

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20659 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1008/

[GitHub] spark issue #20659: [DNM] Try to update Hive to 2.3.2

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20659 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1005/

[GitHub] spark issue #20659: [DNM] Try to update Hive to 2.3.2

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20659 **[Test build #87613 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87613/testReport)** for PR 20659 at commit

[GitHub] spark pull request #20651: [SPARK-23475][UI] Show also skipped stages

2018-02-22 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20651#discussion_r169919844 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllStagesPage.scala --- @@ -172,6 +185,20 @@ private[ui] class AllStagesPage(parent: StagesTab)

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20653 **[Test build #87614 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87614/testReport)** for PR 20653 at commit

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20653 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2018-02-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r169899114 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -295,8 +307,8 @@ private[spark] abstract class MapOutputTracker(conf:

[GitHub] spark issue #20651: [SPARK-23475][UI] Show also skipped stages

2018-02-22 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20651 @zsxwing SKIPPED stages were previously shown as PENDING. Despite I agree that your fix is needed, I think this is needed too. Do you agree @vanzin? ---

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20653 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20653 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1006/

[GitHub] spark issue #20651: [SPARK-23475][UI] Show also skipped stages

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20651 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1007/

[GitHub] spark issue #20651: [SPARK-23475][UI] Show also skipped stages

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20651 **[Test build #87615 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87615/testReport)** for PR 20651 at commit

[GitHub] spark issue #20651: [SPARK-23475][UI] Show also skipped stages

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20651 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20659: [DNM] Try to update Hive to 2.3.2

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20659 **[Test build #87616 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87616/testReport)** for PR 20659 at commit

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20653 **[Test build #87614 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87614/testReport)** for PR 20653 at commit

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20653 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20653 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87614/ Test PASSed. ---

[GitHub] spark issue #20648: [SPARK-23448][SQL] JSON parser should return partial row...

2018-02-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20648 From the codes, looks like there is an intention to have partial results when failing to parse the documents. This patch makes the partial results. But this should be considered as behavior change,

[GitHub] spark issue #20660: [SPARK-23490][SQL]Check storage.locationUri with existin...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20660 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1010/

[GitHub] spark pull request #20624: [SPARK-23445] ColumnStat refactoring

2018-02-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20624#discussion_r170028784 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -387,6 +390,101 @@ case class CatalogStatistics(

[GitHub] spark pull request #20624: [SPARK-23445] ColumnStat refactoring

2018-02-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20624#discussion_r170031873 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -387,6 +390,101 @@ case class CatalogStatistics(

[GitHub] spark issue #20660: [SPARK-23490][SQL]Check storage.locationUri with existin...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20660 **[Test build #87618 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87618/testReport)** for PR 20660 at commit

[GitHub] spark issue #20652: [SPARK-23476][CORE] Generate secret in local mode when a...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20652 **[Test build #87619 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87619/testReport)** for PR 20652 at commit

[GitHub] spark issue #20651: [SPARK-23475][UI] Show also skipped stages

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20651 **[Test build #87617 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87617/testReport)** for PR 20651 at commit

[GitHub] spark issue #20622: [SPARK-23441][SS] Remove queryExecutionThread.interrupt(...

2018-02-22 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20622 The difference in ContinuousExecution is that the thread isn't doing any metadata work like looking for new batches - it's either running the Spark job or cleaning up after finishing it.

[GitHub] spark issue #10942: [SPARK-12850] [SQL] Support Bucket Pruning (Predicate Pu...

2018-02-22 Thread lonehacker
Github user lonehacker commented on the issue: https://github.com/apache/spark/pull/10942 @gatorsmile Can you confirm if this feature is active in current master? It seems like this code was removed in [SPARK-14535][SQL] Remove buildInternalScan from FileFormat ---

[GitHub] spark pull request #20653: [SPARK-23459][SQL] Improve the error message when...

2018-02-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20653#discussion_r170025695 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategySuite.scala --- @@ -491,6 +491,22 @@ class

[GitHub] spark issue #20651: [SPARK-23475][UI] Show also skipped stages

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20651 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87617/ Test PASSed. ---

[GitHub] spark issue #20660: [SPARK-23490][SQL]Check storage.locationUri with existin...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20660 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20660: [SPARK-23490][SQL]Check storage.locationUri with existin...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20660 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87618/ Test PASSed. ---

[GitHub] spark pull request #20652: [SPARK-23476][CORE] Generate secret in local mode...

2018-02-22 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/20652#discussion_r170003945 --- Diff: core/src/test/scala/org/apache/spark/SecurityManagerSuite.scala --- @@ -440,23 +440,41 @@ class SecurityManagerSuite extends SparkFunSuite

[GitHub] spark issue #20362: [Spark-22886][ML][TESTS] ML test for structured streamin...

2018-02-22 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/20362 ping @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-02-22 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r169995800 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -45,38 +44,149 @@ */ public static final int

[GitHub] spark issue #20645: SPARK-23472: Add defaultJavaOptions for drivers and exec...

2018-02-22 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/20645 I agree it would be nicer to have this be a more general feature. I would prefer an approach which didn't require a different configuration name, just as its more to document & for users to keep

[GitHub] spark issue #20362: [Spark-22886][ML][TESTS] ML test for structured streamin...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20362 **[Test build #4126 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4126/testReport)** for PR 20362 at commit

[GitHub] spark issue #20645: SPARK-23472: Add defaultJavaOptions for drivers and exec...

2018-02-22 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20645 I like the `ConfigBuilder` approach. That would make this much more useful. I'll add an implementation like that. I think append option syntax would be confusing for users and

[GitHub] spark issue #20660: [SPARK-23490][SQL]Check storage.locationUri with existin...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20660 **[Test build #87618 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87618/testReport)** for PR 20660 at commit

[GitHub] spark issue #20660: [SPARK-23490][SQL]Check storage.locationUri with existin...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20660 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20658: [SPARK-23488][python] Add missing catalog methods...

2018-02-22 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20658#discussion_r170003647 --- Diff: python/pyspark/sql/catalog.py --- @@ -137,6 +138,78 @@ def listColumns(self, tableName, dbName=None):

[GitHub] spark pull request #20658: [SPARK-23488][python] Add missing catalog methods...

2018-02-22 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20658#discussion_r170003442 --- Diff: python/pyspark/sql/catalog.py --- @@ -28,7 +28,7 @@ Database = namedtuple("Database", "name description locationUri") Table =

[GitHub] spark issue #20651: [SPARK-23475][UI] Show also skipped stages

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20651 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #87620 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87620/testReport)** for PR 20622 at commit

[GitHub] spark pull request #20660: [SPARK-23490][SQL]Check storage.locationUri with ...

2018-02-22 Thread gengliangwang
GitHub user gengliangwang opened a pull request: https://github.com/apache/spark/pull/20660 [SPARK-23490][SQL]Check storage.locationUri with existing table in CreateTable ## What changes were proposed in this pull request? For CreateTable with Append mode, we should check

[GitHub] spark pull request #20652: [SPARK-23476][CORE] Generate secret in local mode...

2018-02-22 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20652#discussion_r169996395 --- Diff: core/src/test/scala/org/apache/spark/SecurityManagerSuite.scala --- @@ -440,23 +440,41 @@ class SecurityManagerSuite extends SparkFunSuite

[GitHub] spark pull request #20362: [Spark-22886][ML][TESTS] ML test for structured s...

2018-02-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20362#discussion_r170046857 --- Diff: mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala --- @@ -413,34 +411,36 @@ class ALSSuite .setSeed(0)

[GitHub] spark pull request #20362: [Spark-22886][ML][TESTS] ML test for structured s...

2018-02-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20362#discussion_r170046788 --- Diff: mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala --- @@ -586,6 +586,68 @@ class ALSSuite allModelParamSettings,

[GitHub] spark pull request #20362: [Spark-22886][ML][TESTS] ML test for structured s...

2018-02-22 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20362#discussion_r170047180 --- Diff: mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala --- @@ -693,7 +766,7 @@ class ALSSuite val data = ratings.toDF

[GitHub] spark issue #20657: [SPARK-23361][yarn] Allow AM to restart after initial to...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20657 **[Test build #87621 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87621/testReport)** for PR 20657 at commit

[GitHub] spark issue #20657: [SPARK-23361][yarn] Allow AM to restart after initial to...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20657 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1011/

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20653 **[Test build #87622 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87622/testReport)** for PR 20653 at commit

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170089368 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -107,17 +106,24 @@ case class

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170089465 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala --- @@ -23,11 +23,11 @@ import

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #87623 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87623/testReport)** for PR 20622 at commit

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170091954 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -415,12 +418,14 @@ class MicroBatchExecution(

[GitHub] spark issue #20640: [SPARK-19755][Mesos] Blacklist is always active for Meso...

2018-02-22 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/20640 @susanxhuynh I agree in case it is not enabled we can log failures as usual, but not for blacklisting as it is disabled it wouldnt make sense. User should have this option not to care. >

[GitHub] spark pull request #20604: [SPARK-23365][CORE] Do not adjust num executors w...

2018-02-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20604#discussion_r170102582 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -334,6 +336,11 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #20604: [SPARK-23365][CORE] Do not adjust num executors w...

2018-02-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20604#discussion_r170103841 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationClient.scala --- @@ -55,18 +55,18 @@ private[spark] trait ExecutorAllocationClient {

[GitHub] spark pull request #20604: [SPARK-23365][CORE] Do not adjust num executors w...

2018-02-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20604#discussion_r170102063 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -334,6 +336,11 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #20604: [SPARK-23365][CORE] Do not adjust num executors w...

2018-02-22 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20604#discussion_r170102363 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationClient.scala --- @@ -55,18 +55,18 @@ private[spark] trait ExecutorAllocationClient {

[GitHub] spark issue #20657: [SPARK-23361][yarn] Allow AM to restart after initial to...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20657 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20651: [SPARK-23475][UI] Show also skipped stages

2018-02-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20651 Merged to master, but there were conflicts in 2.3; please open a separate PR for the backport. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20653 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1012/

[GitHub] spark issue #20653: [SPARK-23459][SQL] Improve the error message when unknow...

2018-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20653 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20362: [Spark-22886][ML][TESTS] ML test for structured s...

2018-02-22 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/20362#discussion_r170072578 --- Diff: mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala --- @@ -413,34 +411,36 @@ class ALSSuite .setSeed(0)

[GitHub] spark issue #20652: [SPARK-23476][CORE] Generate secret in local mode when a...

2018-02-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20652 Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20624: [SPARK-23445] ColumnStat refactoring

2018-02-22 Thread juliuszsompolski
Github user juliuszsompolski commented on a diff in the pull request: https://github.com/apache/spark/pull/20624#discussion_r170086812 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -387,6 +390,101 @@ case class

[GitHub] spark pull request #20622: [SPARK-23491][SS] Remove explicit job cancellatio...

2018-02-22 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20622#discussion_r170088438 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala --- @@ -266,6 +265,10 @@ class

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170089913 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/PushDownOperatorsToDataSource.scala --- @@ -26,7 +26,7 @@ object

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4134 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4134/testReport)** for PR 20622 at commit

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4131 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4131/testReport)** for PR 20622 at commit

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4130 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4130/testReport)** for PR 20622 at commit

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4132 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4132/testReport)** for PR 20622 at commit

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4133 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4133/testReport)** for PR 20622 at commit

[GitHub] spark issue #20657: [SPARK-23361][yarn] Allow AM to restart after initial to...

2018-02-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20657 Known flaky (SPARK-23458). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20657: [SPARK-23361][yarn] Allow AM to restart after initial to...

2018-02-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20657 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20640: [SPARK-19755][Mesos] Blacklist is always active for Meso...

2018-02-22 Thread susanxhuynh
Github user susanxhuynh commented on the issue: https://github.com/apache/spark/pull/20640 @skonto We should not remove the logging. The logging

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4127 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4127/testReport)** for PR 20622 at commit

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4128 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4128/testReport)** for PR 20622 at commit

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4129 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4129/testReport)** for PR 20622 at commit

[GitHub] spark issue #20651: [SPARK-23475][UI] Show also skipped stages

2018-02-22 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20651 LGTM, merging to master / 2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20652: [SPARK-23476][CORE] Generate secret in local mode...

2018-02-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20652 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20622: [SPARK-23491][SS] Remove explicit job cancellation from ...

2018-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20622 **[Test build #4135 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4135/testReport)** for PR 20622 at commit

[GitHub] spark pull request #20647: [SPARK-23303][SQL] improve the explain result for...

2018-02-22 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20647#discussion_r170106104 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -415,12 +418,14 @@ class

[GitHub] spark pull request #20651: [SPARK-23475][UI] Show also skipped stages

2018-02-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20651 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

  1   2   3   >