[GitHub] spark pull request #21565: [SPARK-24558][Core]wrong Idle Timeout value is us...

2018-07-04 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/21565#discussion_r200019612 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -488,9 +488,16 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21657 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92590/ Test PASSed. ---

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21657 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21657 **[Test build #92590 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92590/testReport)** for PR 21657 at commit

[GitHub] spark issue #21687: [SPARK-24165][SQL] Fixing CaseWhenCoercion rule to handl...

2018-07-04 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21687 Tightening `CaseWhenCoercion` rule should be okay, but I guess we need to fix the output data type of `CaseWhen` and `If` with the same reason as

[GitHub] spark issue #21675: [SPARK-24698][PYTHON]: Fixed typo in pyspark.ml's Identi...

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21675 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92625/ Test PASSed. ---

[GitHub] spark issue #21675: [SPARK-24698][PYTHON]: Fixed typo in pyspark.ml's Identi...

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21675 **[Test build #92625 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92625/testReport)** for PR 21675 at commit

[GitHub] spark issue #21675: [SPARK-24698][PYTHON]: Fixed typo in pyspark.ml's Identi...

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21675 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21305 **[Test build #92627 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92627/testReport)** for PR 21305 at commit

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-04 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21305 @cloud-fan, can you also review this PR for DataSourceV2? This adds the first of the logical plans proposed in [SPIP: Standardize Logical

[GitHub] spark pull request #21608: [SPARK-24626] [SQL] Improve location size calcula...

2018-07-04 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21608#discussion_r200207191 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/CommandUtils.scala --- @@ -47,22 +47,34 @@ object CommandUtils extends Logging

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-07-04 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r200210431 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -82,7 +83,12 @@ class UnivocityParser(

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-07-04 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r200210434 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -197,15 +203,21 @@ class UnivocityParser(

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-07-04 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21596 Yes, long term that is the right thing to do. It still introduces a user-visible classpath change, which is why I'm on the fence about doing so in Spark 2.4. I feel like several people out there use

[GitHub] spark pull request #21698: [SPARK-23243][Core] Fix RDD.repartition() data co...

2018-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21698#discussion_r200220771 --- Diff: core/src/main/scala/org/apache/spark/scheduler/Stage.scala --- @@ -82,15 +82,15 @@ private[scheduler] abstract class Stage( private var

[GitHub] spark pull request #21552: [SPARK-24544][SQL] Print actual failure cause whe...

2018-07-04 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21552#discussion_r200224239 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala --- @@ -131,6 +132,8 @@ private[sql] class HiveSessionCatalog(

[GitHub] spark issue #21652: [SPARK-24551][K8S] Add integration tests for secrets

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21652 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/691/

[GitHub] spark issue #21652: [SPARK-24551][K8S] Add integration tests for secrets

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21652 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/691/ ---

[GitHub] spark issue #21652: [SPARK-24551][K8S] Add integration tests for secrets

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21652 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21656: [SPARK-24677][Core]MedianHeap is empty when speculation ...

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21656 **[Test build #92631 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92631/testReport)** for PR 21656 at commit

[GitHub] spark pull request #21664: [SPARK-24678][CORE] NoClassDefFoundError will not...

2018-07-04 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21664#discussion_r200220083 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1049,6 +1049,13 @@ class DAGScheduler( abortStage(stage,

[GitHub] spark pull request #21667: [SPARK-24691][SQL]Dispatch the type support check...

2018-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21667#discussion_r200236731 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala --- @@ -178,6 +178,24 @@ class OrcFileFormat extends FileFormat with

[GitHub] spark pull request #21405: [SPARK-24361][SQL] Polish code block manipulation...

2018-07-04 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21405#discussion_r200236745 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -22,14 +22,17 @@ import java.lang.{Boolean =>

[GitHub] spark pull request #21405: [SPARK-24361][SQL] Polish code block manipulation...

2018-07-04 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21405#discussion_r200236723 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -22,14 +22,17 @@ import java.lang.{Boolean =>

[GitHub] spark pull request #21667: [SPARK-24691][SQL]Dispatch the type support check...

2018-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21667#discussion_r200236759 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormat.scala --- @@ -152,6 +152,11 @@ trait FileFormat { }

[GitHub] spark pull request #21667: [SPARK-24691][SQL]Dispatch the type support check...

2018-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21667#discussion_r200236630 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/text/TextFileFormat.scala --- @@ -141,6 +136,11 @@ class TextFileFormat

[GitHub] spark issue #21713: [SPARK-24737][SQL] Type coercion between StructTypes.

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21713 **[Test build #92636 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92636/testReport)** for PR 21713 at commit

[GitHub] spark issue #21713: [SPARK-24737][SQL] Type coercion between StructTypes.

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21713 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/696/

[GitHub] spark issue #21713: [SPARK-24737][SQL] Type coercion between StructTypes.

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21713 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21608: [SPARK-24626] [SQL] Improve location size calculation in...

2018-07-04 Thread Achuth17
Github user Achuth17 commented on the issue: https://github.com/apache/spark/pull/21608 @maropu I have made the changes. What are the next steps? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21716: [SPARK-17213][FOLLOW-UP] Improve the test of SPARK-17213

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21716 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/688/

[GitHub] spark issue #21697: [SPARK-24711][K8S] Fix tags for integration tests

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21697 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92624/ Test PASSed. ---

[GitHub] spark issue #21716: [SPARK-17213][FOLLOW-UP] Improve the test of SPARK-17213

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21716 **[Test build #92626 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92626/testReport)** for PR 21716 at commit

[GitHub] spark issue #21716: [SPARK-17213][FOLLOW-UP] Improve the test of SPARK-17213

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21716 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-07-04 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r200210059 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -82,7 +83,12 @@ class UnivocityParser(

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21657 **[Test build #92629 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92629/testReport)** for PR 21657 at commit

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21657 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/690/

[GitHub] spark issue #21652: [SPARK-24551][K8S] Add integration tests for secrets

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21652 **[Test build #92630 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92630/testReport)** for PR 21652 at commit

[GitHub] spark issue #21652: [SPARK-24551][K8S] Add integration tests for secrets

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21652 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/691/ ---

[GitHub] spark issue #21603: [SPARK-17091][SQL] Add rule to convert IN predicate to e...

2018-07-04 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21603 @wangyum, mind resolving the conflicts please? I think it's quite close. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21716: [SPARK-17213][FOLLOW-UP] Improve the test of SPARK-17213

2018-07-04 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21716 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21305 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92627/ Test PASSed. ---

[GitHub] spark issue #21677: [SPARK-24692][TESTS] Improvement FilterPushdownBenchmark

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21677 **[Test build #92633 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92633/testReport)** for PR 21677 at commit

[GitHub] spark issue #21677: [SPARK-24692][TESTS] Improvement FilterPushdownBenchmark

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21677 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21658: [SPARK-24678][Spark-Streaming] Give priority in u...

2018-07-04 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21658#discussion_r200226750 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -1569,7 +1569,7 @@ private[spark] object BlockManager {

[GitHub] spark issue #21658: [SPARK-24678][Spark-Streaming] Give priority in use of '...

2018-07-04 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21658 Would you please add a UT for it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21305#discussion_r200230131 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -172,6 +173,7 @@ class Analyzer(

[GitHub] spark pull request #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21305#discussion_r200230408 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -40,17 +44,24 @@ case class

[GitHub] spark issue #21656: [SPARK-24677][Core]MedianHeap is empty when speculation ...

2018-07-04 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21656 cc @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21584: [SPARK-24433][K8S][WIP] Initial R Bindings for SparkR on...

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21584 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21667: [SPARK-24691][SQL]Dispatch the type support check...

2018-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21667#discussion_r200236504 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVFileFormat.scala --- @@ -153,6 +151,16 @@ class CSVFileFormat

[GitHub] spark issue #21584: [SPARK-24433][K8S][WIP] Initial R Bindings for SparkR on...

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21584 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/695/

[GitHub] spark issue #21584: [SPARK-24433][K8S][WIP] Initial R Bindings for SparkR on...

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21584 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/695/ ---

[GitHub] spark issue #21656: [SPARK-24677][Core]MedianHeap is empty when speculation ...

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21656 **[Test build #92631 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92631/testReport)** for PR 21656 at commit

[GitHub] spark issue #21709: [SPARK-5152][CORE] Read metrics config file from Hadoop ...

2018-07-04 Thread jzhuge
Github user jzhuge commented on the issue: https://github.com/apache/spark/pull/21709 Thx @jerryshao for looking at the PR. Compared to spark.metrics.conf.* in SparkConf, a metrics config file can group the properties together, separate from the rest of the properties. In my

[GitHub] spark issue #21675: [SPARK-24698][PYTHON]: Fixed typo in pyspark.ml's Identi...

2018-07-04 Thread mcteo
Github user mcteo commented on the issue: https://github.com/apache/spark/pull/21675 Hey @HyukjinKwon, made that change. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21675: [SPARK-24698][PYTHON]: Fixed typo in pyspark.ml's Identi...

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21675 **[Test build #92625 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92625/testReport)** for PR 21675 at commit

[GitHub] spark issue #21697: [SPARK-24711][K8S] Fix tags for integration tests

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21697 **[Test build #92624 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92624/testReport)** for PR 21697 at commit

[GitHub] spark issue #21697: [SPARK-24711][K8S] Fix tags for integration tests

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21697 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21608: [SPARK-24626] [SQL] Improve location size calculation in...

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21608 **[Test build #92628 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92628/testReport)** for PR 21608 at commit

[GitHub] spark issue #21657: [SPARK-24676][SQL] Project required data from CSV parsed...

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21657 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21712: [SPARK-22384][SQL][followup] Refine partition pruning wh...

2018-07-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21712 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21716: [SPARK-17213][FOLLOW-UP] Improve the test of SPAR...

2018-07-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21716 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21677: [SPARK-24692][TESTS] Improvement FilterPushdownBenchmark

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21677 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/694/

[GitHub] spark issue #21552: [SPARK-24544][SQL] Print actual failure cause when look ...

2018-07-04 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21552 Why do we call `GenericUDTF.initialize` during function lookup? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21305#discussion_r200230884 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -344,6 +344,36 @@ case class Join(

[GitHub] spark issue #21663: [SPARK-24680][Deploy]Support spark.executorEnv.JAVA_HOME...

2018-07-04 Thread stanzhai
Github user stanzhai commented on the issue: https://github.com/apache/spark/pull/21663 @jerryshao My Spark Application is built on top of JDK10, but the standalone cluster manager is running with JDK8 which does not support JDK10. Java 7 support has been removed since Spark

[GitHub] spark issue #21652: [SPARK-24551][K8S] Add integration tests for secrets

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21652 **[Test build #92630 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92630/testReport)** for PR 21652 at commit

[GitHub] spark pull request #21712: [SPARK-22384][SQL][followup] Refine partition pru...

2018-07-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21712 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21709: [SPARK-5152][CORE] Read metrics config file from ...

2018-07-04 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21709#discussion_r200219590 --- Diff: core/src/main/scala/org/apache/spark/metrics/MetricsConfig.scala --- @@ -129,8 +131,11 @@ private[spark] class MetricsConfig(conf: SparkConf)

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21305 **[Test build #92627 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92627/testReport)** for PR 21305 at commit

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21305 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21633: [SPARK-24646][CORE] Minor change to spark.yarn.dist.forc...

2018-07-04 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21633 Any more comment @vanzin ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19233: [Spark-22008][Streaming]Spark Streaming Dynamic Allocati...

2018-07-04 Thread mayuehappy
Github user mayuehappy commented on the issue: https://github.com/apache/spark/pull/19233 @srowen Really thanks for your reply.I think maybe I didn't express it well.Let's assume that there is a situation like this. If we use spark streaming to consume a Kafka topic with 10

[GitHub] spark pull request #21619: [SPARK-24635][SQL] Remove Blocks class from JavaC...

2018-07-04 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21619#discussion_r200225290 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -190,26 +194,29 @@ object Block {

[GitHub] spark issue #21658: [SPARK-24678][Spark-Streaming] Give priority in use of '...

2018-07-04 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21658 ok to test. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21405: [SPARK-24361][SQL] Polish code block manipulation...

2018-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21405#discussion_r200227776 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -22,14 +22,17 @@ import java.lang.{Boolean

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-07-04 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r200204856 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -197,15 +203,21 @@ class UnivocityParser(

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-07-04 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r200197165 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -82,7 +83,12 @@ class UnivocityParser(

[GitHub] spark pull request #21716: [SPARK-17213][FOLLOW-UP] Improve the test of SPAR...

2018-07-04 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/21716 [SPARK-17213][FOLLOW-UP] Improve the test of SPARK-17213 ## What changes were proposed in this pull request? This is a minor improvement for the test of SPARK-17213 ## How was this

[GitHub] spark issue #21652: [SPARK-24551][K8S] Add integration tests for secrets

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21652 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92630/ Test PASSed. ---

[GitHub] spark issue #21652: [SPARK-24551][K8S] Add integration tests for secrets

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21652 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21716: [SPARK-17213][FOLLOW-UP] Improve the test of SPARK-17213

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21716 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21716: [SPARK-17213][FOLLOW-UP] Improve the test of SPARK-17213

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21716 **[Test build #92626 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92626/testReport)** for PR 21716 at commit

[GitHub] spark issue #21709: [SPARK-5152][CORE] Read metrics config file from Hadoop ...

2018-07-04 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21709 From my point, I don't see a special reason that it cannot be achieved by SparkConf. Also you can specify different local metrics conf file for different users/purpose, I don't think it is

[GitHub] spark pull request #21698: [SPARK-23243][Core] Fix RDD.repartition() data co...

2018-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21698#discussion_r200220663 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1386,8 +1386,12 @@ class DAGScheduler( )

[GitHub] spark pull request #21698: [SPARK-23243][Core] Fix RDD.repartition() data co...

2018-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21698#discussion_r200220378 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -452,6 +452,10 @@ abstract class RDD[T: ClassTag]( /** Distributes elements

[GitHub] spark issue #21677: [SPARK-24692][TESTS] Improvement FilterPushdownBenchmark

2018-07-04 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21677 @HyukjinKwon Can you merge this to master first? I would like to update the [Benchmark results](https://github.com/apache/spark/pull/21677/files#diff-c5c0bfc86983d5779269cf75da8ed645) of several

[GitHub] spark pull request #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-04 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21305#discussion_r200230683 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -240,21 +238,27 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark issue #21584: [SPARK-24433][K8S][WIP] Initial R Bindings for SparkR on...

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21584 **[Test build #92635 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92635/testReport)** for PR 21584 at commit

[GitHub] spark issue #21584: [SPARK-24433][K8S][WIP] Initial R Bindings for SparkR on...

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21584 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/695/ ---

[GitHub] spark issue #21697: [SPARK-24711][K8S] Fix tags for integration tests

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21697 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21697: [SPARK-24711][K8S] Fix tags for integration tests

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21697 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92622/ Test PASSed. ---

[GitHub] spark issue #21697: [SPARK-24711][K8S] Fix tags for integration tests

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21697 **[Test build #92622 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92622/testReport)** for PR 21697 at commit

[GitHub] spark pull request #21657: [SPARK-24676][SQL] Project required data from CSV...

2018-07-04 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21657#discussion_r200197250 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -82,7 +83,12 @@ class UnivocityParser(

[GitHub] spark issue #21608: [SPARK-24626] [SQL] Improve location size calculation in...

2018-07-04 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21608 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21675: [SPARK-24698][PYTHON]: Fixed typo in pyspark.ml's Identi...

2018-07-04 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21675 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-07-04 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20345 **[Test build #92632 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92632/testReport)** for PR 20345 at commit

[GitHub] spark issue #21677: [SPARK-24692][TESTS] Improvement FilterPushdownBenchmark

2018-07-04 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21677 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

  1   2   3   4   5   >