[GitHub] spark pull request #21258: [SPARK-23933][SQL] Add map_from_arrays function

2018-06-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21258#discussion_r192551411 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -235,6 +235,86 @@ case class

[GitHub] spark issue #21390: [SPARK-24340][Core] Clean up non-shuffle disk block mana...

2018-06-02 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21390 sounds like we need to have some sort of lifetime management, TTL -like design shuffle file - should we have a new JIRA on that? ---

[GitHub] spark pull request #21480: [SPARK-23668][K8S] Added missing config property ...

2018-06-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21480 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21484: [WIP] Replace queryExecution by planWithBarrier in toDF

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21484 **[Test build #91414 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91414/testReport)** for PR 21484 at commit

[GitHub] spark issue #21484: [WIP] Replace queryExecution by planWithBarrier in toDF

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21484 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21484: [WIP] Replace queryExecution by planWithBarrier in toDF

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21484 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3776/

[GitHub] spark issue #21485: [SPARK-24455][CORE] fix typo in TaskSchedulerImpl commen...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21485 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21485: [SPARK-24455][CORE] fix typo in TaskSchedulerImpl...

2018-06-02 Thread xueyumusic
GitHub user xueyumusic opened a pull request: https://github.com/apache/spark/pull/21485 [SPARK-24455][CORE] fix typo in TaskSchedulerImpl comment change runTasks to submitTasks in the TaskSchedulerImpl.scala 's comment You can merge this pull request into a Git

[GitHub] spark issue #21485: [SPARK-24455][CORE] fix typo in TaskSchedulerImpl commen...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21485 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21474: [SPARK-24297][CORE] Fetch-to-disk by default for ...

2018-06-02 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21474#discussion_r192551649 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -429,7 +429,11 @@ package object config { "external

[GitHub] spark pull request #21476: [SPARK-24446][yarn] Properly quote library path f...

2018-06-02 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21476#discussion_r192551676 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -1485,6 +1486,22 @@ private object Client extends

[GitHub] spark issue #21483: [SPARK-24454][ML][PYTHON] Imports image module in ml/__i...

2018-06-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21483 yea, that was my question. I think @mengxr actually meant that in the JIRA? I'm okay eithrt way. --- - To unsubscribe,

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21370 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21370 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91410/ Test FAILed. ---

[GitHub] spark issue #21479: [SPARK-23903][SQL] Add support for date extract

2018-06-02 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21479 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21479: [SPARK-23903][SQL] Add support for date extract

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21479 **[Test build #91415 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91415/testReport)** for PR 21479 at commit

[GitHub] spark pull request #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-02 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21482#discussion_r192551752 --- Diff: R/pkg/R/functions.R --- @@ -907,6 +907,30 @@ setMethod("initcap", column(jc) }) +#' @details +#'

[GitHub] spark pull request #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-02 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21482#discussion_r192551777 --- Diff: R/pkg/NAMESPACE --- @@ -281,6 +281,8 @@ exportMethods("%<=>%", "initcap", "input_file_name",

[GitHub] spark issue #21480: [SPARK-23668][K8S] Added missing config property in runn...

2018-06-02 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21480 merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21484: [WIP] Replace queryExecution by planWithBarrier i...

2018-06-02 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/21484 [WIP] Replace queryExecution by planWithBarrier in toDF ## What changes were proposed in this pull request? TODO ## How was this patch tested? TODO You can merge this pull

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19602 **[Test build #91413 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91413/testReport)** for PR 19602 at commit

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91413/ Test FAILed. ---

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21370 **[Test build #91410 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91410/testReport)** for PR 21370 at commit

[GitHub] spark issue #21479: [SPARK-23903][SQL] Add support for date extract

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21479 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91412/ Test FAILed. ---

[GitHub] spark issue #21479: [SPARK-23903][SQL] Add support for date extract

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21479 **[Test build #91412 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91412/testReport)** for PR 21479 at commit

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21479: [SPARK-23903][SQL] Add support for date extract

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21479 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-06-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21370 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21370 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21370 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3778/

[GitHub] spark issue #21486: [SPARK-24387][Core] Heartbeat-timeout executor is added ...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21486 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21482 **[Test build #91418 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91418/testReport)** for PR 21482 at commit

[GitHub] spark issue #21485: [SPARK-24455][CORE] fix typo in TaskSchedulerImpl commen...

2018-06-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21485 Minor changes don't need a JIRA. lets avoid next time btw. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21485: [SPARK-24455][CORE] fix typo in TaskSchedulerImpl commen...

2018-06-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21485 that's fine but mind taking another look and see if there are other typos while we are here? I'm pretty sure there are more. ---

[GitHub] spark issue #21473: [SPARK-21896][SQL] Fix StackOverflow caused by window fu...

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21473 **[Test build #91419 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91419/testReport)** for PR 21473 at commit

[GitHub] spark issue #21473: [SPARK-21896][SQL] Fix StackOverflow caused by window fu...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21473 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21473: [SPARK-21896][SQL] Fix StackOverflow caused by window fu...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21473 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3779/

[GitHub] spark issue #21473: [SPARK-21896][SQL] Fix StackOverflow caused by window fu...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21473 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21484: [WIP] Replace queryExecution by planWithBarrier in toDF

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21484 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21484: [WIP] Replace queryExecution by planWithBarrier in toDF

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21484 **[Test build #91414 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91414/testReport)** for PR 21484 at commit

[GitHub] spark issue #21479: [SPARK-23903][SQL] Add support for date extract

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21479 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3777/

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21370 **[Test build #91416 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91416/testReport)** for PR 21370 at commit

[GitHub] spark issue #21473: [SPARK-21896][SQL] Fix StackOverflow caused by window fu...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21473 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3780/

[GitHub] spark issue #21481: [SPARK-24452][SQL][Core] Avoid possible overflow in int ...

2018-06-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21481 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21479: [SPARK-23903][SQL] Add support for date extract

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21479 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21482 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21482 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91417/ Test PASSed. ---

[GitHub] spark issue #21479: [SPARK-23903][SQL] Add support for date extract

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21479 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21486: [SPARK-24387][Core] Heartbeat-timeout executor is...

2018-06-02 Thread lirui-apache
GitHub user lirui-apache opened a pull request: https://github.com/apache/spark/pull/21486 [SPARK-24387][Core] Heartbeat-timeout executor is added back and used again ## What changes were proposed in this pull request? When an executor's heartbeat is lost, we call

[GitHub] spark issue #21486: [SPARK-24387][Core] Heartbeat-timeout executor is added ...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21486 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21473: [SPARK-21896][SQL] Fix StackOverflow caused by window fu...

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21473 **[Test build #91420 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91420/testReport)** for PR 21473 at commit

[GitHub] spark issue #21479: [SPARK-23903][SQL] Add support for date extract

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21479 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91415/ Test PASSed. ---

[GitHub] spark issue #21479: [SPARK-23903][SQL] Add support for date extract

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21479 **[Test build #91415 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91415/testReport)** for PR 21479 at commit

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21370 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91416/ Test PASSed. ---

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21370 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21482 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21482 **[Test build #91418 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91418/testReport)** for PR 21482 at commit

[GitHub] spark issue #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21482 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91418/ Test PASSed. ---

[GitHub] spark issue #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21482 **[Test build #91417 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91417/testReport)** for PR 21482 at commit

[GitHub] spark issue #21484: [WIP] Replace queryExecution by planWithBarrier in toDF

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21484 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91414/ Test PASSed. ---

[GitHub] spark issue #21370: [SPARK-24215][PySpark] Implement _repr_html_ for datafra...

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21370 **[Test build #91416 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91416/testReport)** for PR 21370 at commit

[GitHub] spark issue #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21482 **[Test build #91417 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91417/testReport)** for PR 21482 at commit

[GitHub] spark issue #21473: [SPARK-21896][SQL] Fix StackOverflow caused by window fu...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21473 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91419/ Test PASSed. ---

[GitHub] spark issue #21473: [SPARK-21896][SQL] Fix StackOverflow caused by window fu...

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21473 **[Test build #91419 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91419/testReport)** for PR 21473 at commit

[GitHub] spark issue #21473: [SPARK-21896][SQL] Fix StackOverflow caused by window fu...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21473 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21346: [SPARK-6237][NETWORK] Network-layer changes to al...

2018-06-02 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/21346#discussion_r192565980 --- Diff: common/network-common/src/main/java/org/apache/spark/network/server/RpcHandler.java --- @@ -38,15 +38,24 @@ * * This method

[GitHub] spark pull request #21313: [SPARK-24187][R][SQL]Add array_join function to S...

2018-06-02 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21313#discussion_r192564730 --- Diff: R/pkg/tests/fulltests/test_sparkSQL.R --- @@ -1518,6 +1518,16 @@ test_that("column functions", { result <- collect(select(df,

[GitHub] spark issue #21473: [SPARK-21896][SQL] Fix StackOverflow caused by window fu...

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21473 **[Test build #91420 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91420/testReport)** for PR 21473 at commit

[GitHub] spark issue #21473: [SPARK-21896][SQL] Fix StackOverflow caused by window fu...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21473 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91420/ Test PASSed. ---

[GitHub] spark issue #21473: [SPARK-21896][SQL] Fix StackOverflow caused by window fu...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21473 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21313: [SPARK-24187][R][SQL]Add array_join function to S...

2018-06-02 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21313#discussion_r192564574 --- Diff: R/pkg/R/functions.R --- @@ -3006,6 +3008,28 @@ setMethod("array_contains", column(jc) }) +#'

[GitHub] spark pull request #21313: [SPARK-24187][R][SQL]Add array_join function to S...

2018-06-02 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21313#discussion_r192564618 --- Diff: R/pkg/R/functions.R --- @@ -3006,6 +3008,28 @@ setMethod("array_contains", column(jc) }) +#'

[GitHub] spark pull request #21346: [SPARK-6237][NETWORK] Network-layer changes to al...

2018-06-02 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/21346#discussion_r192566116 --- Diff: common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java --- @@ -141,26 +141,14 @@ public void fetchChunk(

[GitHub] spark pull request #21484: [WIP] Replace queryExecution by planWithBarrier i...

2018-06-02 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21484#discussion_r192564074 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -390,7 +390,7 @@ class Dataset[T] private[sql]( */ // This is

[GitHub] spark issue #21390: [SPARK-24340][Core] Clean up non-shuffle disk block mana...

2018-06-02 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/21390 Feel free to do the TTL in a followup. My feeling is that it won't be super useful in practice, though: 1. Cleanup of non-shuffle disk block manager files following executor exit only

[GitHub] spark issue #21481: [SPARK-24452][SQL][Core] Avoid possible overflow in int ...

2018-06-02 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/21481 Hey @kiszk, thanks for tracking this down. This change looks good to me. I have a couple of questions, mostly aimed towards figuring out how we can categorically solve this problem:

[GitHub] spark pull request #21346: [SPARK-6237][NETWORK] Network-layer changes to al...

2018-06-02 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/21346#discussion_r192565530 --- Diff: common/network-common/src/main/java/org/apache/spark/network/protocol/UploadStream.java --- @@ -0,0 +1,107 @@ +/* + * Licensed to the

[GitHub] spark pull request #21456: [SPARK-24356] [CORE] Duplicate strings in File.pa...

2018-06-02 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21456#discussion_r192567034 --- Diff: common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolverSuite.java --- @@ -135,4 +136,23 @@ public void

[GitHub] spark issue #21282: [SPARK-23934][SQL] Adding map_from_entries function

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21282 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91421/ Test PASSed. ---

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21221 **[Test build #91424 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91424/testReport)** for PR 21221 at commit

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21221 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21221: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21221 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91424/ Test FAILed. ---

[GitHub] spark issue #21282: [SPARK-23934][SQL] Adding map_from_entries function

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21282 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21456: [SPARK-24356] [CORE] Duplicate strings in File.pa...

2018-06-02 Thread countmdm
Github user countmdm commented on a diff in the pull request: https://github.com/apache/spark/pull/21456#discussion_r192576365 --- Diff: common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolverSuite.java --- @@ -135,4 +136,23 @@ public

[GitHub] spark issue #21487: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-06-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21487 cc @maropu @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21487: [SPARK-24369][SQL] Correct handling for multiple ...

2018-06-02 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/21487 [SPARK-24369][SQL] Correct handling for multiple distinct aggregations having the same argument set ## What changes were proposed in this pull request? bring back

[GitHub] spark issue #21487: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21487 **[Test build #91425 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91425/testReport)** for PR 21487 at commit

[GitHub] spark issue #21487: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-06-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21487 also cc @vanzin , This is a bug fix that should be backported, but it's not a regression and not critical, so it should not block the release. That said, if the current RC passes, this fix will

[GitHub] spark issue #21481: [SPARK-24452][SQL][Core] Avoid possible overflow in int ...

2018-06-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21481 +1 on @JoshRosen 's ideas. I've already seen several overflow fixes from @kiszk , it will be good if we have some tools to check it, even we need to run the tool manually. One idea may

[GitHub] spark issue #21456: [SPARK-24356] [CORE] Duplicate strings in File.path mana...

2018-06-02 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21456 Merged to master. Probably Ok to backport if anyone feels strongly about it, but, let's let 2.3.1 and other imminent releases go out first. ---

[GitHub] spark issue #21487: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21487 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3782/

[GitHub] spark issue #21487: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21487 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21456: [SPARK-24356] [CORE] Duplicate strings in File.pa...

2018-06-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21456 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total size of states in HDFSBac...

2018-06-02 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21469 @jose-torres Ah yes I forgot that shallow copy has been occurring, so while new map should hold necessary size of map entries but row object will be shared across versions. Thanks for

[GitHub] spark pull request #19602: [SPARK-22384][SQL] Refine partition pruning when ...

2018-06-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19602#discussion_r192576685 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -657,17 +656,29 @@ private[client] class Shim_v0_13 extends

[GitHub] spark issue #21449: [SPARK-24385][SQL] Resolve self-join condition ambiguity...

2018-06-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21449 My point is that, we may have a different design if we wanna solve this problem holistically, which may conflict with this patch. We should prove that this is in the right direction and future

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

2018-06-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19602 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19602: [SPARK-22384][SQL] Refine partition pruning when ...

2018-06-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19602#discussion_r192576715 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/HiveClientSuite.scala --- @@ -59,38 +62,62 @@ class HiveClientSuite(version: String)

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

2018-06-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19602 **[Test build #91426 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91426/testReport)** for PR 19602 at commit

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

2018-06-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3783/

  1   2   >