[GitHub] spark pull request #21250: [SPARK-23291][SQL][R][BRANCH-2.3] R's substr shou...

2018-05-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21250#discussion_r186410353 --- Diff: docs/sparkr.md --- @@ -663,3 +663,7 @@ You can inspect the search path in R with [`search()`](https://stat.ethz.ch/R-ma - The

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2997/

[GitHub] spark issue #21106: [SPARK-23711][SQL][WIP] Add fallback logic for UnsafePro...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21106: [SPARK-23711][SQL][WIP] Add fallback logic for UnsafePro...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2996/

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21252: [SPARK-24193] Sort by disk when number of limit is big i...

2018-05-07 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/21252 The changes looks good to me, but it should also be great to have a test suite to cover this change. Seems we don't have a test suite for the rule `SpecialLimits` ? ---

[GitHub] spark issue #21096: [SPARK-24011][CORE][WIP] cache rdd's immediate parent Sh...

2018-05-07 Thread Ngone51
Github user Ngone51 commented on the issue: https://github.com/apache/spark/pull/21096 Thanks for your opinions @squito @markhamstra . Maybe, I should leave it for now. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21106: [SPARK-23711][SQL][WIP] Add fallback logic for UnsafePro...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21106 **[Test build #90319 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90319/testReport)** for PR 21106 at commit

[GitHub] spark issue #21106: [SPARK-23711][SQL][WIP] Add fallback logic for UnsafePro...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21106 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3001/

[GitHub] spark issue #21240: [SPARK-21274][SQL] Add a new generator function replicat...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21240 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21256: [SPARK-24160][FOLLOWUP] Fix compilation failure

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21256 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90300/ Test PASSed. ---

[GitHub] spark issue #21256: [SPARK-24160][FOLLOWUP] Fix compilation failure

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21256 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21106: [SPARK-23711][SQL][WIP] Add fallback logic for UnsafePro...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21106 **[Test build #90313 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90313/testReport)** for PR 21106 at commit

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistics to improve ...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16677 **[Test build #90311 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90311/testReport)** for PR 16677 at commit

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2995/

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2018-05-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18581 Nope not yet, I will try to make it within the next release soon. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2999/

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21190: [SPARK-22938][SQL][followup] Assert that SQLConf.get is ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21190 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3000/

[GitHub] spark pull request #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21258#discussion_r186418197 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -235,6 +235,69 @@ case class

[GitHub] spark pull request #20604: [SPARK-23365][CORE] Do not adjust num executors w...

2018-05-07 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/20604#discussion_r186425765 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1643,7 +1646,10 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark issue #21144: [SPARK-24043][SQL] Interpreted Predicate should initiali...

2018-05-07 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/21144 @hvanhovell @maropu Is there anything on this PR that I should do? --- - To unsubscribe, e-mail:

[GitHub] spark issue #18581: [SPARK-21289][SQL][ML] Supports custom line separator fo...

2018-05-07 Thread beickhoff
Github user beickhoff commented on the issue: https://github.com/apache/spark/pull/18581 @HyukjinKwon, is there another PR to handle CSV? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21249: [SPARK-23291][R][FOLLOWUP] Update SparkR migration note ...

2018-05-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21249 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21250: [SPARK-23291][SQL][R][BRANCH-2.3] R's substr shou...

2018-05-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21250#discussion_r186413375 --- Diff: docs/sparkr.md --- @@ -663,3 +663,7 @@ You can inspect the search path in R with [`search()`](https://stat.ethz.ch/R-ma - The

[GitHub] spark issue #21190: [SPARK-22938][SQL][followup] Assert that SQLConf.get is ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21190 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21238: [SPARK-24137][K8s] Mount local directories as empty dir ...

2018-05-07 Thread andrusha
Github user andrusha commented on the issue: https://github.com/apache/spark/pull/21238 Seems like it addresses similar problem to https://github.com/apache/spark/pull/21095. It might be worth investigating how to unify both. ---

[GitHub] spark issue #21251: [SPARK-10878][core] Fix race condition when multiple cli...

2018-05-07 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21251 cc @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21256: [SPARK-24160][FOLLOWUP] Fix compilation failure

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21256 **[Test build #90300 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90300/testReport)** for PR 21256 at commit

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-05-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r186411390 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -207,9 +207,25 @@ case class

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-07 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r186417334 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -28,6 +30,34 @@ import

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-07 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r186417229 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -28,6 +30,34 @@ import

[GitHub] spark issue #21209: [SPARK-24141][CORE] Fix bug in CoarseGrainedSchedulerBac...

2018-05-07 Thread Ngone51
Github user Ngone51 commented on the issue: https://github.com/apache/spark/pull/21209 ping @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21073 **[Test build #90320 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90320/testReport)** for PR 21073 at commit

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2018-05-07 Thread metasim
Github user metasim commented on the issue: https://github.com/apache/spark/pull/16478 Does this PR remove the ability to read directly from Tungsten memory, in cases where one wants to avoid the serialization/deserialization costs? IOW, in an `Expression` over a UDT, can one still

[GitHub] spark issue #21082: [SPARK-22239][SQL][Python] Enable grouped aggregate pand...

2018-05-07 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/21082 Hi @ueshin @hvanhovell can you take a look at this please? Thanks! --- - To unsubscribe, e-mail:

[GitHub] spark issue #20787: [MINOR][DOCS] Documenting months_between direction

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20787 **[Test build #90321 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90321/testReport)** for PR 20787 at commit

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21028 **[Test build #90322 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90322/testReport)** for PR 21028 at commit

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r186445302 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -28,6 +30,34 @@ import

[GitHub] spark issue #21250: [SPARK-23291][SQL][R][BRANCH-2.3] R's substr should not ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21250 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21250: [SPARK-23291][SQL][R][BRANCH-2.3] R's substr should not ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21250 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3005/

[GitHub] spark issue #21118: SPARK-23325: Use InternalRow when reading with DataSourc...

2018-05-07 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21118 > Actually the `SupportsScanUnsafeRow` is only there to avoid perf regression for migrating file sources. If you think that's not a good public API, we can move it to internal package and only use

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21258 **[Test build #90314 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90314/testReport)** for PR 21258 at commit

[GitHub] spark issue #21249: [SPARK-23291][R][FOLLOWUP] Update SparkR migration note ...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21249 **[Test build #90326 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90326/testReport)** for PR 21249 at commit

[GitHub] spark issue #21249: [SPARK-23291][R][FOLLOWUP] Update SparkR migration note ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21249 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21249: [SPARK-23291][R][FOLLOWUP] Update SparkR migration note ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21249 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90326/ Test PASSed. ---

[GitHub] spark pull request #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21258#discussion_r186464404 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -235,6 +235,69 @@ case class

[GitHub] spark pull request #20787: [MINOR][DOCS] Documenting months_between directio...

2018-05-07 Thread aditkumar
Github user aditkumar commented on a diff in the pull request: https://github.com/apache/spark/pull/20787#discussion_r186464442 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -1194,13 +1194,21 @@ case class

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r186445741 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -28,6 +30,34 @@ import

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3003/

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20929: [SPARK-23772][SQL][WIP] Provide an option to ignore colu...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20929 **[Test build #90324 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90324/testReport)** for PR 20929 at commit

[GitHub] spark issue #20929: [SPARK-23772][SQL][WIP] Provide an option to ignore colu...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20929 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3004/

[GitHub] spark issue #20929: [SPARK-23772][SQL][WIP] Provide an option to ignore colu...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20929 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20787: [MINOR][DOCS] Documenting months_between directio...

2018-05-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20787#discussion_r186458260 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -1194,13 +1194,21 @@ case class

[GitHub] spark issue #21249: [SPARK-23291][R][FOLLOWUP] Update SparkR migration note ...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21249 **[Test build #90326 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90326/testReport)** for PR 21249 at commit

[GitHub] spark issue #21250: [SPARK-23291][SQL][R][BRANCH-2.3] R's substr should not ...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21250 **[Test build #90325 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90325/testReport)** for PR 21250 at commit

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-07 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r186458781 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -28,6 +30,34 @@ import

[GitHub] spark issue #20787: [MINOR][DOCS] Documenting months_between direction

2018-05-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20787 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21249: [SPARK-23291][R][FOLLOWUP] Update SparkR migration note ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21249 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3006/

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90314/ Test FAILed. ---

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186463018 --- Diff: hadoop-cloud/src/main/scala/org/apache/spark/internal/io/cloud/PathCommitterConstants.scala --- @@ -0,0 +1,87 @@ +/* + * Licensed

[GitHub] spark pull request #21066: [SPARK-23977][CLOUD][WIP] Add commit protocol bin...

2018-05-07 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21066#discussion_r186463919 --- Diff: hadoop-cloud/src/main/scala/org/apache/spark/internal/io/cloud/PathCommitterConstants.scala --- @@ -0,0 +1,87 @@ +/* + * Licensed

[GitHub] spark pull request #21250: [SPARK-23291][SQL][R][BRANCH-2.3] R's substr shou...

2018-05-07 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21250#discussion_r186438139 --- Diff: docs/sparkr.md --- @@ -663,3 +663,7 @@ You can inspect the search path in R with [`search()`](https://stat.ethz.ch/R-ma - The

[GitHub] spark issue #21248: [SPARK-24191][ML]Example code for Power Iteration Cluste...

2018-05-07 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/21248 cc @mengxr @WeichenXu123 @felixcheung. Can you please verify this patch? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

2018-05-07 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/19602 I rebased this pr and resolved conflicts. cc @cloud-fan @jiangxb1987 Not sure if you have interest on this. Take a look if have time. Thanks ! ---

[GitHub] spark pull request #21122: [SPARK-24017] [SQL] Refactor ExternalCatalog to b...

2018-05-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21122#discussion_r186452677 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -1354,7 +1354,8 @@ class HiveDDLSuite val

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-07 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r186455988 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -28,6 +30,34 @@ import

[GitHub] spark issue #21249: [SPARK-23291][R][FOLLOWUP] Update SparkR migration note ...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21249 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10....

2018-05-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21070#discussion_r186464674 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedPlainValuesReader.java --- @@ -63,115 +59,157 @@ public final

[GitHub] spark pull request #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10....

2018-05-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21070#discussion_r186464557 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedPlainValuesReader.java --- @@ -63,115 +59,157 @@ public final

[GitHub] spark pull request #20787: [MINOR][DOCS] Documenting months_between directio...

2018-05-07 Thread aditkumar
Github user aditkumar commented on a diff in the pull request: https://github.com/apache/spark/pull/20787#discussion_r186436718 --- Diff: R/pkg/R/functions.R --- @@ -1906,6 +1906,7 @@ setMethod("atan2", signature(y = "Column"), #' @details #' \code{datediff}:

[GitHub] spark issue #21250: [SPARK-23291][SQL][R][BRANCH-2.3] R's substr should not ...

2018-05-07 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21250 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21028 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3002/

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21028 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19602 **[Test build #90323 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90323/testReport)** for PR 19602 at commit

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3014/

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21258 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21239: [SPARK-24040][SS] Support single partition aggregates in...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21239 **[Test build #90328 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90328/testReport)** for PR 21239 at commit

[GitHub] spark issue #21259: [SPARK-24112][SQL] Add `convertMetastoreTableProperty` c...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21259 **[Test build #90331 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90331/testReport)** for PR 21259 at commit

[GitHub] spark issue #21259: [SPARK-24112][SQL] Add `convertMetastoreTableProperty` c...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21259 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90331/ Test PASSed. ---

[GitHub] spark pull request #21261: Add spark.executor.bindAddress

2018-05-07 Thread lukmajercak
GitHub user lukmajercak opened a pull request: https://github.com/apache/spark/pull/21261 Add spark.executor.bindAddress ## What changes were proposed in this pull request? Adding spark.executor.bindAddress to allow overriding the address to which the RPC server will bind to on

[GitHub] spark issue #21253: [SPARK-24158][SS] Enable no-data batches for streaming j...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21253 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21253: [SPARK-24158][SS] Enable no-data batches for streaming j...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21253 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3017/

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21073 **[Test build #90336 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90336/testReport)** for PR 21073 at commit

[GitHub] spark issue #21258: [SPARK-23933][SQL] Add map_fromarray function

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21258 **[Test build #90338 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90338/testReport)** for PR 21258 at commit

[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21242 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21242 **[Test build #90337 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90337/testReport)** for PR 21242 at commit

[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21242 **[Test build #90339 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90339/testReport)** for PR 21242 at commit

[GitHub] spark issue #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10.0.

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21070 **[Test build #90327 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90327/testReport)** for PR 21070 at commit

[GitHub] spark pull request #21195: [Spark-23975][ML] Add support of array input for ...

2018-05-07 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21195#discussion_r186556808 --- Diff: mllib/src/test/scala/org/apache/spark/ml/util/MLTestingUtils.scala --- @@ -247,4 +247,21 @@ object MLTestingUtils extends SparkFunSuite {

[GitHub] spark pull request #21195: [Spark-23975][ML] Add support of array input for ...

2018-05-07 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21195#discussion_r186556119 --- Diff: mllib/src/test/scala/org/apache/spark/ml/util/MLTestingUtils.scala --- @@ -247,4 +247,21 @@ object MLTestingUtils extends SparkFunSuite {

[GitHub] spark pull request #21195: [Spark-23975][ML] Add support of array input for ...

2018-05-07 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21195#discussion_r186555908 --- Diff: mllib/src/test/scala/org/apache/spark/ml/clustering/LDASuite.scala --- @@ -323,4 +324,21 @@ class LDASuite extends SparkFunSuite with

[GitHub] spark pull request #21195: [Spark-23975][ML] Add support of array input for ...

2018-05-07 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21195#discussion_r186555425 --- Diff: mllib/src/test/scala/org/apache/spark/ml/clustering/GaussianMixtureSuite.scala --- @@ -256,6 +257,22 @@ class GaussianMixtureSuite extends

[GitHub] spark pull request #21195: [Spark-23975][ML] Add support of array input for ...

2018-05-07 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21195#discussion_r186556798 --- Diff: mllib/src/test/scala/org/apache/spark/ml/util/MLTestingUtils.scala --- @@ -247,4 +247,21 @@ object MLTestingUtils extends SparkFunSuite {

[GitHub] spark issue #21195: [Spark-23975][ML] Add support of array input for all clu...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21195 **[Test build #90344 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90344/testReport)** for PR 21195 at commit

[GitHub] spark issue #21195: [Spark-23975][ML] Add support of array input for all clu...

2018-05-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21195 **[Test build #90344 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90344/testReport)** for PR 21195 at commit

[GitHub] spark issue #21195: [Spark-23975][ML] Add support of array input for all clu...

2018-05-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21195 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90344/ Test PASSed. ---

[GitHub] spark pull request #20929: [SPARK-23772][SQL][WIP] Provide an option to igno...

2018-05-07 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20929#discussion_r186581349 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -887,6 +887,14 @@ object SQLConf { .booleanConf

<    1   2   3   4   5   6   7   >