[GitHub] spark pull request #21269: [SPARK-23355][SQL][DOC][FOLLOWUP] Add migration d...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21269#discussion_r186782118 --- Diff: docs/sql-programming-guide.md --- @@ -1812,6 +1812,8 @@ working with timestamps in `pandas_udf`s to get the best performance, see - Since

[GitHub] spark issue #21269: [SPARK-23355][SQL][DOC][FOLLOWUP] Add migration doc for ...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21269 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apa

[GitHub] spark issue #21259: [SPARK-24112][SQL] Add `convertMetastoreTableProperty` c...

2018-05-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21259 Thank you all. SPARK-24112 is closed and the doc issue goes as [FOLLOWUP PR for SPARK-23355](https://github.com/apache/spark/pull/21269). ---

[GitHub] spark pull request #21259: [SPARK-24112][SQL] Add `convertMetastoreTableProp...

2018-05-08 Thread dongjoon-hyun
Github user dongjoon-hyun closed the pull request at: https://github.com/apache/spark/pull/21259 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.

[GitHub] spark issue #21269: [SPARK-23355][SQL][DOC][FOLLOWUP] Add migration doc for ...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21269 **[Test build #90372 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90372/testReport)** for PR 21269 at commit [`1647961`](https://github.com/apache/spark/commit/16

[GitHub] spark pull request #21269: [SPARK-23355][SQL][DOC][FOLLOWUP] Add migration d...

2018-05-08 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/21269 [SPARK-23355][SQL][DOC][FOLLOWUP] Add migration doc for TBLPROPERTIES ## What changes were proposed in this pull request? In Apache Spark 2.4, [SPARK-23355](https://issues.apache.org

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21028 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90370/ Test FAILed. ---

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21028 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21028 **[Test build #90370 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90370/testReport)** for PR 21028 at commit [`964f7af`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #19498: [SPARK-17756][PYTHON][STREAMING] Workaround to avoid ret...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19498 **[Test build #90371 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90371/testReport)** for PR 19498 at commit [`174ec21`](https://github.com/apache/spark/commit/17

[GitHub] spark pull request #21189: [SPARK-24117][SQL] Unified the getSizePerRow

2018-05-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21189 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21252: [SPARK-24193] Sort by disk when number of limit is big i...

2018-05-08 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/21252 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revie

[GitHub] spark issue #21118: SPARK-23325: Use InternalRow when reading with DataSourc...

2018-05-08 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21118 > If we want to go this way, I think we should fully bring back #10511 to make this contract explicitly, i.e. which operator produce unsafe row and which operator only accepts unsafe row as input.

[GitHub] spark issue #21189: [SPARK-24117][SQL] Unified the getSizePerRow

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21189 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21212: [SPARK-24143] filter empty blocks when convert mapstatus...

2018-05-08 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/21212 @jinxing64 > I guess your concern is ArrayBuffer will do lots of copy as size of elements grows, and we don't need fast random access in ShuffleBlockFetcherIterator my concern wasn't th

[GitHub] spark pull request #21199: [SPARK-24127][SS] Continuous text socket source

2018-05-08 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21199#discussion_r186764630 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousTextSocketSource.scala --- @@ -0,0 +1,304 @@ +/*

[GitHub] spark pull request #21199: [SPARK-24127][SS] Continuous text socket source

2018-05-08 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21199#discussion_r186765402 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousTextSocketSource.scala --- @@ -0,0 +1,304 @@ +/*

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r186772705 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -207,9 +207,25 @@ case class

[GitHub] spark pull request #21185: [SPARK-23894][CORE][SQL] Defensively clear Active...

2018-05-08 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/21185#discussion_r186772234 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -229,6 +229,23 @@ private[spark] class Executor( ManagementFactory.get

[GitHub] spark pull request #21185: [SPARK-23894][CORE][SQL] Defensively clear Active...

2018-05-08 Thread squito
Github user squito closed the pull request at: https://github.com/apache/spark/pull/21185 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21185: [SPARK-23894][CORE][SQL] Defensively clear ActiveSession...

2018-05-08 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/21185 I'm closing this in favor of https://github.com/apache/spark/pull/21190 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.ap

[GitHub] spark issue #21230: [SPARK-24172][SQL] we should not apply operator pushdown...

2018-05-08 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21230 Sounds good to me. Lets plan on getting this one in to fix the current problem, and commit the other approach when stats are fixed. --- -

[GitHub] spark issue #21252: [SPARK-24193] Sort by disk when number of limit is big i...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21252 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90368/ Test FAILed. ---

[GitHub] spark issue #21252: [SPARK-24193] Sort by disk when number of limit is big i...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21252 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21252: [SPARK-24193] Sort by disk when number of limit is big i...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21252 **[Test build #90368 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90368/testReport)** for PR 21252 at commit [`4a8a061`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #21252: [SPARK-24193] Sort by disk when number of limit i...

2018-05-08 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21252#discussion_r186739581 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1238,6 +1238,14 @@ object SQLConf { .booleanConf

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21028 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3041/ Tes

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21028 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21252: [SPARK-24193] Sort by disk when number of limit i...

2018-05-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21252#discussion_r186736697 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1238,6 +1238,14 @@ object SQLConf { .booleanConf

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21028 **[Test build #90370 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90370/testReport)** for PR 21028 at commit [`964f7af`](https://github.com/apache/spark/commit/96

[GitHub] spark issue #21268: [SPARK-24209][SHS] Automatic retrieve proxyBase from Kno...

2018-05-08 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21268 @nrchakradhar if the load balancer fills the header `X-Forwarded-Context` with the proper base path, then this will work. --- -

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-08 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r186728795 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -28,6 +30,34 @@ import org.apache.spa

[GitHub] spark issue #21268: [SPARK-24209][SHS] Automatic retrieve proxyBase from Kno...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21268 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3040/ Tes

[GitHub] spark issue #21268: [SPARK-24209][SHS] Automatic retrieve proxyBase from Kno...

2018-05-08 Thread nrchakradhar
Github user nrchakradhar commented on the issue: https://github.com/apache/spark/pull/21268 @mgaido91 Just a query. Is this specific to Knox only or will it work when the HistoryServer UI is proxied by different load balancer? ---

[GitHub] spark issue #21268: [SPARK-24209][SHS] Automatic retrieve proxyBase from Kno...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21268 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21211: [SPARK-24131][PYSPARK][Followup] Add majorMinorVersion A...

2018-05-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21211 Thanks @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21268: [SPARK-24209][SHS] Automatic retrieve proxyBase from Kno...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21268 **[Test build #90369 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90369/testReport)** for PR 21268 at commit [`c7fafd6`](https://github.com/apache/spark/commit/c7

[GitHub] spark pull request #21254: [SPARK-23094][SPARK-23723][SPARK-23724][SQL][FOLL...

2018-05-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21254 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21211: [SPARK-24131][PYSPARK][Followup] Add majorMinorVe...

2018-05-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21211 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20787: [MINOR][DOCS] Documenting months_between directio...

2018-05-08 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20787#discussion_r186725257 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -1194,13 +1194,21 @@ case class AddMon

[GitHub] spark issue #21254: [SPARK-23094][SPARK-23723][SPARK-23724][SQL][FOLLOW-UP] ...

2018-05-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21254 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #21211: [SPARK-24131][PYSPARK][Followup] Add majorMinorVersion A...

2018-05-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21211 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #21189: [SPARK-24117][SQL] Unified the getSizePerRow

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21189 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21189: [SPARK-24117][SQL] Unified the getSizePerRow

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21189 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90365/ Test PASSed. ---

[GitHub] spark issue #21189: [SPARK-24117][SQL] Unified the getSizePerRow

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21189 **[Test build #90365 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90365/testReport)** for PR 21189 at commit [`f72084e`](https://github.com/apache/spark/commit/f

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-05-08 Thread zheh12
Github user zheh12 commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r186719888 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -207,9 +207,25 @@ case class Ins

[GitHub] spark issue #21252: [SPARK-24193] Sort by disk when number of limit is big i...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21252 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3039/ Tes

[GitHub] spark issue #21252: [SPARK-24193] Sort by disk when number of limit is big i...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21252 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21252: [SPARK-24193] Sort by disk when number of limit is big i...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21252 **[Test build #90368 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90368/testReport)** for PR 21252 at commit [`4a8a061`](https://github.com/apache/spark/commit/4a

[GitHub] spark issue #21268: [SPARK-24209][SHS] Automatic retrieve proxyBase from Kno...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21268 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21268: [SPARK-24209][SHS] Automatic retrieve proxyBase from Kno...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21268 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90366/ Test FAILed. ---

[GitHub] spark issue #21268: [SPARK-24209][SHS] Automatic retrieve proxyBase from Kno...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21268 **[Test build #90366 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90366/testReport)** for PR 21268 at commit [`0af9c2a`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #20936: [SPARK-23503][SS] Enforce sequencing of committed epochs...

2018-05-08 Thread efimpoberezkin
Github user efimpoberezkin commented on the issue: https://github.com/apache/spark/pull/20936 @tdas Hi, seems like it's good to go. Could you merge this when you have time? --- - To unsubscribe, e-mail: reviews-unsu

[GitHub] spark issue #21268: [SPARK-24209][SHS] Automatic retrieve proxyBase from Kno...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21268 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3038/ Tes

[GitHub] spark issue #21268: [SPARK-24209][SHS] Automatic retrieve proxyBase from Kno...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21268 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21236: [SPARK-23935][SQL] Adding map_entries function

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21236 **[Test build #90367 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90367/testReport)** for PR 21236 at commit [`6aa90ef`](https://github.com/apache/spark/commit/6a

[GitHub] spark issue #21268: [SPARK-24209][SHS] Automatic retrieve proxyBase from Kno...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21268 **[Test build #90366 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90366/testReport)** for PR 21268 at commit [`0af9c2a`](https://github.com/apache/spark/commit/0a

[GitHub] spark issue #21268: [SPARK-24209][SHS] Automatic retrieve proxyBase from Kno...

2018-05-08 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21268 @jerryshao may you please review this when you have time? Thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache

[GitHub] spark pull request #21268: [SPARK-24209][SHS] Automatic retrieve proxyBase f...

2018-05-08 Thread mgaido91
GitHub user mgaido91 opened a pull request: https://github.com/apache/spark/pull/21268 [SPARK-24209][SHS] Automatic retrieve proxyBase from Knox headers ## What changes were proposed in this pull request? The PR retrieves the proxyBase automatically from the header `X-Forwa

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-08 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r186710034 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -28,6 +30,34 @@ import org.apache.spark.

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r186707079 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -207,9 +207,25 @@ case class

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r186706829 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -28,6 +30,34 @@ import org.apache.spark

[GitHub] spark pull request #21106: [SPARK-23711][SQL][WIP] Add fallback logic for Un...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21106#discussion_r186706306 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Projection.scala --- @@ -108,7 +108,31 @@ abstract class UnsafeProjection

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21129 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21129 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90360/ Test PASSed. ---

[GitHub] spark issue #21129: [SPARK-7132][ML] Add fit with validation set to spark.ml...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21129 **[Test build #90360 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90360/testReport)** for PR 21129 at commit [`54f73af`](https://github.com/apache/spark/commit/5

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-08 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r186690973 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -28,6 +30,34 @@ import org.apache.spark.

[GitHub] spark issue #21149: [SPARK-24076][SQL] Use different seed in HashAggregate t...

2018-05-08 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/21149 @maropu @hvanhovell thanks very much! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r186677863 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -100,17 +101,18 @@ abstract class Expression extends

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r186681857 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala --- @@ -623,8 +624,14 @@ case class Cast(child: Expression, dataT

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r186683015 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -56,19 +57,19 @@ import org.apache.spark.

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r186674665 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala --- @@ -623,8 +624,14 @@ case class Cast(child: Expression, dataT

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r186679356 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -119,7 +121,7 @@ abstract class Expression extends T

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r186679287 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -114,6 +114,113 @@ object JavaCode { }

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r186679018 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -114,6 +114,113 @@ object JavaCode { }

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90362/ Test FAILed. ---

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21193 **[Test build #90362 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90362/testReport)** for PR 21193 at commit [`53b329a`](https://github.com/apache/spark/commit/5

[GitHub] spark pull request #21231: [SPARK-24119][SQL]Add interpreted execution to So...

2018-05-08 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21231#discussion_r186673964 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala --- @@ -147,7 +148,44 @@ case class SortPrefix(child: So

[GitHub] spark pull request #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work ...

2018-05-08 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21267#discussion_r186673789 --- Diff: python/pyspark/context.py --- @@ -211,9 +211,23 @@ def _do_init(self, master, appName, sparkHome, pyFiles, environment, batchSize,

[GitHub] spark pull request #21231: [SPARK-24119][SQL]Add interpreted execution to So...

2018-05-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21231#discussion_r186672482 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala --- @@ -147,7 +148,44 @@ case class SortPrefix(child: SortOr

[GitHub] spark pull request #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work ...

2018-05-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21267#discussion_r186670486 --- Diff: python/pyspark/context.py --- @@ -211,9 +211,23 @@ def _do_init(self, master, appName, sparkHome, pyFiles, environment, batchSize, for

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-08 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r186670840 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -114,6 +114,113 @@ object JavaCode {

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-08 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r186670675 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -114,6 +114,113 @@ object JavaCode {

[GitHub] spark pull request #21149: [SPARK-24076][SQL] Use different seed in HashAggr...

2018-05-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21149 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21193: [SPARK-24121][SQL] Add API for handling expressio...

2018-05-08 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21193#discussion_r186669860 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -119,7 +121,7 @@ abstract class Expression exten

[GitHub] spark issue #20611: [SPARK-23425][SQL]Support wildcard in HDFS path for load...

2018-05-08 Thread sujith71955
Github user sujith71955 commented on the issue: https://github.com/apache/spark/pull/20611 Currently all the load command use-cases with wildcard in a load path are blocked for the users --- - To unsubscribe, e-mai

[GitHub] spark issue #21149: [SPARK-24076][SQL] Use different seed in HashAggregate t...

2018-05-08 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/21149 LGTM - merging to master. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21189: [SPARK-24117][SQL] Unified the getSizePerRow

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21189 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21095: [SPARK-23529][K8s] Support mounting hostPath volumes

2018-05-08 Thread andrusha
Github user andrusha commented on the issue: https://github.com/apache/spark/pull/21095 @liyinan926 that's the idea. We should have a single step to mount all kinds of volumes including hostPath and emptyDir. The only problem is the configuration awkwardness. --- --

[GitHub] spark pull request #21231: [SPARK-24119][SQL]Add interpreted execution to So...

2018-05-08 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21231#discussion_r186668643 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala --- @@ -147,7 +148,44 @@ case class SortPrefix(child: So

[GitHub] spark issue #21189: [SPARK-24117][SQL] Unified the getSizePerRow

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21189 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3037/ Tes

[GitHub] spark issue #20611: [SPARK-23425][SQL]Support wildcard in HDFS path for load...

2018-05-08 Thread sujith71955
Github user sujith71955 commented on the issue: https://github.com/apache/spark/pull/20611 @wzhfy @HyukjinKwon @jiangxb1987 @gatorsmile @vinodkc, Please let me know for any suggestions regarding this PR. I covered all scenarios related to this PR. Working fine. ---

[GitHub] spark issue #21231: [SPARK-24119][SQL]Add interpreted execution to SortPrefi...

2018-05-08 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21231 LGTM cc: @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark pull request #21231: [SPARK-24119][SQL]Add interpreted execution to So...

2018-05-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21231#discussion_r186667062 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/SortOrderExpressionsSuite.scala --- @@ -0,0 +1,96 @@ +/* + * License

[GitHub] spark issue #21149: [SPARK-24076][SQL] Use different seed in HashAggregate t...

2018-05-08 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21149 ping @hvanhovell @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21189: [SPARK-24117][SQL] Unified the getSizePerRow

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21189 **[Test build #90365 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90365/testReport)** for PR 21189 at commit [`f72084e`](https://github.com/apache/spark/commit/f7

[GitHub] spark pull request #21240: [SPARK-21274][SQL] Add a new generator function r...

2018-05-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21240#discussion_r186663637 --- Diff: sql/core/src/test/resources/sql-tests/inputs/udtf_replicate_rows.sql --- @@ -0,0 +1,41 @@ +CREATE TEMPORARY VIEW tab1 AS SELECT * FROM VALUES

[GitHub] spark pull request #21240: [SPARK-21274][SQL] Add a new generator function r...

2018-05-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21240#discussion_r186663298 --- Diff: sql/core/src/test/resources/sql-tests/inputs/udtf_replicate_rows.sql --- @@ -0,0 +1,41 @@ +CREATE TEMPORARY VIEW tab1 AS SELECT * FROM VALUES

[GitHub] spark pull request #21240: [SPARK-21274][SQL] Add a new generator function r...

2018-05-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21240#discussion_r186662955 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -222,6 +222,54 @@ case class Stack(children: Seq[Exp

<    1   2   3   4   5   >