[GitHub] spark pull request #21895: [SPARK-24948][SHS] Delegate check access permissi...

2018-08-02 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21895#discussion_r207133206 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -973,6 +985,38 @@ private[history] object FsHistoryProvider {

[GitHub] spark issue #21895: [SPARK-24948][SHS] Delegate check access permissions to ...

2018-08-02 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21895 @mridulm would you please also take a review. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For a

[GitHub] spark issue #21952: [SPARK-24993] [SQL] [WIP] Make Avro Fast Again

2018-08-02 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21952 @cloud-fan as you suggested, I benchmarked cache read performance, and the performance is the same. This makes sense, since it's unlikely that cache read performance is that bad so we can see the imp

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21102 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1637/

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21102 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21102 **[Test build #93979 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93979/testReport)** for PR 21102 at commit [`c6b9a41`](https://github.com/apache/spark/commit/c6

[GitHub] spark issue #21953: [SPARK-24992][Core] spark should randomize yarn local di...

2018-08-02 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21953 What kind of behavior did you see? This local dir is only used to store some temporary files, which is not IO intensive, so I don't think the problem here is severe. --- ---

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21102 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21102 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93979/ Test FAILed. ---

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21102 **[Test build #93979 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93979/testReport)** for PR 21102 at commit [`c6b9a41`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #21754: [SPARK-24705][SQL] ExchangeCoordinator broken when dupli...

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21754 yea, I think that is another approach to fix this issue. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For ad

[GitHub] spark pull request #21895: [SPARK-24948][SHS] Delegate check access permissi...

2018-08-02 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21895#discussion_r207136738 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -80,8 +80,8 @@ import org.apache.spark.util.kvstore._ *

[GitHub] spark pull request #21895: [SPARK-24948][SHS] Delegate check access permissi...

2018-08-02 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21895#discussion_r207136973 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -461,32 +462,37 @@ private[history] class FsHistoryProvider(co

[GitHub] spark issue #21754: [SPARK-24705][SQL] ExchangeCoordinator broken when dupli...

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21754 How about the fix based on [the suggestion](https://github.com/apache/spark/pull/21754/commits/f961760f64ceabd582bd78fd2b383f1405988816)? --- ---

[GitHub] spark issue #21754: [SPARK-24705][SQL] ExchangeCoordinator broken when dupli...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21754 **[Test build #93980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93980/testReport)** for PR 21754 at commit [`f961760`](https://github.com/apache/spark/commit/f9

[GitHub] spark pull request #21895: [SPARK-24948][SHS] Delegate check access permissi...

2018-08-02 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21895#discussion_r207138029 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -973,6 +985,38 @@ private[history] object FsHistoryProvider {

[GitHub] spark issue #21754: [SPARK-24705][SQL] ExchangeCoordinator broken when dupli...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21754 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1638/

[GitHub] spark issue #21754: [SPARK-24705][SQL] ExchangeCoordinator broken when dupli...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21754 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21895: [SPARK-24948][SHS] Delegate check access permissi...

2018-08-02 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21895#discussion_r207140685 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -973,6 +985,38 @@ private[history] object FsHistoryProvider {

[GitHub] spark pull request #21954: [SPARK-23908][SQL] Add transform function.

2018-08-02 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21954#discussion_r207141916 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -0,0 +1,212 @@ +/* + * License

[GitHub] spark issue #21754: [SPARK-24705][SQL] ExchangeCoordinator broken when dupli...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21754 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21754: [SPARK-24705][SQL] ExchangeCoordinator broken when dupli...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21754 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1639/

[GitHub] spark issue #21754: [SPARK-24705][SQL] ExchangeCoordinator broken when dupli...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21754 **[Test build #93981 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93981/testReport)** for PR 21754 at commit [`7f98b88`](https://github.com/apache/spark/commit/7f

[GitHub] spark pull request #21954: [SPARK-23908][SQL] Add transform function.

2018-08-02 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21954#discussion_r207143138 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -0,0 +1,212 @@ +/* + * Licensed to

[GitHub] spark pull request #21954: [SPARK-23908][SQL] Add transform function.

2018-08-02 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21954#discussion_r207145478 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -0,0 +1,212 @@ +/* + * License

[GitHub] spark pull request #21895: [SPARK-24948][SHS] Delegate check access permissi...

2018-08-02 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21895#discussion_r207146314 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -973,6 +985,38 @@ private[history] object FsHistoryProvider {

[GitHub] spark pull request #21949: [SPARK-24957][SQL][BACKPORT-2.2] Average with dec...

2018-08-02 Thread mgaido91
Github user mgaido91 closed the pull request at: https://github.com/apache/spark/pull/21949 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21949: [SPARK-24957][SQL][BACKPORT-2.2] Average with decimal fo...

2018-08-02 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21949 Thanks, closed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark pull request #21892: [SPARK-24945][SQL] Switching to uniVocity 2.7.2

2018-08-02 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21892#discussion_r207146710 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala --- @@ -203,19 +203,11 @@ class UnivocityParser(

[GitHub] spark issue #21919: [SPARK-24933][SS] Report numOutputRows in SinkProgress v...

2018-08-02 Thread vackosar
Github user vackosar commented on the issue: https://github.com/apache/spark/pull/21919 @jose-torres thx for good point. The reason for placing this into WriterCommitMessage is to set a standard information that should passed at the commit time. But I agree that row counting

[GitHub] spark pull request #21954: [SPARK-23908][SQL] Add transform function.

2018-08-02 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21954#discussion_r207148555 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -0,0 +1,212 @@ +/* + * Licensed to

[GitHub] spark pull request #21236: [SPARK-23935][SQL] Adding map_entries function

2018-08-02 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/21236#discussion_r207148777 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -98,6 +98,9 @@ trait ExpressionEvalHelpe

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-08-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19449 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19449 **[Test build #93982 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93982/testReport)** for PR 19449 at commit [`253bc19`](https://github.com/apache/spark/commit/25

[GitHub] spark pull request #21965: [WIP][SPARK-23909][SQL] Add filter function.

2018-08-02 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/21965 [WIP][SPARK-23909][SQL] Add filter function. ## What changes were proposed in this pull request? This pr adds `filter` function which filters the input array using the given predicate.

[GitHub] spark issue #21965: [WIP][SPARK-23909][SQL] Add filter function.

2018-08-02 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21965 This is based on #21954. I'll update after it is merged. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For ad

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19449 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19449 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1641/

[GitHub] spark issue #21965: [WIP][SPARK-23909][SQL] Add filter function.

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21965 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21965: [WIP][SPARK-23909][SQL] Add filter function.

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21965 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1640/

[GitHub] spark issue #21965: [WIP][SPARK-23909][SQL] Add filter function.

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21965 **[Test build #93983 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93983/testReport)** for PR 21965 at commit [`fe48510`](https://github.com/apache/spark/commit/fe

[GitHub] spark issue #21087: [SPARK-23997][SQL] Configurable maximum number of bucket...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21087 **[Test build #93984 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93984/testReport)** for PR 21087 at commit [`aad1068`](https://github.com/apache/spark/commit/aa

[GitHub] spark pull request #21957: [SPARK-24994][SQL] When the data type of the fiel...

2018-08-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21957#discussion_r207155086 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -436,8 +436,9 @@ object DataSourceStrategy {

[GitHub] spark pull request #21944: [SPARK-24988][SQL]Add a castBySchema method which...

2018-08-02 Thread mahmoudmahdi24
Github user mahmoudmahdi24 commented on a diff in the pull request: https://github.com/apache/spark/pull/21944#discussion_r207156078 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1367,6 +1367,22 @@ class Dataset[T] private[sql]( }: _*)

[GitHub] spark issue #21935: [SPARK-24773] Avro: support logical timestamp type with ...

2018-08-02 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/21935 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #21941: [SPARK-24966][SQL] Implement precedence rules for set op...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21941 **[Test build #93966 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93966/testReport)** for PR 21941 at commit [`e7d69db`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #21941: [SPARK-24966][SQL] Implement precedence rules for set op...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21941 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21941: [SPARK-24966][SQL] Implement precedence rules for set op...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21941 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93966/ Test FAILed. ---

[GitHub] spark issue #21958: [minor] remove dead code in ExpressionEvalHelper

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21958 **[Test build #93976 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93976/testReport)** for PR 21958 at commit [`19dd614`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #21958: [minor] remove dead code in ExpressionEvalHelper

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21958 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93976/ Test FAILed. ---

[GitHub] spark issue #21958: [minor] remove dead code in ExpressionEvalHelper

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21958 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21935: [SPARK-24773] Avro: support logical timestamp type with ...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21935 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21935: [SPARK-24773] Avro: support logical timestamp type with ...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21935 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1642/

[GitHub] spark issue #21935: [SPARK-24773] Avro: support logical timestamp type with ...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21935 **[Test build #93985 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93985/testReport)** for PR 21935 at commit [`921e6cb`](https://github.com/apache/spark/commit/92

[GitHub] spark pull request #21954: [SPARK-23908][SQL] Add transform function.

2018-08-02 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21954#discussion_r207158636 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -0,0 +1,212 @@ +/* + * License

[GitHub] spark pull request #21935: [SPARK-24773] Avro: support logical timestamp typ...

2018-08-02 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/21935#discussion_r207159637 --- Diff: external/avro/src/test/scala/org/apache/spark/sql/avro/AvroSuite.scala --- @@ -41,6 +41,7 @@ import org.apache.spark.sql.types._ class

[GitHub] spark pull request #21966: simplify codegen of array_except

2018-08-02 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/21966 simplify codegen of array_except ## What changes were proposed in this pull request? simplify the codegen: 1. only do real codegen if the type can be specialized by the hash set 2

[GitHub] spark issue #21966: simplify codegen of array_except

2018-08-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21966 cc @kiszk @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark issue #21961: Spark 20597

2018-08-02 Thread mahmoudmahdi24
Github user mahmoudmahdi24 commented on the issue: https://github.com/apache/spark/pull/21961 Hello @Satyajitv, please rename the title of this PR properly. The PR title should be of the form [SPARK-][COMPONENT] Title, where SPARK- is the relevant JIRA number, COMPONENT is

[GitHub] spark pull request #21944: [SPARK-24988][SQL]Add a castBySchema method which...

2018-08-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21944#discussion_r207160479 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1367,6 +1367,22 @@ class Dataset[T] private[sql]( }: _*) }

[GitHub] spark issue #21966: [SPARK-23915][SQL][followup] Add array_except function

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21966 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21966: [SPARK-23915][SQL][followup] Add array_except function

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21966 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1643/

[GitHub] spark issue #21966: [SPARK-23915][SQL][followup] Add array_except function

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21966 **[Test build #93986 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93986/testReport)** for PR 21966 at commit [`313ad29`](https://github.com/apache/spark/commit/31

[GitHub] spark issue #21958: [minor] remove dead code in ExpressionEvalHelper

2018-08-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21958 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark pull request #21954: [SPARK-23908][SQL] Add transform function.

2018-08-02 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21954#discussion_r207162167 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -0,0 +1,212 @@ +/* + * Licensed to

[GitHub] spark pull request #21954: [SPARK-23908][SQL] Add transform function.

2018-08-02 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21954#discussion_r207162371 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -0,0 +1,212 @@ +/* + * Licensed to

[GitHub] spark issue #21958: [minor] remove dead code in ExpressionEvalHelper

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21958 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21958: [minor] remove dead code in ExpressionEvalHelper

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21958 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1644/

[GitHub] spark issue #21958: [minor] remove dead code in ExpressionEvalHelper

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21958 **[Test build #93987 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93987/testReport)** for PR 21958 at commit [`19dd614`](https://github.com/apache/spark/commit/19

[GitHub] spark issue #21733: [SPARK-24763][SS] Remove redundant key data from value i...

2018-08-02 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21733 @tdas I found the spare time to run performance tests though I've run only one app for now... I couldn't run the tests concurrently. Please let me know if you are not confident with th

[GitHub] spark pull request #21944: [SPARK-24988][SQL]Add a castBySchema method which...

2018-08-02 Thread mahmoudmahdi24
Github user mahmoudmahdi24 commented on a diff in the pull request: https://github.com/apache/spark/pull/21944#discussion_r207164547 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1367,6 +1367,22 @@ class Dataset[T] private[sql]( }: _*)

[GitHub] spark pull request #21754: [SPARK-24705][SQL] ExchangeCoordinator broken whe...

2018-08-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21754#discussion_r207164799 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ExchangeCoordinator.scala --- @@ -83,16 +83,17 @@ import org.apache.spark.sql.e

[GitHub] spark issue #21962: [SPARK-24865][FOLLOW-UP] Remove AnalysisBarrier LogicalP...

2018-08-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21962 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark issue #21962: [SPARK-24865][FOLLOW-UP] Remove AnalysisBarrier LogicalP...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21962 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1645/

[GitHub] spark issue #21962: [SPARK-24865][FOLLOW-UP] Remove AnalysisBarrier LogicalP...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21962 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21962: [SPARK-24865][FOLLOW-UP] Remove AnalysisBarrier LogicalP...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21962 **[Test build #93988 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93988/testReport)** for PR 21962 at commit [`0135ba4`](https://github.com/apache/spark/commit/01

[GitHub] spark issue #21599: [SPARK-24598][SQL] Overflow on arithmetic operations ret...

2018-08-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21599 I don't think we can change behavior like this without introducing a config. In Spark 3.0 we will add a "strict mode" and closely follow SQL standard. But for Spark 2.4, how about we keep the "jav

[GitHub] spark pull request #21954: [SPARK-23908][SQL] Add transform function.

2018-08-02 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21954#discussion_r207169967 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -0,0 +1,212 @@ +/* + * Licensed to

[GitHub] spark pull request #21954: [SPARK-23908][SQL] Add transform function.

2018-08-02 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/21954#discussion_r207171941 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -0,0 +1,212 @@ +/* + * License

[GitHub] spark issue #21754: [SPARK-24705][SQL] ExchangeCoordinator broken when dupli...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21754 **[Test build #93967 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93967/testReport)** for PR 21754 at commit [`544e917`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #21754: [SPARK-24705][SQL] ExchangeCoordinator broken when dupli...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21754 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21754: [SPARK-24705][SQL] ExchangeCoordinator broken when dupli...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21754 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93967/ Test FAILed. ---

[GitHub] spark pull request #21954: [SPARK-23908][SQL] Add transform function.

2018-08-02 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21954#discussion_r207172497 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -0,0 +1,212 @@ +/* + * Licensed to

[GitHub] spark issue #21599: [SPARK-24598][SQL] Overflow on arithmetic operations ret...

2018-08-02 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21599 @cloud-fan I think this is just a wrong behavior (being against SQL standard is not the only issue), so I am not sure that makes sense to have it controlled by a config. Anyway, I agree that as it

[GitHub] spark issue #21941: [SPARK-24966][SQL] Implement precedence rules for set op...

2018-08-02 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/21941 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #21941: [SPARK-24966][SQL] Implement precedence rules for set op...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21941 **[Test build #93989 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93989/testReport)** for PR 21941 at commit [`e7d69db`](https://github.com/apache/spark/commit/e7

[GitHub] spark issue #21859: [SPARK-24900][SQL]speed up sort when the dataset is smal...

2018-08-02 Thread sddyljsx
Github user sddyljsx commented on the issue: https://github.com/apache/spark/pull/21859 @felixcheung Thanks for review. **1. How small is 'small':** This optimazition works when the sampled data of the RangePartitioner covers all the data to sort. Th

[GitHub] spark issue #21957: [SPARK-24994][SQL] When the data type of the field is co...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21957 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21941: [SPARK-24966][SQL] Implement precedence rules for set op...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21941 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21941: [SPARK-24966][SQL] Implement precedence rules for set op...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21941 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1647/

[GitHub] spark issue #21957: [SPARK-24994][SQL] When the data type of the field is co...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21957 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1646/

[GitHub] spark issue #21895: [SPARK-24948][SHS] Delegate check access permissions to ...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21895 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1648/

[GitHub] spark issue #21895: [SPARK-24948][SHS] Delegate check access permissions to ...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21895 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21957: [SPARK-24994][SQL] When the data type of the field is co...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21957 **[Test build #93990 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93990/testReport)** for PR 21957 at commit [`24c061f`](https://github.com/apache/spark/commit/24

[GitHub] spark issue #21895: [SPARK-24948][SHS] Delegate check access permissions to ...

2018-08-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21895 **[Test build #93991 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93991/testReport)** for PR 21895 at commit [`c620fff`](https://github.com/apache/spark/commit/c6

[GitHub] spark pull request #21754: [SPARK-24705][SQL] ExchangeCoordinator broken whe...

2018-08-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21754#discussion_r207178639 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ExchangeCoordinator.scala --- @@ -83,16 +83,17 @@ import org.apache.spark.sql.exec

[GitHub] spark issue #21967: [SPARK-24598][DOCS] State in the documentation the behav...

2018-08-02 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21967 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@

[GitHub] spark pull request #21967: [SPARK-24598][DOCS] State in the documentation th...

2018-08-02 Thread mgaido91
GitHub user mgaido91 opened a pull request: https://github.com/apache/spark/pull/21967 [SPARK-24598][DOCS] State in the documentation the behavior when arithmetic operations cause overflow ## What changes were proposed in this pull request? According to the discussion in ht

[GitHub] spark issue #21754: [SPARK-24705][SQL] ExchangeCoordinator broken when dupli...

2018-08-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21754 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h..

[GitHub] spark issue #21967: [SPARK-24598][DOCS] State in the documentation the behav...

2018-08-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21967 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

<    5   6   7   8   9   10   11   12   13   >