[GitHub] spark pull request #20616: [SPARK-23434][SQL] Spark should not warn `metadat...

2018-02-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20616#discussion_r169236998 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSink.scala --- @@ -42,9 +42,13 @@ object FileStreamSink extends

[GitHub] spark pull request #20624: [SPARK-23445] ColumnStat refactoring

2018-02-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20624#discussion_r169235761 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/AggregateEstimation.scala --- @@ -32,13 +32,18 @@

[GitHub] spark pull request #20624: [SPARK-23445] ColumnStat refactoring

2018-02-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20624#discussion_r169235322 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/Statistics.scala --- @@ -305,15 +260,15 @@ object ColumnStat extends

[GitHub] spark pull request #20624: [SPARK-23445] ColumnStat refactoring

2018-02-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20624#discussion_r169234962 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -387,6 +390,101 @@ case class CatalogStatistics(

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2018-02-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19788 The idea LGTM, but I think @JoshRosen has a valid concern. My 2 cents: 1. The concept of reading multiple reducer partitions in one shot was introduced by `ShuffleManager.getReader`. Although

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20637 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87549/ Test PASSed. ---

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20637 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-02-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20637 **[Test build #87549 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87549/testReport)** for PR 20637 at commit

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2018-02-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r169229650 --- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala --- @@ -81,16 +81,17 @@ case object Resubmitted extends TaskFailedReason { */

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2018-02-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r169229539 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockId.scala --- @@ -51,11 +51,25 @@ case class RDDBlockId(rddId: Int, splitIndex: Int) extends

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2018-02-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r169229060 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -280,6 +281,16 @@ private[spark] abstract class MapOutputTracker(conf:

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2018-02-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r169228298 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolver.java --- @@ -157,21 +157,34 @@ public void

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2018-02-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r169227734 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockHandler.java --- @@ -196,42 +196,51 @@ private

[GitHub] spark issue #20616: [SPARK-23434][SQL] Spark should not warn `metadata direc...

2018-02-19 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20616 Thank you for review, @cloud-fan . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20616: [SPARK-23434][SQL] Spark should not warn `metadata direc...

2018-02-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20616 LGTM, cc @zsxwing --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20612: [SPARK-23424][SQL]Add codegenStageId in comment

2018-02-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20612 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20612: [SPARK-23424][SQL]Add codegenStageId in comment

2018-02-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20612#discussion_r169224985 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1226,14 +1226,24 @@ class

[GitHub] spark issue #20043: [SPARK-22856][SQL] Add wrappers for codegen output and n...

2018-02-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20043 Spark 2.3 should be ready soon, let's resume this PR :) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-19 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20619 Thank you all! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20043: [SPARK-22856][SQL] Add wrappers for codegen outpu...

2018-02-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20043#discussion_r169223825 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/BoundAttribute.scala --- @@ -75,7 +75,7 @@ case class

[GitHub] spark pull request #20043: [SPARK-22856][SQL] Add wrappers for codegen outpu...

2018-02-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20043#discussion_r169223570 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/BoundAttribute.scala --- @@ -75,7 +75,7 @@ case class

[GitHub] spark pull request #20621: [SPARK-23436][SQL] Infer partition as Date only i...

2018-02-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20621 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20621: [SPARK-23436][SQL] Infer partition as Date only if it ca...

2018-02-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20621 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20619: [SPARK-23457][SQL] Register task completion liste...

2018-02-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20619 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20619: [SPARK-23457][SQL] Register task completion listeners fi...

2018-02-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20619 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169213260 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -521,7 +542,15 @@ case class

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169213138 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -262,6 +273,11 @@ case class Cos(child:

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169213439 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -1313,131 +1313,165 @@ object functions {

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20637 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/963/

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20637 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-02-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20637 **[Test build #87549 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87549/testReport)** for PR 20637 at commit

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-02-19 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20637 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20637 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20637 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87548/ Test FAILed. ---

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-02-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20637 **[Test build #87548 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87548/testReport)** for PR 20637 at commit

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20618 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20618 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87547/ Test PASSed. ---

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-02-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20618 **[Test build #87547 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87547/testReport)** for PR 20618 at commit

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20618 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87546/ Test PASSed. ---

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20618 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-02-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20618 **[Test build #87546 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87546/testReport)** for PR 20618 at commit

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-02-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20637 **[Test build #87548 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87548/testReport)** for PR 20637 at commit

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20637 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/962/

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20637 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-02-19 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20637 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20632: [SPARK-3159] added subtree pruning in the translation fr...

2018-02-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20632 **[Test build #4100 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4100/testReport)** for PR 20632 at commit

[GitHub] spark issue #20632: [SPARK-3159] added subtree pruning in the translation fr...

2018-02-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20632 **[Test build #4100 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4100/testReport)** for PR 20632 at commit

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-02-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20618 **[Test build #87547 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87547/testReport)** for PR 20618 at commit

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread misutoth
Github user misutoth commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169187041 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -804,7 +858,6 @@ case class Pow(left:

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread misutoth
Github user misutoth commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169186706 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -538,8 +559,14 @@ case class Sinh(child:

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread misutoth
Github user misutoth commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169186651 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -512,16 +522,27 @@ case class Rint(child:

[GitHub] spark issue #20638: [SPARK-23417][python] Fix the build instructions supplie...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20638 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20638: [SPARK-23417][python] Fix the build instructions supplie...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20638 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87545/ Test PASSed. ---

[GitHub] spark issue #20638: [SPARK-23417][python] Fix the build instructions supplie...

2018-02-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20638 **[Test build #87545 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87545/testReport)** for PR 20638 at commit

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-02-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20618 **[Test build #87546 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87546/testReport)** for PR 20618 at commit

[GitHub] spark issue #20638: [SPARK-23417][python] Fix the build instructions supplie...

2018-02-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20638 **[Test build #87545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87545/testReport)** for PR 20638 at commit

[GitHub] spark issue #20638: [SPARK-23417][python] Fix the build instructions supplie...

2018-02-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20638 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20638: [SPARK-23417][python] Fix the build instructions supplie...

2018-02-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20638 add to whitelist --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-02-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20618 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169182568 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -804,7 +858,6 @@ case class Pow(left:

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169182146 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -512,16 +522,27 @@ case class Rint(child:

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169182499 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -562,7 +595,12 @@ case class Cot(child:

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169182420 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -548,7 +575,13 @@ case class Sqrt(child:

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169182396 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -538,8 +559,14 @@ case class Sinh(child:

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169182053 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -512,16 +522,27 @@ case class Rint(child:

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169182585 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -916,7 +969,6 @@ case class

[GitHub] spark issue #20637: Remove redundant null checks in generated Java code by G...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20637 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87544/ Test FAILed. ---

[GitHub] spark issue #20637: Remove redundant null checks in generated Java code by G...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20637 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20637: Remove redundant null checks in generated Java code by G...

2018-02-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20637 **[Test build #87544 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87544/testReport)** for PR 20637 at commit

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-19 Thread asolimando
Github user asolimando commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r169172406 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -402,20 +406,40 @@ class RandomForestSuite extends

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-19 Thread asolimando
Github user asolimando commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r169172053 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -287,6 +291,34 @@ private[tree] class LearningNode( } }

[GitHub] spark issue #20638: [SPARK-23417][python] Fix the build instructions supplie...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20638 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20638: [SPARK-23417][python] Fix the build instructions supplie...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20638 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20638: [SPARK-23417][python] Fix the build instructions ...

2018-02-19 Thread bersprockets
GitHub user bersprockets opened a pull request: https://github.com/apache/spark/pull/20638 [SPARK-23417][python] Fix the build instructions supplied by exception messages in python streaming tests ## What changes were proposed in this pull request? Fix the build

[GitHub] spark issue #20637: Remove redundant null checks in generated Java code by G...

2018-02-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20637 **[Test build #87544 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87544/testReport)** for PR 20637 at commit

[GitHub] spark issue #20637: Remove redundant null checks in generated Java code by G...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20637 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20637: Remove redundant null checks in generated Java code by G...

2018-02-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20637 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/961/

[GitHub] spark pull request #20627: [SPARK-23217][ML][PYTHON] Add distanceMeasure par...

2018-02-19 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/20627#discussion_r169156086 --- Diff: python/pyspark/ml/evaluation.py --- @@ -394,15 +397,30 @@ def getMetricName(self): @keyword_only @since("2.3.0")

[GitHub] spark pull request #20627: [SPARK-23217][ML][PYTHON] Add distanceMeasure par...

2018-02-19 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/20627#discussion_r169156415 --- Diff: python/pyspark/ml/tests.py --- @@ -541,6 +541,16 @@ def test_java_params(self):

[GitHub] spark pull request #20627: [SPARK-23217][ML][PYTHON] Add distanceMeasure par...

2018-02-19 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/20627#discussion_r169158234 --- Diff: python/pyspark/ml/tests.py --- @@ -541,6 +541,16 @@ def test_java_params(self):

[GitHub] spark pull request #20627: [SPARK-23217][ML][PYTHON] Add distanceMeasure par...

2018-02-19 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/20627#discussion_r169156894 --- Diff: python/pyspark/ml/tests.py --- @@ -52,7 +52,7 @@ from pyspark.ml.clustering import * from pyspark.ml.common import _java2py,

[GitHub] spark pull request #20637: Remove redundant null checks in generated Java co...

2018-02-19 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/20637 Remove redundant null checks in generated Java code by GenerateUnsafeProjection ## What changes were proposed in this pull request? This PR works for one of TODOs in

[GitHub] spark pull request #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferH...

2018-02-19 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20636#discussion_r169151430 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/BufferHolderSparkSubmitSuite.scala --- @@ -58,15 +58,20 @@

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-02-19 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20636 Thank you for pining me and working on this issue, @kiszk . --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20631: [SPARK-23454][SS][DOCS] Added trigger information...

2018-02-19 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/20631#discussion_r169146175 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -1051,6 +1053,16 @@ from the aggregation column. For example,

[GitHub] spark issue #19881: [SPARK-22683][CORE] Add tasksPerExecutorSlot parameter

2018-02-19 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/19881 ping @jcuquemelle can you update this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20571: [SPARK-23383][Build][Minor]Make a distribution should ex...

2018-02-19 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/20571 yes, single dash means the coming ones are all maven options,they will be handled later by mvn command. we do not parse each single option in this while loop from then till now. ---

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169120482 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -770,7 +837,14 @@ case class Unhex(child:

[GitHub] spark pull request #20627: [SPARK-23217][ML][PYTHON] Add distanceMeasure par...

2018-02-19 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20627#discussion_r169116845 --- Diff: python/pyspark/ml/evaluation.py --- @@ -394,15 +397,30 @@ def getMetricName(self): @keyword_only @since("2.3.0") def

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-02-19 Thread misutoth
Github user misutoth commented on the issue: https://github.com/apache/spark/pull/20618 Thanks @srowen and @HyukjinKwon for your comments so far ... --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread misutoth
Github user misutoth commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169100931 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -770,7 +837,14 @@ case class Unhex(child:

[GitHub] spark issue #20632: [SPARK-3159] added subtree pruning in the translation fr...

2018-02-19 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20632 Another one that @jkbradley or @MLnick might want to look at, but seems like a nice win. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r169091069 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala --- @@ -287,6 +291,34 @@ private[tree] class LearningNode( } } +

[GitHub] spark pull request #20632: [SPARK-3159] added subtree pruning in the transla...

2018-02-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20632#discussion_r169092238 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala --- @@ -402,20 +406,40 @@ class RandomForestSuite extends SparkFunSuite

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-02-19 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20636 cc @dongjoon-hyun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20627: [SPARK-23217][ML][PYTHON] Add distanceMeasure par...

2018-02-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20627#discussion_r169090021 --- Diff: python/pyspark/ml/evaluation.py --- @@ -394,15 +397,30 @@ def getMetricName(self): @keyword_only @since("2.3.0") def

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169086289 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -1313,131 +1313,168 @@ object functions {

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169085966 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala --- @@ -770,7 +837,14 @@ case class Unhex(child:

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169086344 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -1313,131 +1313,168 @@ object functions {

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-02-19 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r169086471 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -1500,31 +1537,35 @@ object functions { } /** - *

  1   2   >