[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156874375 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -154,6 +154,44 @@ class

[GitHub] spark issue #19892: [SPARK-20542][FollowUp][PySpark] Bucketizer support mult...

2017-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19892 **[Test build #84898 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84898/testReport)** for PR 19892 at commit

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156874146 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -154,6 +154,44 @@ class

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156874184 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -154,6 +154,44 @@ class

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156874096 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -154,6 +154,44 @@ class

[GitHub] spark issue #19892: [SPARK-20542][FollowUp][PySpark] Bucketizer support mult...

2017-12-13 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/19892 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19925: [SPARK-22732] Add Structured Streaming APIs to DataSourc...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19925 shall we move these new interfaces to `org.apache.spark.sql.sources.v2.reader/write.streaming` package? --- - To unsubscribe,

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19811 **[Test build #84897 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84897/testReport)** for PR 19811 at commit

[GitHub] spark pull request #19925: [SPARK-22732] Add Structured Streaming APIs to Da...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19925#discussion_r156872558 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/ContinuousWriter.java --- @@ -0,0 +1,41 @@ +/* + * Licensed to the

[GitHub] spark pull request #19925: [SPARK-22732] Add Structured Streaming APIs to Da...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19925#discussion_r156872002 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/MicroBatchReader.java --- @@ -0,0 +1,64 @@ +/* + * Licensed to the

[GitHub] spark pull request #19925: [SPARK-22732] Add Structured Streaming APIs to Da...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19925#discussion_r156871819 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/MicroBatchReader.java --- @@ -0,0 +1,64 @@ +/* + * Licensed to the

[GitHub] spark pull request #19925: [SPARK-22732] Add Structured Streaming APIs to Da...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19925#discussion_r156871677 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/ContinuousReader.java --- @@ -0,0 +1,68 @@ +/* + * Licensed to the

[GitHub] spark pull request #19925: [SPARK-22732] Add Structured Streaming APIs to Da...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19925#discussion_r156871374 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/ContinuousDataReader.java --- @@ -0,0 +1,36 @@ +/* + * Licensed to the

[GitHub] spark pull request #19925: [SPARK-22732] Add Structured Streaming APIs to Da...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19925#discussion_r156870726 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/MicroBatchWriteSupport.java --- @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19811 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19811 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84895/ Test FAILed. ---

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19811 **[Test build #84895 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84895/testReport)** for PR 19811 at commit

[GitHub] spark issue #19975: [SPARK-22781][SS] Support creating streaming dataset wit...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19975 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19975: [SPARK-22781][SS] Support creating streaming dataset wit...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19975 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84893/ Test PASSed. ---

[GitHub] spark issue #19975: [SPARK-22781][SS] Support creating streaming dataset wit...

2017-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19975 **[Test build #84893 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84893/testReport)** for PR 19975 at commit

[GitHub] spark issue #19972: [SPARK-22778][Kubernetes] Added the missing service meta...

2017-12-13 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19972 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19865: [SPARK-22668][SQL] Ensure no global variables in ...

2017-12-13 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19865#discussion_r156868316 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -217,6 +217,18 @@ class CodegenContext {

[GitHub] spark issue #19970: [SPARK-22775][SQL] move dictionary related APIs from Col...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19970 retest this pease --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19975: [SPARK-22781][SS] Support creating streaming dataset wit...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19975 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19975: [SPARK-22781][SS] Support creating streaming dataset wit...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19975 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84892/ Test PASSed. ---

[GitHub] spark issue #19975: [SPARK-22781][SS] Support creating streaming dataset wit...

2017-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19975 **[Test build #84892 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84892/testReport)** for PR 19975 at commit

[GitHub] spark pull request #19865: [SPARK-22668][SQL] Ensure no global variables in ...

2017-12-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19865#discussion_r156866988 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -842,7 +856,10 @@ class CodegenContext {

[GitHub] spark pull request #19865: [SPARK-22668][SQL] Ensure no global variables in ...

2017-12-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19865#discussion_r156866827 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -217,6 +217,18 @@ class CodegenContext {

[GitHub] spark pull request #19865: [SPARK-22668][SQL] Ensure no global variables in ...

2017-12-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19865#discussion_r156866697 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -217,6 +217,18 @@ class CodegenContext {

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-12-13 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19813 > Probably, we'd better to move this discussion to the jira? I summary it and also post the design doc link to the jira. ---

[GitHub] spark pull request #19974: [SPARK-22779][sql] Resolve default values for fal...

2017-12-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19974 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19974: [SPARK-22779][sql] Resolve default values for fallback c...

2017-12-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19974 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19974: [SPARK-22779][sql] Resolve default values for fallback c...

2017-12-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19974 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-12-13 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19813 > can we have a google doc so that we can leave comments inline? thanks! Sure. The google doc is at

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2017-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14151 **[Test build #84896 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84896/testReport)** for PR 14151 at commit

[GitHub] spark pull request #19925: [SPARK-22732] Add Structured Streaming APIs to Da...

2017-12-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19925 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19974: [SPARK-22779][sql] Resolve default values for fal...

2017-12-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19974#discussion_r156863889 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/internal/SQLConfSuite.scala --- @@ -280,4 +280,34 @@ class SQLConfSuite extends QueryTest with

[GitHub] spark issue #19925: [SPARK-22732] Add Structured Streaming APIs to DataSourc...

2017-12-13 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/19925 Thanks! Merging to master to unblock #19926. If there are more comments, we can address them in #19926. --- - To unsubscribe,

[GitHub] spark pull request #19865: [SPARK-22668][SQL] Ensure no global variables in ...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19865#discussion_r156863124 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -217,6 +217,18 @@ class

[GitHub] spark issue #19974: [SPARK-22779][sql] Resolve default values for fallback c...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19974 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84890/ Test PASSed. ---

[GitHub] spark issue #19974: [SPARK-22779][sql] Resolve default values for fallback c...

2017-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19974 **[Test build #84890 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84890/testReport)** for PR 19974 at commit

[GitHub] spark issue #19974: [SPARK-22779][sql] Resolve default values for fallback c...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19974 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19865: [SPARK-22668][SQL] Ensure no global variables in ...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19865#discussion_r156862608 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -217,6 +217,18 @@ class

[GitHub] spark issue #19970: [SPARK-22775][SQL] move dictionary related APIs from Col...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19970 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84891/ Test FAILed. ---

[GitHub] spark issue #19970: [SPARK-22775][SQL] move dictionary related APIs from Col...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19970 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19970: [SPARK-22775][SQL] move dictionary related APIs from Col...

2017-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19970 **[Test build #84891 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84891/testReport)** for PR 19970 at commit

[GitHub] spark pull request #19971: [SPARK-22774][SQL][Test] Add compilation check in...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19971#discussion_r156861190 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/TPCDSQuerySuite.scala --- @@ -348,13 +351,38 @@ class TPCDSQuerySuite extends QueryTest with

[GitHub] spark pull request #19971: [SPARK-22774][SQL][Test] Add compilation check in...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19971#discussion_r156860978 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/TPCDSQuerySuite.scala --- @@ -348,13 +351,38 @@ class TPCDSQuerySuite extends QueryTest with

[GitHub] spark pull request #19594: [SPARK-21984] [SQL] Join estimation based on equi...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19594#discussion_r156861642 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/JoinEstimation.scala --- @@ -191,8 +191,16 @@ case

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156860058 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -179,11 +209,64 @@ class CodegenContext {

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-12-13 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19813 Probably, we'd better to move this discussion to the jira? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19813 can we have a google doc so that we can leave comments inline? thanks! --- - To unsubscribe, e-mail:

[GitHub] spark issue #19971: [SPARK-22774][SQL][Test] Add compilation check into TPCD...

2017-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19971 **[Test build #84894 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84894/testReport)** for PR 19971 at commit

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19811 **[Test build #84895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84895/testReport)** for PR 19811 at commit

[GitHub] spark issue #19811: [SPARK-18016][SQL] Code Generation: Constant Pool Limit ...

2017-12-13 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19811 Just resolved conflicts. Next commit will address review comments. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19862: [SPARK-22671][SQL] Make SortMergeJoin shuffle read less ...

2017-12-13 Thread gczsjdy
Github user gczsjdy commented on the issue: https://github.com/apache/spark/pull/19862 cc @cloud-fan @hvanhovell @viirya --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-12-13 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19813 I wrote a design doc at https://paper.dropbox.com/doc/Split-deeply-nested-expressions-under-wholestage-codegen-WXkQ9iIlN3zkGdn8MHgiW The design is based on what did in this PR.

[GitHub] spark issue #19955: [SPARK-21867][CORE] Support async spilling in UnsafeShuf...

2017-12-13 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/19955 @ericvandenbergfb - In the testing section, let's put the benchmark numbers for large shuffle heavy jobs we ran internally. ---

[GitHub] spark issue #19955: [SPARK-21867][CORE] Support async spilling in UnsafeShuf...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19955 and also what's the behavior if the memory is used up but `spark.shuffle.async.num.sorter` is not hit yet? --- - To

[GitHub] spark issue #19955: [SPARK-21867][CORE] Support async spilling in UnsafeShuf...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19955 shall we do it for all the external sorters? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19955: [SPARK-21867][CORE] Support async spilling in UnsafeShuf...

2017-12-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19955 cc @JoshRosen @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #17619: [SPARK-19755][Mesos] Blacklist is always active for Meso...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17619 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19020: [SPARK-3181] [ML] Implement huber loss for Linear...

2017-12-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19020 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19020: [SPARK-3181] [ML] Implement huber loss for LinearRegress...

2017-12-13 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/19020 Merged into master, thanks for all your reviewing. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #19020: [SPARK-3181] [ML] Implement huber loss for Linear...

2017-12-13 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/19020#discussion_r156856635 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala --- @@ -480,10 +640,14 @@ object LinearRegression extends

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156855675 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -179,11 +209,64 @@ class

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156853851 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -205,18 +287,35 @@ class

[GitHub] spark issue #19975: [SPARK-22781][SS] Support creating streaming dataset wit...

2017-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19975 **[Test build #84893 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84893/testReport)** for PR 19975 at commit

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156853203 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -179,11 +209,64 @@ class CodegenContext {

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156852077 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -179,11 +209,64 @@ class

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156851936 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -179,11 +209,64 @@ class

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156851771 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -179,11 +209,64 @@ class

[GitHub] spark issue #19975: [SPARK-22781][SS] Support creating streaming dataset wit...

2017-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19975 **[Test build #84892 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84892/testReport)** for PR 19975 at commit

[GitHub] spark pull request #19811: [SPARK-18016][SQL] Code Generation: Constant Pool...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19811#discussion_r156851696 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -179,11 +209,64 @@ class

[GitHub] spark pull request #19975: [SPARK-22781][SS] Support creating streaming data...

2017-12-13 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/19975 [SPARK-22781][SS] Support creating streaming dataset with ORC files ## What changes were proposed in this pull request? This PR supports creating streaming dataset with ORC file

[GitHub] spark issue #19970: [SPARK-22775][SQL] move dictionary related APIs from Col...

2017-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19970 **[Test build #84891 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84891/testReport)** for PR 19970 at commit

[GitHub] spark issue #19972: [SPARK-22778][Kubernetes] Added the missing service meta...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19972 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84888/ Test PASSed. ---

[GitHub] spark issue #19972: [SPARK-22778][Kubernetes] Added the missing service meta...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19972 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19972: [SPARK-22778][Kubernetes] Added the missing service meta...

2017-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19972 **[Test build #84888 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84888/testReport)** for PR 19972 at commit

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19813 I'm not sure, it depends on how many places we already doing it and what's the drawback if we forbid it. Let's have a design doc and gather more feedbacks. Thanks for your understanding!

[GitHub] spark pull request #19864: [SPARK-22673][SQL] InMemoryRelation should utiliz...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19864#discussion_r156847927 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -80,6 +80,14 @@ class CacheManager extends Logging {

[GitHub] spark pull request #19594: [SPARK-21984] [SQL] Join estimation based on equi...

2017-12-13 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/19594#discussion_r156847785 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/JoinEstimation.scala --- @@ -191,8 +191,16 @@ case class

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-12-13 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19813 @cloud-fan Do you think we should still allow something like `a + 1` as the output of codegen? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19594: [SPARK-21984] [SQL] Join estimation based on equi...

2017-12-13 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/19594#discussion_r156847046 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/EstimationUtils.scala --- @@ -114,4 +115,183 @@ object

[GitHub] spark pull request #19594: [SPARK-21984] [SQL] Join estimation based on equi...

2017-12-13 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/19594#discussion_r156846872 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/EstimationUtils.scala --- @@ -114,4 +115,183 @@ object

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-12-13 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19813 Also needs to revert #19969 which is based on this. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-12-13 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19813 @cloud-fan Ok. Thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19813: [SPARK-22600][SQL] Fix 64kb limit for deeply nested expr...

2017-12-13 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19813 After some more thoughts, this PR makes a new contract that Spark doesn't promise before: `Expression.genCode` must output something that can be used as parameter name or literal. I do

[GitHub] spark issue #19955: [SPARK-21867][CORE] Support async spilling in UnsafeShuf...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19955 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84889/ Test FAILed. ---

[GitHub] spark issue #19955: [SPARK-21867][CORE] Support async spilling in UnsafeShuf...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19955 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19955: [SPARK-21867][CORE] Support async spilling in UnsafeShuf...

2017-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19955 **[Test build #84889 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84889/testReport)** for PR 19955 at commit

[GitHub] spark pull request #19970: [SPARK-22775][SQL] move dictionary related APIs f...

2017-12-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19970#discussion_r156843445 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnVector.java --- @@ -145,39 +135,39 @@ public abstract double[]

[GitHub] spark issue #19974: [SPARK-22779][sql] Resolve default values for fallback c...

2017-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19974 **[Test build #84890 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84890/testReport)** for PR 19974 at commit

[GitHub] spark issue #19751: [SPARK-20653][core] Add cleaning of old elements from th...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19751 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84885/ Test PASSed. ---

[GitHub] spark issue #19751: [SPARK-20653][core] Add cleaning of old elements from th...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19751 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19751: [SPARK-20653][core] Add cleaning of old elements from th...

2017-12-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19751 **[Test build #84885 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84885/testReport)** for PR 19751 at commit

[GitHub] spark issue #19974: [SPARK-22779][sql] Resolve default values for fallback c...

2017-12-13 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19974 @rxin I believe this is what you were trying to do? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #19974: [SPARK-22779][sql] Resolve default values for fal...

2017-12-13 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/19974 [SPARK-22779][sql] Resolve default values for fallback configs. SQLConf allows some callers to define a custom default value for configs, and that complicates a little bit the handling of

[GitHub] spark issue #19971: [SPARK-22774][SQL][Test] Add compilation check into TPCD...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19971 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19971: [SPARK-22774][SQL][Test] Add compilation check into TPCD...

2017-12-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19971 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84886/ Test PASSed. ---

  1   2   3   4   5   6   >