[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...

2017-11-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18424 Now we will have this for row group filtering in most cases after #15049. I believe it makes sense in this case. --- - To un

[GitHub] spark pull request #17631: [SPARK-20319][SQL] Already quoted identifiers are...

2017-11-14 Thread umesh9794
Github user umesh9794 commented on a diff in the pull request: https://github.com/apache/spark/pull/17631#discussion_r150809485 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala --- @@ -121,7 +121,8 @@ object JdbcUtils extends Loggin

[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18424 **[Test build #83844 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83844/testReport)** for PR 18424 at commit [`8c443a7`](https://github.com/apache/spark/commit/8c

[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18424 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83844/ Test FAILed. ---

[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18424 **[Test build #83844 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83844/testReport)** for PR 18424 at commit [`8c443a7`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18424 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...

2017-11-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18424 I believe I need to cc @jiangxb1987 and @viirya too who activiely reviewed my PR. --- - To unsubscribe, e-mail: reviews-unsu

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2017-11-14 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/17702 @zsxwing Thanks for your comments, ready for review. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...

2017-11-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18424 cc @liancheng too who I know is insightful in this. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For ad

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19691 **[Test build #83839 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83839/testReport)** for PR 19691 at commit [`9832ec5`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #19743: [SPARK-22515] [SQL] Estimation relation size based on nu...

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19743 **[Test build #83841 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83841/testReport)** for PR 19743 at commit [`cc9ecc6`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #19743: [SPARK-22515] [SQL] Estimation relation size based on nu...

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19743 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19691 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19691 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83839/ Test PASSed. ---

[GitHub] spark issue #19743: [SPARK-22515] [SQL] Estimation relation size based on nu...

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19743 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83841/ Test PASSed. ---

[GitHub] spark issue #19640: [SPARK-16986][WEB-UI] Converter Started, Completed and L...

2017-11-14 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/19640 It looks like this: https://user-images.githubusercontent.com/5399861/32779430-be47bac8-c978-11e7-97d1-fd3e3db9166f.png";> ---

[GitHub] spark issue #19640: [SPARK-16986][WEB-UI] Converter Started, Completed and L...

2017-11-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19640 why it has `PM` postfix at the end? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19439 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19439 FWIW, I have no more comments and seems fine. I happened to review this and manually test multiple times. --- - To unsubscri

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83845 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83845/testReport)** for PR 19439 at commit [`c4a6dfe`](https://github.com/apache/spark/commit/c4

[GitHub] spark issue #19640: [SPARK-16986][WEB-UI] Converter Started, Completed and L...

2017-11-14 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/19640 Generated by the `toLocaleTimeString ()` method: ![image](https://user-images.githubusercontent.com/5399861/32780670-c2d61678-c907-11e7-837f-f01503d22aec.png) ---

[GitHub] spark pull request #19740: [SPARK-22514][SQL] move ColumnVector.Array and Co...

2017-11-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19740#discussion_r150825051 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnVectorSuite.scala --- @@ -179,7 +179,7 @@ class ColumnVectorSuite extends S

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2017-11-14 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/18853 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18853 **[Test build #83847 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83847/testReport)** for PR 18853 at commit [`22d0355`](https://github.com/apache/spark/commit/22

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18906 **[Test build #83846 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83846/testReport)** for PR 18906 at commit [`9856be6`](https://github.com/apache/spark/commit/98

[GitHub] spark issue #19741: [SPARK-14228][CORE][YARN] Lost executor of RPC disassoci...

2017-11-14 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19741 cc @jerryshao --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h..

[GitHub] spark issue #19720: [SPARK-22494][SQL] Fix 64KB limit exception with Coalesc...

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19720 **[Test build #83843 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83843/testReport)** for PR 19720 at commit [`3a5c683`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #19720: [SPARK-22494][SQL] Fix 64KB limit exception with Coalesc...

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19720 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83843/ Test PASSed. ---

[GitHub] spark issue #19720: [SPARK-22494][SQL] Fix 64KB limit exception with Coalesc...

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19720 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19640: [SPARK-16986][WEB-UI] Converter Started, Completed and L...

2017-11-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19640 sorry, I mean we should follow the format we use in completed, started... --- - To unsubscribe, e-mail: reviews-unsubscr...@spa

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2017-11-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18906 > Satisfying a use case that I described in my initial comments. I am actually not quite clear of the usecases. Providing actual codes and elaborating it should be helpful. > C

[GitHub] spark pull request #18641: [SPARK-21413][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18641#discussion_r150839052 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -2158,4 +2158,21 @@ class DataFrameSuite extends QueryTest with SharedSQLC

[GitHub] spark pull request #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "...

2017-11-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17436#discussion_r150839810 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -80,6 +80,11 @@ package object config { .bytesConf(ByteUnit.M

[GitHub] spark pull request #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "...

2017-11-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17436#discussion_r150840015 --- Diff: core/src/main/scala/org/apache/spark/memory/MemoryManager.scala --- @@ -195,7 +196,7 @@ private[spark] abstract class MemoryManager( */

[GitHub] spark pull request #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "...

2017-11-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17436#discussion_r150840518 --- Diff: core/src/test/scala/org/apache/spark/memory/StaticMemoryManagerSuite.scala --- @@ -43,12 +43,13 @@ class StaticMemoryManagerSuite extends Memor

[GitHub] spark issue #19640: [SPARK-16986][WEB-UI] Converter Started, Completed and L...

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19640 **[Test build #83848 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83848/testReport)** for PR 19640 at commit [`4a78965`](https://github.com/apache/spark/commit/4a

[GitHub] spark pull request #19640: [SPARK-16986][WEB-UI] Converter Started, Complete...

2017-11-14 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/19640#discussion_r150841579 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala --- @@ -352,7 +352,7 @@ class HistoryServerSuite extends SparkFunSuit

[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...

2017-11-14 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18424 Please rebase this PR to the latest master. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For ad

[GitHub] spark issue #19743: [SPARK-22515] [SQL] Estimation relation size based on nu...

2017-11-14 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/19743 cc @cloud-fan @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revi

[GitHub] spark pull request #19640: [SPARK-16986][WEB-UI] Converter Started, Complete...

2017-11-14 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/19640#discussion_r150842384 --- Diff: core/src/main/resources/org/apache/spark/ui/static/utils.js --- @@ -46,3 +46,25 @@ function formatBytes(bytes, type) { var i = Math.floor(

[GitHub] spark pull request #19640: [SPARK-16986][WEB-UI] Converter Started, Complete...

2017-11-14 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/19640#discussion_r150842538 --- Diff: core/src/main/resources/org/apache/spark/ui/static/historypage.js --- @@ -37,11 +37,6 @@ function makeIdNumeric(id) { return resl; }

[GitHub] spark pull request #19745: [SPARK-2926][Core][Follow Up] Sort shuffle reader...

2017-11-14 Thread xuanyuanking
GitHub user xuanyuanking opened a pull request: https://github.com/apache/spark/pull/19745 [SPARK-2926][Core][Follow Up] Sort shuffle reader for Spark 2.x ## What changes were proposed in this pull request? As comment in [SPARK-2926][https://issues.apache.org/jira/browse/SP

[GitHub] spark pull request #18641: [SPARK-21413][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18641#discussion_r150845392 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -262,37 +262,91 @@ case class CaseWhenCod

[GitHub] spark pull request #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "...

2017-11-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17436#discussion_r150845467 --- Diff: core/src/test/scala/org/apache/spark/memory/StaticMemoryManagerSuite.scala --- @@ -43,12 +44,14 @@ class StaticMemoryManagerSuite extends Memor

[GitHub] spark pull request #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "...

2017-11-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17436#discussion_r150845551 --- Diff: core/src/test/scala/org/apache/spark/memory/UnifiedMemoryManagerSuite.scala --- @@ -40,10 +42,12 @@ class UnifiedMemoryManagerSuite extends Mem

[GitHub] spark pull request #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "...

2017-11-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17436#discussion_r150846694 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -351,11 +351,12 @@ class ParquetFileF

[GitHub] spark issue #19745: [SPARK-2926][Core][Follow Up] Sort shuffle reader for Sp...

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19745 **[Test build #83849 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83849/testReport)** for PR 19745 at commit [`ca43f1b`](https://github.com/apache/spark/commit/ca

[GitHub] spark pull request #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "...

2017-11-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17436#discussion_r150847402 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/AggregateHashMap.java --- @@ -40,7 +42,7 @@ */ public class Aggrega

[GitHub] spark pull request #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "...

2017-11-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17436#discussion_r150848938 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -60,9 +62,15 @@ case class InMemoryTableScan

[GitHub] spark pull request #19640: [SPARK-16986][WEB-UI] Converter Started, Complete...

2017-11-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19640#discussion_r150849659 --- Diff: core/src/main/resources/org/apache/spark/ui/static/utils.js --- @@ -46,3 +46,25 @@ function formatBytes(bytes, type) { var i = Math.floo

[GitHub] spark pull request #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "...

2017-11-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/17436#discussion_r150852190 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/AggregateHashMap.java --- @@ -40,7 +42,7 @@ */ public class AggregateHa

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-11-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17436 after some more thoughts, I think we should not rely on `spark.memory.offHeap.enabled`. The reason is that: 1) `OffHeapColumnVector` may have performance regression, users may want off-heap memory

[GitHub] spark pull request #18641: [SPARK-21413][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18641#discussion_r150856256 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -262,37 +262,91 @@ case class CaseWhe

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2017-11-14 Thread ptkool
Github user ptkool commented on the issue: https://github.com/apache/spark/pull/18906 Here are the similar changes in the Scala API: https://github.com/apache/spark/pull/17911 --- - To unsubscribe, e-mail: reviews-u

[GitHub] spark issue #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "spark.m...

2017-11-14 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/17436 Good idea. We will introduce the new config independent of `spark.memory.offHeap.enabled`. --- - To unsubscribe, e-mail: reviews-

[GitHub] spark pull request #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "...

2017-11-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/17436#discussion_r150859426 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -60,9 +62,15 @@ case class InMemoryTableScanExec

[GitHub] spark pull request #19746: [SPARK-22346][ML]

2017-11-14 Thread MrBago
GitHub user MrBago opened a pull request: https://github.com/apache/spark/pull/19746 [SPARK-22346][ML] ## What changes were proposed in this pull request? A new VectorSizeHint transformer was added. This transformer is meant to be used as a pipeline stage ahead of VectorAss

[GitHub] spark issue #19746: [SPARK-22346][ML]

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19746 **[Test build #83850 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83850/testReport)** for PR 19746 at commit [`df990ed`](https://github.com/apache/spark/commit/df

[GitHub] spark issue #19746: [SPARK-22346][ML]

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19746 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83850/ Test FAILed. ---

[GitHub] spark issue #19746: [SPARK-22346][ML]

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19746 **[Test build #83850 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83850/testReport)** for PR 19746 at commit [`df990ed`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #19746: [SPARK-22346][ML]

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19746 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2017-11-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18906 I meant the actual equivalent endpoints and actual codes with usecases. --- - To unsubscribe, e-mail: reviews-unsubscr...@spa

[GitHub] spark pull request #18641: [SPARK-21413][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18641#discussion_r150863818 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -262,37 +262,91 @@ case class CaseWhenCod

[GitHub] spark pull request #19640: [SPARK-16986][WEB-UI] Converter Started, Complete...

2017-11-14 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/19640#discussion_r150863877 --- Diff: core/src/main/resources/org/apache/spark/ui/static/utils.js --- @@ -46,3 +46,25 @@ function formatBytes(bytes, type) { var i = Math.floor(

[GitHub] spark issue #19740: [SPARK-22514][SQL] move ColumnVector.Array and ColumnarB...

2017-11-14 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19740 `ColumnVectorArray` or `ArrayInColumnVector` both good to me. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org F

[GitHub] spark issue #19720: [SPARK-22494][SQL] Fix 64KB limit exception with Coalesc...

2017-11-14 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19720 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache

[GitHub] spark issue #19635: [SPARK-22413][SQL] Type coercion for IN is not coherent ...

2017-11-14 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19635 @gatorsmile any further comment on this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #19640: [SPARK-16986][WEB-UI] Converter Started, Complete...

2017-11-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19640#discussion_r150868951 --- Diff: core/src/main/resources/org/apache/spark/ui/static/utils.js --- @@ -46,3 +46,25 @@ function formatBytes(bytes, type) { var i = Math.floo

[GitHub] spark pull request #19640: [SPARK-16986][WEB-UI] Converter Started, Complete...

2017-11-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19640#discussion_r150869426 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/HistoryPage.scala --- @@ -56,6 +56,10 @@ private[history] class HistoryPage(parent: History

[GitHub] spark pull request #19640: [SPARK-16986][WEB-UI] Converter Started, Complete...

2017-11-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19640#discussion_r150869830 --- Diff: core/src/main/resources/org/apache/spark/ui/static/utils.js --- @@ -46,3 +46,25 @@ function formatBytes(bytes, type) { var i = Math.floo

[GitHub] spark issue #19479: [SPARK-17074] [SQL] Generate equi-height histogram in co...

2017-11-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19479 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19638: [SPARK-22422][ML] Add Adjusted R2 to RegressionMetrics

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19638 **[Test build #3986 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3986/testReport)** for PR 19638 at commit [`68e467e`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #19479: [SPARK-17074] [SQL] Generate equi-height histogra...

2017-11-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19479 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19638: [SPARK-22422][ML] Add Adjusted R2 to RegressionMetrics

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19638 **[Test build #3986 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3986/testReport)** for PR 19638 at commit [`68e467e`](https://github.com/apache/spark/commit/

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18906 **[Test build #83846 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83846/testReport)** for PR 18906 at commit [`9856be6`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18906 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83846/ Test PASSed. ---

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18906 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #19720: [SPARK-22494][SQL] Fix 64KB limit exception with ...

2017-11-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19720#discussion_r150875807 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/nullExpressions.scala --- @@ -72,14 +72,10 @@ case class Coalesce(children: S

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83845 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83845/testReport)** for PR 19439 at commit [`c4a6dfe`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83845/ Test PASSed. ---

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19439 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #19720: [SPARK-22494][SQL] Fix 64KB limit exception with ...

2017-11-14 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19720#discussion_r150877087 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/nullExpressions.scala --- @@ -72,14 +72,10 @@ case class Coalesce(children

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18853 **[Test build #83847 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83847/testReport)** for PR 18853 at commit [`22d0355`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18853 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83847/ Test PASSed. ---

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18853 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #1737: [SPARK-2489] [SQL] Parquet support for fixed_len_byte_arr...

2017-11-14 Thread mukunku
Github user mukunku commented on the issue: https://github.com/apache/spark/pull/1737 I'm using spark 2.2.0 and still have this issue: Illegal Parquet type: FIXED_LEN_BYTE_ARRAY --- - To unsubscribe, e-mail: revi

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-14 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150881619 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0 +1

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19468 **[Test build #83851 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83851/testReport)** for PR 19468 at commit [`0ab9ca7`](https://github.com/apache/spark/commit/0a

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19468 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19468 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83851/ Test FAILed. ---

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19468 **[Test build #83851 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83851/testReport)** for PR 19468 at commit [`0ab9ca7`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #19698: [SPARK-20648][core] Port JobsTab and StageTab to the new...

2017-11-14 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19698 I merged this to master, since its holding up a bunch more of the history server stuff, but I'd encourage other reviews to take a look still @cloud-fan @ajbozarth @jerryshao --- -

[GitHub] spark pull request #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "...

2017-11-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/17436#discussion_r150889105 --- Diff: core/src/test/scala/org/apache/spark/memory/StaticMemoryManagerSuite.scala --- @@ -43,12 +44,14 @@ class StaticMemoryManagerSuite extends MemoryMan

[GitHub] spark pull request #19698: [SPARK-20648][core] Port JobsTab and StageTab to ...

2017-11-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19698 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #17436: [SPARK-20101][SQL] Use OffHeapColumnVector when "...

2017-11-14 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/17436#discussion_r150889421 --- Diff: core/src/test/scala/org/apache/spark/memory/UnifiedMemoryManagerSuite.scala --- @@ -40,10 +42,12 @@ class UnifiedMemoryManagerSuite extends MemoryM

[GitHub] spark issue #19681: [SPARK-20652][sql] Store SQL UI data in the new app stat...

2017-11-14 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19681 @vanzin merge conflict since I merged the Job & stage page change --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.o

[GitHub] spark issue #19640: [SPARK-16986][WEB-UI] Converter Started, Completed and L...

2017-11-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19640 **[Test build #83848 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83848/testReport)** for PR 19640 at commit [`4a78965`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #19640: [SPARK-16986][WEB-UI] Converter Started, Completed and L...

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19640 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83848/ Test FAILed. ---

[GitHub] spark issue #19640: [SPARK-16986][WEB-UI] Converter Started, Completed and L...

2017-11-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19640 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19726: [SPARK-22490][DOC] Add PySpark doc for SparkSession.buil...

2017-11-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19726 Thank you for review, @gatorsmile and @ueshin . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For addi

<    1   2   3   4   5   >