[GitHub] spark issue #20945: [SPARK-23790][Mesos] fix metastore connection issue

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20945 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20945: [SPARK-23790][Mesos] fix metastore connection issue

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20945 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88788/ Test FAILed. ---

[GitHub] spark issue #20945: [SPARK-23790][Mesos] fix metastore connection issue

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20945 **[Test build #88788 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88788/testReport)** for PR 20945 at commit

[GitHub] spark issue #20934: [SPARK-23818][SQL][WIP] an official UDF interface for Sp...

2018-03-31 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/20934 Will be open again when interface decision made for this. Thanks. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20934: [SPARK-23818][SQL][WIP] an official UDF interface...

2018-03-31 Thread WeichenXu123
Github user WeichenXu123 closed the pull request at: https://github.com/apache/spark/pull/20934 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec complex typ...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20935 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88786/ Test PASSed. ---

[GitHub] spark issue #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec complex typ...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20935 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec complex typ...

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20935 **[Test build #88786 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88786/testReport)** for PR 20935 at commit

[GitHub] spark issue #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec complex typ...

2018-03-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20935 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178446592 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/ColumnStatsSuite.scala --- @@ -103,4 +140,90 @@ class ColumnStatsSuite extends

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178446501 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala --- @@ -322,19 +324,76 @@ private[columnar] final class

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178446454 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala --- @@ -322,19 +324,76 @@ private[columnar] final class

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178446463 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala --- @@ -322,19 +324,76 @@ private[columnar] final class

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178446476 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala --- @@ -322,19 +324,76 @@ private[columnar] final class

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20850 **[Test build #88787 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88787/testReport)** for PR 20850 at commit

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20850 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20850 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88787/ Test FAILed. ---

[GitHub] spark pull request #20934: [SPARK-23818][SQL][WIP] an official UDF interface...

2018-03-31 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/20934#discussion_r178446367 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/JavaUDF.scala --- @@ -0,0 +1,135 @@ +/* + * Licensed to the

[GitHub] spark issue #20945: [SPARK-23790][Mesos] fix metastore connection issue

2018-03-31 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/20945 @vanzin I think its on the right path now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20945: [SPARK-23790][Mesos] fix metastore connection iss...

2018-03-31 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/20945#discussion_r178445703 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -506,6 +506,10 @@

[GitHub] spark issue #20945: [SPARK-23790][Mesos] fix metastore connection issue

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20945 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1885/

[GitHub] spark issue #20945: [SPARK-23790][Mesos] fix metastore connection issue

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20945 **[Test build #88788 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88788/testReport)** for PR 20945 at commit

[GitHub] spark issue #20945: [SPARK-23790][Mesos] fix metastore connection issue

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20945 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec complex typ...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20935 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec complex typ...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20935 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88785/ Test PASSed. ---

[GitHub] spark issue #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec complex typ...

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20935 **[Test build #88785 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88785/testReport)** for PR 20935 at commit

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20850 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1884/

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20850 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and BufferHolder...

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20850 **[Test build #88787 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88787/testReport)** for PR 20850 at commit

[GitHub] spark issue #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec complex typ...

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20935 **[Test build #88786 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88786/testReport)** for PR 20935 at commit

[GitHub] spark pull request #20850: [SPARK-23713][SQL] Cleanup UnsafeWriter and Buffe...

2018-03-31 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20850#discussion_r178445097 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeArrayWriter.java --- @@ -32,141 +30,129 @@ */ public

[GitHub] spark issue #20953: [SPARK-23822][SQL] Improve error message for Parquet sch...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20953 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20953: [SPARK-23822][SQL] Improve error message for Parquet sch...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20953 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88784/ Test PASSed. ---

[GitHub] spark issue #20953: [SPARK-23822][SQL] Improve error message for Parquet sch...

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20953 **[Test build #88784 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88784/testReport)** for PR 20953 at commit

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread pwoody
Github user pwoody commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178444724 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala --- @@ -307,7 +308,7 @@ private[columnar] final class

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread pwoody
Github user pwoody commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178444713 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/ColumnStatsSuite.scala --- @@ -103,4 +119,109 @@ class ColumnStatsSuite extends

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread pwoody
Github user pwoody commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178444711 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala --- @@ -322,19 +323,75 @@ private[columnar] final class

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread pwoody
Github user pwoody commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178444706 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala --- @@ -322,19 +323,75 @@ private[columnar] final class

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread pwoody
Github user pwoody commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178444697 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala --- @@ -322,19 +323,75 @@ private[columnar] final class

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178443991 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala --- @@ -322,19 +323,75 @@ private[columnar] final class

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178444386 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/ColumnStatsSuite.scala --- @@ -103,4 +119,109 @@ class ColumnStatsSuite extends

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178444088 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/ColumnStatsSuite.scala --- @@ -103,4 +119,109 @@ class ColumnStatsSuite extends

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178444014 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala --- @@ -322,19 +323,75 @@ private[columnar] final class

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178444321 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala --- @@ -322,19 +323,75 @@ private[columnar] final class

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178444026 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala --- @@ -307,7 +308,7 @@ private[columnar] final class

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178444325 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala --- @@ -322,19 +323,75 @@ private[columnar] final class

[GitHub] spark pull request #20947: [SPARK-23705][SQL]Handle non-distinct columns in ...

2018-03-31 Thread TRANTANKHOA
Github user TRANTANKHOA commented on a diff in the pull request: https://github.com/apache/spark/pull/20947#discussion_r178444354 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1593,7 +1596,9 @@ class Dataset[T] private[sql]( def groupBy(col1:

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread pwoody
Github user pwoody commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178442892 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/ColumnarTestUtils.scala --- @@ -54,12 +54,22 @@ object ColumnarTestUtils {

[GitHub] spark issue #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec complex typ...

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20935 **[Test build #88785 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88785/testReport)** for PR 20935 at commit

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread pwoody
Github user pwoody commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178442841 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/ColumnStats.scala --- @@ -323,18 +324,31 @@ private[columnar] final class

[GitHub] spark pull request #20935: [SPARK-23819][SQL] Fix InMemoryTableScanExec comp...

2018-03-31 Thread pwoody
Github user pwoody commented on a diff in the pull request: https://github.com/apache/spark/pull/20935#discussion_r178442839 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/ColumnStatsSuite.scala --- @@ -18,18 +18,35 @@ package

[GitHub] spark pull request #20953: [SPARK-23822][SQL] Improve error message for Parq...

2018-03-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20953#discussion_r178441790 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaSuite.scala --- @@ -382,6 +384,59 @@ class

[GitHub] spark pull request #20953: [SPARK-23822][SQL] Improve error message for Parq...

2018-03-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20953#discussion_r178441777 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaSuite.scala --- @@ -382,6 +384,59 @@ class

[GitHub] spark pull request #20953: [SPARK-23822][SQL] Improve error message for Parq...

2018-03-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20953#discussion_r178441770 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaColumnConvertNotSupportedException.java --- @@ -0,0

[GitHub] spark issue #20953: [SPARK-23822][SQL] Improve error message for Parquet sch...

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20953 **[Test build #88784 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88784/testReport)** for PR 20953 at commit

[GitHub] spark pull request #20953: [SPARK-23822][SQL] Improve error message for Parq...

2018-03-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20953#discussion_r178441756 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedColumnReader.java --- @@ -231,6 +232,18 @@ private

[GitHub] spark pull request #20953: [SPARK-23822][SQL] Improve error message for Parq...

2018-03-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20953#discussion_r178441751 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedColumnReader.java --- @@ -231,6 +232,18 @@ private

[GitHub] spark pull request #20953: [SPARK-23822][SQL] Improve error message for Parq...

2018-03-31 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20953#discussion_r178441740 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaColumnConvertNotSupportedException.java --- @@ -0,0

[GitHub] spark issue #20953: [SPARK-23822][SQL] Improve error message for Parquet sch...

2018-03-31 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20953 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20937: [SPARK-23723][SPARK-23724][SQL] Support custom encoding ...

2018-03-31 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20937 @HyukjinKwon I blacklisted the `UTF-16` and `UTF-32` encodings, and renamed `charset` to `encoding`. Please, have a look at it again. ---

[GitHub] spark issue #20937: [SPARK-23723][SPARK-23724][SQL] Support custom encoding ...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20937 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88783/ Test PASSed. ---

[GitHub] spark issue #20937: [SPARK-23723][SPARK-23724][SQL] Support custom encoding ...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20937 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20937: [SPARK-23723][SPARK-23724][SQL] Support custom encoding ...

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20937 **[Test build #88783 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88783/testReport)** for PR 20937 at commit

[GitHub] spark issue #20937: [SPARK-23723][SPARK-23724][SQL] Support custom encoding ...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20937 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20937: [SPARK-23723][SPARK-23724][SQL] Support custom encoding ...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20937 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88782/ Test PASSed. ---

[GitHub] spark issue #20937: [SPARK-23723][SPARK-23724][SQL] Support custom encoding ...

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20937 **[Test build #88782 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88782/testReport)** for PR 20937 at commit

[GitHub] spark issue #20954: [BACKPORT][SPARK-23040][CORE] Returns interruptible iter...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20954 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88781/ Test PASSed. ---

[GitHub] spark issue #20954: [BACKPORT][SPARK-23040][CORE] Returns interruptible iter...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20954 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20954: [BACKPORT][SPARK-23040][CORE] Returns interruptible iter...

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20954 **[Test build #88781 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88781/testReport)** for PR 20954 at commit

[GitHub] spark pull request #20844: [SPARK-23707][SQL] Don't need shuffle exchange wi...

2018-03-31 Thread gczsjdy
Github user gczsjdy commented on a diff in the pull request: https://github.com/apache/spark/pull/20844#discussion_r178437551 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/ConfigBehaviorSuite.scala --- @@ -39,7 +39,7 @@ class ConfigBehaviorSuite extends QueryTest with

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-03-31 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178434446 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonDataSource.scala --- @@ -175,33 +185,43 @@ object

[GitHub] spark issue #19675: [SPARK-14540][BUILD] Support Scala 2.12 closures and Jav...

2018-03-31 Thread hepin1989
Github user hepin1989 commented on the issue: https://github.com/apache/spark/pull/19675 @fedeoasi Be a sponsor --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20937: [SPARK-23723][SPARK-23724][SQL] Support custom encoding ...

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20937 **[Test build #88783 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88783/testReport)** for PR 20937 at commit

[GitHub] spark issue #20937: [SPARK-23723][SPARK-23724][SQL] Support custom encoding ...

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20937 **[Test build #88782 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88782/testReport)** for PR 20937 at commit

[GitHub] spark pull request #19001: [SPARK-19256][SQL] Hive bucketing support

2018-03-31 Thread chrysan
Github user chrysan commented on a diff in the pull request: https://github.com/apache/spark/pull/19001#discussion_r178433504 --- Diff: sql/hive/src/main/java/org/apache/hadoop/hive/ql/io/BucketizedSparkInputFormat.java --- @@ -0,0 +1,107 @@ +/** + * Licensed to the

[GitHub] spark pull request #20934: [SPARK-23818][SQL][WIP] an official UDF interface...

2018-03-31 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20934#discussion_r178433204 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/JavaUDF.scala --- @@ -0,0 +1,135 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #20633: [SPARK-23455][ML] Default Params in ML should be saved s...

2018-03-31 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20633 ping @jkbradley --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20954: [BACKPORT][SPARK-23040][CORE] Returns interruptible iter...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20954 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20954: [BACKPORT][SPARK-23040][CORE] Returns interruptible iter...

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20954 **[Test build #88781 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88781/testReport)** for PR 20954 at commit

[GitHub] spark issue #20954: [BACKPORT][SPARK-23040][CORE] Returns interruptible iter...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20954 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1883/

[GitHub] spark issue #20954: [BACKPORT][SPARK-23040][CORE] Returns interruptible iter...

2018-03-31 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20954 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19876: [ML][SPARK-23783][SPARK-11239] Add PMML export to...

2018-03-31 Thread goungoun
Github user goungoun commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r178432400 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala --- @@ -710,15 +711,58 @@ class LinearRegressionModel private[ml] (

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178430652 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2065,29 +2065,238 @@ class JsonSuite extends

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18853 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2018-03-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18853 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88780/ Test PASSed. ---

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2018-03-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18853 **[Test build #88780 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88780/testReport)** for PR 18853 at commit

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-03-31 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178428640 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2065,29 +2065,238 @@ class JsonSuite extends

[GitHub] spark issue #20930: [SPARK-23811][Core] FetchFailed comes before Success of ...

2018-03-31 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/20930 > What's your proposed fix? I fix this by killing other attempts while receive a FetchFailed in `TaskSetManager`. If we finally ignore the success event of other attempts, might as well

[GitHub] spark issue #20849: [SPARK-23723] New charset option for json datasource

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20849 The last case seems working dependently by Jackson (UTF-16 for the first and UTF-16LE for the second line) if we don't set `encoding` but Jackson parses it by `UTF-16LE` for both if we set

[GitHub] spark issue #20849: [SPARK-23723] New charset option for json datasource

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20849 Thanks for thoughtfully testing out but I believe we can still go with https://github.com/apache/spark/pull/20937 if we whitelist supported encodings for now? If that's right and I

[GitHub] spark issue #20849: [SPARK-23723] New charset option for json datasource

2018-03-31 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20849 @HyukjinKwon I did an experiment on the https://github.com/MaxGekk/spark-1/pull/2 and modified [the

[GitHub] spark issue #20937: [SPARK-23723][SPARK-23724][SQL] Support custom encoding ...

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20937 Others are mostly nits except https://github.com/apache/spark/pull/20937#discussion_r178426866. Let's talk there more. ---

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178427458 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonFileFormat.scala --- @@ -151,7 +153,16 @@ private[json] class

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178427418 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala --- @@ -86,14 +86,30 @@ private[sql] class JSONOptions(

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178427349 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonDataSource.scala --- @@ -175,33 +185,43 @@ object

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178427279 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2065,29 +2065,238 @@ class JsonSuite extends

[GitHub] spark issue #20937: [SPARK-23723][SPARK-23724][SQL] Support custom encoding ...

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20937 Others are mostly nits except https://github.com/apache/spark/pull/20937#discussion_r178426866. Let's talk there more. ---

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178427175 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/text/TextOptions.scala --- @@ -41,13 +41,16 @@ private[text] class

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178427113 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2065,29 +2065,238 @@ class JsonSuite extends

[GitHub] spark pull request #20937: [SPARK-23723][SPARK-23724][SQL] Support custom en...

2018-03-31 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r178427139 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala --- @@ -86,14 +86,30 @@ private[sql] class JSONOptions(

  1   2   >