[GitHub] spark issue #21594: [SPARK-24596][SQL] Non-cascading Cache Invalidation

2018-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21594 **[Test build #92265 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92265/testReport)** for PR 21594 at commit

[GitHub] spark issue #21594: [SPARK-24596][SQL] Non-cascading Cache Invalidation

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21594 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21594: [SPARK-24596][SQL] Non-cascading Cache Invalidation

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21594 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/442/

[GitHub] spark pull request #21625: [SPARK-24206][SQL][FOLLOW-UP] Update DataSourceRe...

2018-06-23 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21625#discussion_r197628635 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/DataSourceReadBenchmark.scala --- @@ -573,32 +578,6 @@ object

[GitHub] spark pull request #21625: [SPARK-24206][SQL][FOLLOW-UP] Update DataSourceRe...

2018-06-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21625#discussion_r197627610 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/DataSourceReadBenchmark.scala --- @@ -573,32 +578,6 @@ object

[GitHub] spark issue #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21482 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92262/ Test PASSed. ---

[GitHub] spark issue #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21482 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21482 **[Test build #92262 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92262/testReport)** for PR 21482 at commit

[GitHub] spark issue #21625: [SPARK-24206][SQL][FOLLOW-UP] Update DataSourceReadBench...

2018-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21625 **[Test build #92264 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92264/testReport)** for PR 21625 at commit

[GitHub] spark issue #21625: [SPARK-24206][SQL][FOLLOW-UP] Update DataSourceReadBench...

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21625 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/441/

[GitHub] spark issue #21625: [SPARK-24206][SQL][FOLLOW-UP] Update DataSourceReadBench...

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21625 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21625: [SPARK-24206][SQL][FOLLOW-UP] Update DataSourceReadBench...

2018-06-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21625 This pr is a follow-up of https://github.com/apache/spark/pull/21288#event-1697426905 --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21625: [SPARK-24206][SQL][FOLLOW-UP] Update DataSourceRe...

2018-06-23 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/21625 [SPARK-24206][SQL][FOLLOW-UP] Update DataSourceReadBenchmark benchmark results ## What changes were proposed in this pull request? This pr corrected the default configuration

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21389 **[Test build #92263 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92263/testReport)** for PR 21389 at commit

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21389 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFi...

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21389 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/440/

[GitHub] spark issue #21288: [SPARK-24206][SQL] Improve FilterPushdownBenchmark bench...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21288 Sure --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21379: [SPARK-24327][SQL] Verify and normalize a partition colu...

2018-06-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21379 ping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-06-23 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r197626497 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceUtils.scala --- @@ -0,0 +1,88 @@ +/* + * Licensed to the

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-06-23 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r197626501 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -202,4 +204,222 @@ class FileBasedDataSourceSuite extends

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20345 cc @maryannxue Please take a look at this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r197626306 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceUtils.scala --- @@ -0,0 +1,88 @@ +/* + * Licensed to the

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r197626232 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala --- @@ -202,4 +204,222 @@ class FileBasedDataSourceSuite extends

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-06-23 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r197626168 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceUtils.scala --- @@ -0,0 +1,88 @@ +/* + * Licensed to the

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-06-23 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r197626164 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceUtils.scala --- @@ -0,0 +1,88 @@ +/* + * Licensed to the

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r197626154 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceUtils.scala --- @@ -0,0 +1,88 @@ +/* + * Licensed to the

[GitHub] spark issue #21598: [SPARK-24605][SQL] size(null) returns null instead of -1

2018-06-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21598 IMHO we need to have clear decision rules for these kinds of behaviour changes in the contribution guide. In the past migration guide descriptions, it seems we've already accept some behaviour

[GitHub] spark pull request #21427: [SPARK-24324][PYTHON] Pandas Grouped Map UDF shou...

2018-06-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21427 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21389: [SPARK-24204][SQL] Verify a schema in Json/Orc/Pa...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21389#discussion_r197626122 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceUtils.scala --- @@ -0,0 +1,88 @@ +/* + * Licensed to the

[GitHub] spark pull request #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21482#discussion_r197626112 --- Diff: python/pyspark/sql/column.py --- @@ -514,6 +514,16 @@ def isin(self, *cols): desc_nulls_first =

[GitHub] spark issue #21427: [SPARK-24324][PYTHON] Pandas Grouped Map UDF should assi...

2018-06-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21427 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21598: [SPARK-24605][SQL] size(null) returns null instead of -1

2018-06-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21598 @markhamstra, Spark sometimes has some behaviour changes for some bug fixes or in few other cases so far. At least, see the similar configurations added in the migration guide. It sounded we

[GitHub] spark issue #21288: [SPARK-24206][SQL] Improve FilterPushdownBenchmark bench...

2018-06-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21288 Thanks for the check! btw, `DataSourceReadBenchmark` has the same issue (`spark.master` setup), so is it ok to fix this as follow-up?

[GitHub] spark pull request #21288: [SPARK-24206][SQL] Improve FilterPushdownBenchmar...

2018-06-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21288 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21288: [SPARK-24206][SQL] Improve FilterPushdownBenchmark bench...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21288 LGTM Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21247: [SPARK-24190][SQL] Allow saving of JSON files in ...

2018-06-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21247 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20949: [SPARK-19018][SQL] Add support for custom encoding on cs...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20949 cc @MaxGekk @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21247: [SPARK-24190][SQL] Allow saving of JSON files in UTF-16 ...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21247 LGTM Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21590: [SPARK-24423][SQL] Add a new option for JDBC sources

2018-06-23 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/21590 @gatorsmile Thanks a lot. I will process your comments and get back. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21482 **[Test build #92262 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92262/testReport)** for PR 21482 at commit

[GitHub] spark issue #21598: [SPARK-24605][SQL] size(null) returns null instead of -1

2018-06-23 Thread markhamstra
Github user markhamstra commented on the issue: https://github.com/apache/spark/pull/21598 @HyukjinKwon this is not new policy. It is what Apache Spark has guaranteed in its version numbering and public API since 1.0.0. It is not a matter of "from now on", but rather of whether

[GitHub] spark pull request #21621: [SPARK-24633][SQL] Fix codegen when split is requ...

2018-06-23 Thread bersprockets
Github user bersprockets commented on a diff in the pull request: https://github.com/apache/spark/pull/21621#discussion_r197623576 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala --- @@ -556,6 +556,17 @@ class DataFrameFunctionsSuite extends

[GitHub] spark issue #12119: [SPARK-14288][SQL] Memory Sink for streaming

2018-06-23 Thread ChandraMadhumanchi
Github user ChandraMadhumanchi commented on the issue: https://github.com/apache/spark/pull/12119 Whether Memory Structured streaming sink is idempotent and provides end-to-end exactly-once semantics? --- - To

[GitHub] spark issue #21622: [SPARK-24637][SS] Add metrics regarding state and waterm...

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21622 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92261/ Test PASSed. ---

[GitHub] spark issue #21622: [SPARK-24637][SS] Add metrics regarding state and waterm...

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21622 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21622: [SPARK-24637][SS] Add metrics regarding state and waterm...

2018-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21622 **[Test build #92261 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92261/testReport)** for PR 21622 at commit

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r197620329 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -65,13 +65,38 @@ class JDBCOptions(

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r197620436 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -65,13 +65,38 @@ class JDBCOptions(

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r197620455 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -109,6 +134,20 @@ class JDBCOptions(

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r197620140 --- Diff: docs/sql-programming-guide.md --- @@ -1302,9 +1302,20 @@ the following case-insensitive options: dbtable -

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r197620099 --- Diff: docs/sql-programming-guide.md --- @@ -1302,9 +1302,20 @@ the following case-insensitive options: dbtable -

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r197620021 --- Diff: docs/sql-programming-guide.md --- @@ -1302,9 +1302,20 @@ the following case-insensitive options: dbtable -

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r197620115 --- Diff: docs/sql-programming-guide.md --- @@ -1302,9 +1302,20 @@ the following case-insensitive options: dbtable -

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r197620496 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcRelationProvider.scala --- @@ -58,6 +58,10 @@ class

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r197620483 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -109,6 +134,20 @@ class JDBCOptions(

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r197620091 --- Diff: docs/sql-programming-guide.md --- @@ -1302,9 +1302,20 @@ the following case-insensitive options: dbtable -

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r197620340 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -65,13 +65,38 @@ class JDBCOptions(

[GitHub] spark pull request #21590: [SPARK-24423][SQL] Add a new option for JDBC sour...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21590#discussion_r197620384 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -65,13 +65,38 @@ class JDBCOptions(

[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...

2018-06-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21588 > is it basically true that Hadoop 3 will work with only minor patches to the Hive fork in Spark? Up to my knowledge so far, yes, it basically works. At least, the regression tests we

[GitHub] spark issue #21598: [SPARK-24605][SQL] size(null) returns null instead of -1

2018-06-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21598 I am good to have the configuration as that's basically what I suggested too. Removing the behaviour in 3.0.0 is fine. My only question left is the default value. > We should change

[GitHub] spark pull request #21598: [SPARK-24605][SQL] size(null) returns null instea...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21598#discussion_r197618603 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -75,28 +75,47 @@ trait

[GitHub] spark pull request #21598: [SPARK-24605][SQL] size(null) returns null instea...

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21598#discussion_r197618318 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -75,28 +75,47 @@ trait

[GitHub] spark issue #21619: [SPARK-24635][SQL] Remove Blocks class from JavaCode cla...

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21619 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21619: [SPARK-24635][SQL] Remove Blocks class from JavaCode cla...

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21619 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92259/ Test PASSed. ---

[GitHub] spark issue #21619: [SPARK-24635][SQL] Remove Blocks class from JavaCode cla...

2018-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21619 **[Test build #92259 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92259/testReport)** for PR 21619 at commit

[GitHub] spark issue #20701: [SPARK-23528][ML] Add numIter to ClusteringSummary

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20701 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92257/ Test PASSed. ---

[GitHub] spark issue #20701: [SPARK-23528][ML] Add numIter to ClusteringSummary

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20701 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20701: [SPARK-23528][ML] Add numIter to ClusteringSummary

2018-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20701 **[Test build #92257 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92257/testReport)** for PR 20701 at commit

[GitHub] spark issue #21598: [SPARK-24605][SQL] size(null) returns null instead of -1

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21598 I created a JIRA https://issues.apache.org/jira/browse/SPARK-24640. We should change the behavior in 3.0. Before 3.0 release, we introduce a conf and make it configurable. The default is to keep

[GitHub] spark issue #21623: [SPARK-24638][SQL] StringStartsWith support push down

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21623 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92258/ Test PASSed. ---

[GitHub] spark issue #21623: [SPARK-24638][SQL] StringStartsWith support push down

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21623 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21623: [SPARK-24638][SQL] StringStartsWith support push down

2018-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21623 **[Test build #92258 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92258/testReport)** for PR 21623 at commit

[GitHub] spark issue #21622: [SPARK-24637][SS] Add metrics regarding state and waterm...

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21622 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92260/ Test FAILed. ---

[GitHub] spark issue #21622: [SPARK-24637][SS] Add metrics regarding state and waterm...

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21622 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21622: [SPARK-24637][SS] Add metrics regarding state and waterm...

2018-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21622 **[Test build #92260 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92260/testReport)** for PR 21622 at commit

[GitHub] spark issue #21610: Updates to LICENSE and NOTICE

2018-06-23 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21610 OK if you want to pare down the PR to fixes we had discussed here and on the thread, I can get it in. I will need to just reevaluate the whole LICENSE and NOTICE anyway as it's clear it's at least a

[GitHub] spark issue #21621: [SPARK-24633][SQL] Fix codegen when split is required fo...

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21621 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21621: [SPARK-24633][SQL] Fix codegen when split is required fo...

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21621 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92255/ Test PASSed. ---

[GitHub] spark issue #21621: [SPARK-24633][SQL] Fix codegen when split is required fo...

2018-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21621 **[Test build #92255 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92255/testReport)** for PR 21621 at commit

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92254/ Test PASSed. ---

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #92254 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92254/testReport)** for PR 21596 at commit

[GitHub] spark issue #21622: [SPARK-24637][SS] Add metrics regarding state and waterm...

2018-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21622 **[Test build #92261 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92261/testReport)** for PR 21622 at commit

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistics to improve ...

2018-06-23 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/16677 If there is some codepath not updating shuffle write metrics (introduced for sql), that would be a bug. On Sat, Jun 23, 2018 at 7:27 AM Liang-Chi Hsieh wrote: > *@viirya*

[GitHub] spark issue #21622: [SPARK-24637][SS] Add metrics regarding state and waterm...

2018-06-23 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21622 retest this, please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21622: [SPARK-24637][SS] Add metrics regarding state and waterm...

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21622 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21622: [SPARK-24637][SS] Add metrics regarding state and waterm...

2018-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21622 **[Test build #92256 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92256/testReport)** for PR 21622 at commit

[GitHub] spark issue #21622: [SPARK-24637][SS] Add metrics regarding state and waterm...

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21622 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92256/ Test FAILed. ---

[GitHub] spark issue #21623: [SPARK-24638][SQL] StringStartsWith support push down

2018-06-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21623 cc @rdblue --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...

2018-06-23 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21588 Also weighing in here to ask: is it basically true that Hadoop 3 will work with only minor patches to the Hive fork in Spark? then that seems worth the hacking. Is the blocker that we can't get a

[GitHub] spark issue #21622: [SPARK-24637][SS] Add metrics regarding state and waterm...

2018-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21622 **[Test build #92260 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92260/testReport)** for PR 21622 at commit

[GitHub] spark issue #21622: [SPARK-24637][SS] Add metrics regarding state and waterm...

2018-06-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21622 add to whitelist --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21542: [SPARK-24529][Build][test-maven] Add spotbugs int...

2018-06-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21542#discussion_r197613977 --- Diff: pom.xml --- @@ -2606,6 +2606,35 @@ + +com.github.spotbugs +

[GitHub] spark issue #21610: Updates to LICENSE and NOTICE

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21610 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92252/ Test PASSed. ---

[GitHub] spark issue #21610: Updates to LICENSE and NOTICE

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21610 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21610: Updates to LICENSE and NOTICE

2018-06-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21610 **[Test build #92252 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92252/testReport)** for PR 21610 at commit

[GitHub] spark pull request #16677: [SPARK-19355][SQL] Use map output statistics to i...

2018-06-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16677#discussion_r197613554 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/BypassMergeSortShuffleWriter.java --- @@ -145,10 +145,12 @@ public void write(Iterator> records)

[GitHub] spark pull request #16677: [SPARK-19355][SQL] Use map output statistics to i...

2018-06-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16677#discussion_r197613004 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -204,6 +204,13 @@ object SQLConf { .intConf

[GitHub] spark issue #21619: [SPARK-24635][SQL] Remove Blocks class from JavaCode cla...

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21619 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/439/

[GitHub] spark issue #21619: [SPARK-24635][SQL] Remove Blocks class from JavaCode cla...

2018-06-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21619 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

  1   2   3   >