[GitHub] spark pull request #21095: [SPARK-23529][K8s] Support mounting hostPath volu...

2018-05-03 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/21095#discussion_r185980118 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesVolumeUtils.scala --- @@ -0,0 +1,142 @@ +/* + *

[GitHub] spark pull request #21095: [SPARK-23529][K8s] Support mounting hostPath volu...

2018-05-03 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/21095#discussion_r185980514 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStep.scala --- @@ -109,7 +109,15 @@ pr

[GitHub] spark pull request #21095: [SPARK-23529][K8s] Support mounting hostPath volu...

2018-05-03 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/21095#discussion_r185960630 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesVolumeUtils.scala --- @@ -0,0 +1,142 @@ +/* + *

[GitHub] spark pull request #21095: [SPARK-23529][K8s] Support mounting hostPath volu...

2018-05-03 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/21095#discussion_r185960149 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesVolumeUtils.scala --- @@ -0,0 +1,142 @@ +/* + *

[GitHub] spark pull request #21095: [SPARK-23529][K8s] Support mounting hostPath volu...

2018-05-03 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/21095#discussion_r185960658 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesVolumeUtils.scala --- @@ -0,0 +1,142 @@ +/* + *

[GitHub] spark pull request #21186: [SPARK-22279][SPARK-24112] Enable `convertMetasto...

2018-05-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21186#discussion_r185980350 --- Diff: docs/sql-programming-guide.md --- @@ -1812,6 +1812,9 @@ working with timestamps in `pandas_udf`s to get the best performance, see - Since

[GitHub] spark issue #21229: [SPARK-23697][CORE] LegacyAccumulatorWrapper should defi...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21229 **[Test build #90175 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90175/testReport)** for PR 21229 at commit [`a2d81df`](https://github.com/apache/spark/commit/a2

[GitHub] spark issue #21229: [SPARK-23697][CORE] LegacyAccumulatorWrapper should defi...

2018-05-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21229 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark issue #21118: SPARK-23325: Use InternalRow when reading with DataSourc...

2018-05-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21118 all the places that use `GenerateUnsafeRowJoiner` assume the input row is unsafe row. `ShuffleExchangeExec` assumes its input is unsafe row, because its serializer is `UnsafeRowSerializer

[GitHub] spark issue #21219: [SPARK-24160] ShuffleBlockFetcherIterator should fail if...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21219 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90161/ Test PASSed. ---

[GitHub] spark issue #21219: [SPARK-24160] ShuffleBlockFetcherIterator should fail if...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21219 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21219: [SPARK-24160] ShuffleBlockFetcherIterator should fail if...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21219 **[Test build #90161 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90161/testReport)** for PR 21219 at commit [`3ecd7da`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #21207: [SPARK-24136][SS] Fix MemoryStreamDataReader.next to ski...

2018-05-03 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/spark/pull/21207 @HyukjinKwon thanks for noticing, updated the title. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2893/ Tes

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21193 **[Test build #90174 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90174/testReport)** for PR 21193 at commit [`d138ee0`](https://github.com/apache/spark/commit/d1

[GitHub] spark issue #21173: [SPARK-23856][SQL] Add an option `queryTimeout` in JDBCO...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21173 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21173: [SPARK-23856][SQL] Add an option `queryTimeout` in JDBCO...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21173 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2892/ Tes

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21193 Thanks for cc. I'll check tonight. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-m

[GitHub] spark issue #21220: [SPARK-24157][SS] Enabled no-data batches in MicroBatchE...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21220 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2891/ Tes

[GitHub] spark issue #21220: [SPARK-24157][SS] Enabled no-data batches in MicroBatchE...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21220 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21061: [SPARK-23914][SQL] Add array_union function

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21061 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21061: [SPARK-23914][SQL] Add array_union function

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21061 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2890/ Tes

[GitHub] spark issue #17332: [SPARK-10764][ML] Add optional caching to Pipelines

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17332 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21173: [SPARK-23856][SQL] Add an option `queryTimeout` in JDBCO...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21173 **[Test build #90173 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90173/testReport)** for PR 21173 at commit [`35759ca`](https://github.com/apache/spark/commit/35

[GitHub] spark issue #21220: [SPARK-24157][SS] Enabled no-data batches in MicroBatchE...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21220 **[Test build #90172 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90172/testReport)** for PR 21220 at commit [`da3fd2f`](https://github.com/apache/spark/commit/da

[GitHub] spark issue #20787: [MINOR][DOCS] Documenting months_between direction

2018-05-03 Thread aditkumar
Github user aditkumar commented on the issue: https://github.com/apache/spark/pull/20787 Not at all On Thu, May 3, 2018 at 9:30 PM Hyukjin Kwon wrote: > *@HyukjinKwon* commented on this pull request. > -- > > In > sql/c

[GitHub] spark issue #21229: [SPARK-23697][CORE] LegacyAccumulatorWrapper should defi...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21229 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21224: [SPARK-24167][SQL] ParquetFilters should not acce...

2018-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21224 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21229: [SPARK-23697][CORE] LegacyAccumulatorWrapper should defi...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21229 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2889/ Tes

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-03 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21193 I think this can be reviewed for now. Thanks. cc @cloud-fan @hvanhovell @kiszk @maropu --- - To unsubscribe, e-mail: reviews-uns

[GitHub] spark issue #21224: [SPARK-24167][SQL] ParquetFilters should not access SQLC...

2018-05-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21224 and branch 2-3 too ..? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: re

[GitHub] spark pull request #20787: [MINOR][DOCS] Documenting months_between directio...

2018-05-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20787#discussion_r185976200 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -887,11 +887,25 @@ object DateTimeUtils { *

[GitHub] spark issue #21224: [SPARK-24167][SQL] ParquetFilters should not access SQLC...

2018-05-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21224 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21229: [SPARK-23697][CORE] LegacyAccumulatorWrapper should defi...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21229 **[Test build #90170 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90170/testReport)** for PR 21229 at commit [`a2d81df`](https://github.com/apache/spark/commit/a2

[GitHub] spark issue #21061: [SPARK-23914][SQL] Add array_union function

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21061 **[Test build #90171 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90171/testReport)** for PR 21061 at commit [`bc2e4c7`](https://github.com/apache/spark/commit/bc

[GitHub] spark issue #20787: [MINOR][DOCS] Documenting months_between direction

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20787 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90167/ Test FAILed. ---

[GitHub] spark issue #20787: [MINOR][DOCS] Documenting months_between direction

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20787 **[Test build #90167 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90167/testReport)** for PR 20787 at commit [`31a126f`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #21231: [SPARK-24119][SQL]Add interpreted execution to SortPrefi...

2018-05-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21231 @hvanhovell @viirya @kiszk --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: re

[GitHub] spark issue #20787: [MINOR][DOCS] Documenting months_between direction

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20787 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21224: [SPARK-24167][SQL] ParquetFilters should not acce...

2018-05-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21224#discussion_r185975883 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -342,6 +342,7 @@ class ParquetFileFor

[GitHub] spark issue #21229: [SPARK-23697][CORE] LegacyAccumulatorWrapper should defi...

2018-05-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21229 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21193 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2888/ Tes

[GitHub] spark issue #21193: [SPARK-24121][SQL] Add API for handling expression code ...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21193 **[Test build #90169 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90169/testReport)** for PR 21193 at commit [`53daaf3`](https://github.com/apache/spark/commit/53

[GitHub] spark issue #21222: [SPARK-24161][SS] Enable debug package feature on struct...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21222 **[Test build #90168 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90168/testReport)** for PR 21222 at commit [`c1ad1c5`](https://github.com/apache/spark/commit/c1

[GitHub] spark pull request #21220: [SPARK-24157][SS] Enabled no-data batches in Micr...

2018-05-03 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21220#discussion_r185975133 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -128,40 +130,49 @@ class MicroBatchExecution(

[GitHub] spark issue #21222: [SPARK-24161][SS] Enable debug package feature on struct...

2018-05-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21222 add to whitelist --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark pull request #21220: [SPARK-24157][SS] Enabled no-data batches in Micr...

2018-05-03 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21220#discussion_r185974949 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -266,93 +276,62 @@ class MicroBatchExecution(

[GitHub] spark pull request #21220: [SPARK-24157][SS] Enabled no-data batches in Micr...

2018-05-03 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21220#discussion_r185974962 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -128,40 +130,49 @@ class MicroBatchExecution(

[GitHub] spark issue #21207: SPARK-24136: Fix MemoryStreamDataReader.next to skip sle...

2018-05-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21207 @arunmahadevan, not a big deal but mind if I ask to fix the PR title to `[SPARK-24136][SS] blabla`? It's actually encouraged in the guide - https://spark.apache.org/contributing.html --- ---

[GitHub] spark issue #20787: [MINOR][DOCS] Documenting months_between direction

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20787 **[Test build #90167 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90167/testReport)** for PR 20787 at commit [`31a126f`](https://github.com/apache/spark/commit/31

[GitHub] spark issue #20787: [MINOR][DOCS] Documenting months_between direction

2018-05-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20787 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10.0.

2018-05-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21070 aha, I see and it might to be true. I'll check the benchmark again. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache

[GitHub] spark pull request #21231: [SPARK-24119][SQL]Add interpreted execution to So...

2018-05-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21231#discussion_r185974124 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala --- @@ -147,7 +148,40 @@ case class SortPrefix(child: SortOr

[GitHub] spark pull request #21231: [SPARK-24119][SQL]Add interpreted execution to So...

2018-05-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21231#discussion_r185973696 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala --- @@ -147,7 +148,40 @@ case class SortPrefix(child: SortOr

[GitHub] spark pull request #21220: [SPARK-24157][SS] Enabled no-data batches in Micr...

2018-05-03 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21220#discussion_r185973467 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/WatermarkTracker.scala --- @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #21231: [SPARK-24119][SQL]Add interpreted execution to So...

2018-05-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21231#discussion_r185973474 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala --- @@ -147,7 +148,40 @@ case class SortPrefix(child: SortOr

[GitHub] spark pull request #21220: [SPARK-24157][SS] Enabled no-data batches in Micr...

2018-05-03 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21220#discussion_r185973062 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -128,40 +130,49 @@ class MicroBatchExecution(

[GitHub] spark pull request #21220: [SPARK-24157][SS] Enabled no-data batches in Micr...

2018-05-03 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21220#discussion_r185972962 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -128,40 +130,49 @@ class MicroBatchExecution(

[GitHub] spark issue #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10.0.

2018-05-03 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21070 @maropu, I suspect that the problem is that comparison is different for strings: `"17297598712"` is less than `"5"` with string comparison. --- -

[GitHub] spark pull request #21208: [SPARK-23925][SQL] Add array_repeat collection fu...

2018-05-03 Thread pepinoflo
Github user pepinoflo commented on a diff in the pull request: https://github.com/apache/spark/pull/21208#discussion_r185972208 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1229,3 +1229,98 @@ case class Flatten(

[GitHub] spark issue #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10.0.

2018-05-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21070 @rdblue Aha, thanks for the explanation! > I think that you were expecting a string comparison case to have a significant benefit over > non->pushdown. But I would only expect that if O

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16478 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2887/ Tes

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16478 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark pull request #21208: [SPARK-23925][SQL] Add array_repeat collection fu...

2018-05-03 Thread pepinoflo
Github user pepinoflo commented on a diff in the pull request: https://github.com/apache/spark/pull/21208#discussion_r185971775 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1229,3 +1229,132 @@ case class Flatten

[GitHub] spark pull request #21218: [SPARK-24155][ML] Instrumentation improvements fo...

2018-05-03 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/21218#discussion_r185970925 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/GaussianMixture.scala --- @@ -423,6 +423,8 @@ class GaussianMixture @Since("2.0.0") (

[GitHub] spark pull request #21225: [SPARK-24168][SQL] WindowExec should not access S...

2018-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21225 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21225: [SPARK-24168][SQL] WindowExec should not access SQLConf ...

2018-05-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21225 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10.0.

2018-05-03 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21070 @gatorsmile, are you happy committing this with the benchmark results? @maropu, thanks for taking the time to add these benchmarks, it is really great to have them so we can monitor the perfo

[GitHub] spark issue #21135: [SPARK-24060][TEST] StreamingSymmetricHashJoinHelperSuit...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21135 **[Test build #90166 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90166/testReport)** for PR 21135 at commit [`dbaa00c`](https://github.com/apache/spark/commit/db

[GitHub] spark issue #21135: [SPARK-24060][TEST] StreamingSymmetricHashJoinHelperSuit...

2018-05-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21135 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10.0.

2018-05-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21070 @rdblue ya, my bad for the simple scan case, you're right. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #21220: [SPARK-24157][SS] Enabled no-data batches in Micr...

2018-05-03 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21220#discussion_r185969637 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/WatermarkTracker.scala --- @@ -0,0 +1,75 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #21225: [SPARK-24168][SQL] WindowExec should not access SQLConf ...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21225 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21225: [SPARK-24168][SQL] WindowExec should not access SQLConf ...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21225 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90158/ Test PASSed. ---

[GitHub] spark issue #21169: [SPARK-23715][SQL] the input of to/from_utc_timestamp ca...

2018-05-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21169 ping @michal-databricks --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: r

[GitHub] spark issue #21225: [SPARK-24168][SQL] WindowExec should not access SQLConf ...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21225 **[Test build #90158 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90158/testReport)** for PR 21225 at commit [`e4151a3`](https://github.com/apache/spark/commit/e

[GitHub] spark pull request #21187: [SPARK-24035][SQL] SQL syntax for Pivot

2018-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21187 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21207: SPARK-24136: Fix MemoryStreamDataReader.next to skip sle...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21207 **[Test build #90165 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90165/testReport)** for PR 21207 at commit [`4daa548`](https://github.com/apache/spark/commit/4d

[GitHub] spark issue #21207: SPARK-24136: Fix MemoryStreamDataReader.next to skip sle...

2018-05-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21207 also ping @zsxwing --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #21207: SPARK-24136: Fix MemoryStreamDataReader.next to skip sle...

2018-05-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21207 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@sp

[GitHub] spark issue #21187: [SPARK-24035][SQL] SQL syntax for Pivot

2018-05-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21187 Thanks for your fast and great work! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21231: [SPARK-24119][SQL]Add interpreted execution to SortPrefi...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21231 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21231: [SPARK-24119][SQL]Add interpreted execution to SortPrefi...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21231 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90159/ Test PASSed. ---

[GitHub] spark issue #21231: [SPARK-24119][SQL]Add interpreted execution to SortPrefi...

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21231 **[Test build #90159 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90159/testReport)** for PR 21231 at commit [`227b6ac`](https://github.com/apache/spark/commit/2

[GitHub] spark pull request #21122: [SPARK-24017] [SQL] Refactor ExternalCatalog to b...

2018-05-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21122#discussion_r185967201 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalog.scala --- @@ -31,10 +30,16 @@ import org.apache.spark.util.L

[GitHub] spark issue #21207: SPARK-24136: Fix MemoryStreamDataReader.next to skip sle...

2018-05-03 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21207 @tdas @jerryshao @HyukjinKwon Kindly ping to trigger test and review. --- - To unsubscribe, e-mail: reviews-unsubscr...@

[GitHub] spark issue #21222: [SPARK-24161][SS] Enable debug package feature on struct...

2018-05-03 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21222 @tdas @jose-torres @jerryshao @arunmahadevan Kindly ping to review. --- - To unsubscribe, e-mail: reviews-unsubscr...@spa

[GitHub] spark pull request #21208: [SPARK-23925][SQL] Add array_repeat collection fu...

2018-05-03 Thread pepinoflo
Github user pepinoflo commented on a diff in the pull request: https://github.com/apache/spark/pull/21208#discussion_r185966094 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1229,3 +1229,98 @@ case class Flatten(

[GitHub] spark issue #21208: [SPARK-23925][SQL] Add array_repeat collection function

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21208 **[Test build #90164 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90164/testReport)** for PR 21208 at commit [`7a8610f`](https://github.com/apache/spark/commit/7a

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21021 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2886/ Tes

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21021 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-05-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21021 **[Test build #90163 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90163/testReport)** for PR 21021 at commit [`e3fcaaa`](https://github.com/apache/spark/commit/e3

[GitHub] spark pull request #21220: [SPARK-24157][SS] Enabled no-data batches in Micr...

2018-05-03 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21220#discussion_r185962542 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -266,93 +276,62 @@ class MicroBatchExecution(

[GitHub] spark pull request #21220: [SPARK-24157][SS] Enabled no-data batches in Micr...

2018-05-03 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21220#discussion_r185962261 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala --- @@ -128,40 +130,49 @@ class MicroBatchExecution(

[GitHub] spark pull request #21208: [SPARK-23925][SQL] Add array_repeat collection fu...

2018-05-03 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21208#discussion_r185962223 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1229,3 +1229,132 @@ case class Flatten(chi

[GitHub] spark issue #21200: [SPARK-24039][SS] Do continuous processing writes with m...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21200 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21208: [SPARK-23925][SQL] Add array_repeat collection fu...

2018-05-03 Thread pepinoflo
Github user pepinoflo commented on a diff in the pull request: https://github.com/apache/spark/pull/21208#discussion_r185962048 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1229,3 +1229,98 @@ case class Flatten(

[GitHub] spark issue #21200: [SPARK-24039][SS] Do continuous processing writes with m...

2018-05-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21200 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90155/ Test PASSed. ---

<    1   2   3   4   5   6   7   8   >