[GitHub] spark pull request #19602: [SPARK-22384][SQL] Refine partition pruning when ...

Github user jinxing64 commented on a diff in the pull request: https://github.com/apache/spark/pull/19602#discussion_r192311969 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/HiveClientSuite.scala --- @@ -207,65 +271,68 @@ class HiveClientSuite(version: String)

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19602 **[Test build #91383 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91383/testReport)** for PR 19602 at commit [`98c2512`](https://github.com/apache/spark/commit/98

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total size of states in HDFSBac...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21469 **[Test build #91382 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91382/testReport)** for PR 21469 at commit [`933fb2e`](https://github.com/apache/spark/commit/93

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total size of states in HDFSBac...

Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21469 Thanks @HyukjinKwon for reviewing. Addressed PR title as well as fixing nit. --- - To unsubscribe, e-mail: reviews-unsubscr..

[GitHub] spark pull request #21472: [SPARK-24445][SQL] Schema in json format for from...

Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21472#discussion_r192309953 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -747,8 +748,13 @@ case class StructsToJson(

[GitHub] spark pull request #21470: [SPARK-24443][SQL] comparison should accept struc...

Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/21470#discussion_r192309419 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2696,16 +2687,6 @@ class SQLQuerySuite extends QueryTest with Shar

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total size of states in HDFSBac...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21469 Shall we make the PR title complete? Looks truncated. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #21469: [SPARK-24441][SS] Expose total size of states in ...

Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21469#discussion_r192308080 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/HDFSBackedStateStoreProvider.scala --- @@ -181,6 +182,12 @@ private[st

[GitHub] spark issue #18826: LogisticRegressionModel.toString should summarize model

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18826 @bravo-zhang, mind if I ask to rebase it and see if the tests pass? BTW, let's fix the PR title to link the JIRA. --- - To u

[GitHub] spark issue #21478: [SPARK-24444][DOCS][PYTHON][BRANCH-2.3] Improve Pandas U...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21478 cc @vanzin FYI. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21478: [SPARK-24444][DOCS][PYTHON][BRANCH-2.3] Improve Pandas U...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21478 Merged to branch-2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #21478: [SPARK-24444][DOCS][PYTHON][BRANCH-2.3] Improve Pandas U...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21478 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91380/ Test PASSed. ---

[GitHub] spark issue #21478: [SPARK-24444][DOCS][PYTHON][BRANCH-2.3] Improve Pandas U...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21478 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21478: [SPARK-24444][DOCS][PYTHON][BRANCH-2.3] Improve Pandas U...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21478 **[Test build #91380 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91380/testReport)** for PR 21478 at commit [`58e7927`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #21475: [SPARK-24416] Fix configuration specification for killBl...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21475 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91381/ Test PASSed. ---

[GitHub] spark issue #21475: [SPARK-24416] Fix configuration specification for killBl...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21475 **[Test build #91381 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91381/testReport)** for PR 21475 at commit [`f08f74a`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #21475: [SPARK-24416] Fix configuration specification for killBl...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21475 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21475: [SPARK-24416] Fix configuration specification for killBl...

Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21475 I think original description is OK, just not too detailed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org F

[GitHub] spark issue #21475: [SPARK-24416] Fix configuration specification for killBl...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21475 **[Test build #91381 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91381/testReport)** for PR 21475 at commit [`f08f74a`](https://github.com/apache/spark/commit/f0

[GitHub] spark issue #21475: [SPARK-24416] Fix configuration specification for killBl...

Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21475 ok to test. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@sp

[GitHub] spark issue #21478: [SPARK-24444][DOCS][PYTHON][BRANCH-2.3] Improve Pandas U...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21478 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21478: [SPARK-24444][DOCS][PYTHON][BRANCH-2.3] Improve Pandas U...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21478 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3753/ Tes

[GitHub] spark issue #21478: [SPARK-24444][DOCS][PYTHON][BRANCH-2.3] Improve Pandas U...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21478 **[Test build #91380 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91380/testReport)** for PR 21478 at commit [`58e7927`](https://github.com/apache/spark/commit/58

[GitHub] spark pull request #21478: [SPARK-24444][DOCS][PYTHON][BRANCH-2.3] Improve P...

GitHub user BryanCutler opened a pull request: https://github.com/apache/spark/pull/21478 [SPARK-2][DOCS][PYTHON][BRANCH-2.3] Improve Pandas UDF docs to explain column assignment ## What changes were proposed in this pull request? Added sections to pandas_udf docs, in the gr

[GitHub] spark issue #18826: LogisticRegressionModel.toString should summarize model

Github user bravo-zhang commented on the issue: https://github.com/apache/spark/pull/18826 This PR recently got tested so it draws my attention. Is this something we want to proceed? @holdenk @yanboliang @jkbradley @dbtsai I don't see how the test failures relate to LogisticRegression

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total size of states in HDFSBac...

Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21469 cc. @tdas @jose-torres @jerryshao @HyukjinKwon @arunmahadevan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91377/ Test PASSed. ---

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19602 **[Test build #91377 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91377/testReport)** for PR 19602 at commit [`8d5057a`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #21069: [SPARK-23920][SQL]add array_remove to remove all element...

Github user huaxingao commented on the issue: https://github.com/apache/spark/pull/21069 Thank you for your help! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: re

[GitHub] spark pull request #21069: [SPARK-23920][SQL]add array_remove to remove all ...

Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21069 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91378/ Test PASSed. ---

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19602 **[Test build #91378 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91378/testReport)** for PR 19602 at commit [`8febfa9`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #21069: [SPARK-23920][SQL]add array_remove to remove all element...

Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21069 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #21471: [SPARK-24444][DOCS][PYTHON] Improve Pandas UDF docs to e...

Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21471 Ya, I can open another pr On Thu, May 31, 2018, 10:01 PM Hyukjin Kwon wrote: > Ooops, there's a conflict. @BryanCutler > can you do it n

[GitHub] spark issue #21471: [SPARK-24444][DOCS][PYTHON] Improve Pandas UDF docs to e...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21471 Ooops, there's a conflict. @BryanCutler can you do it now? Otherwise, I will do in few hours. --- - To unsubscribe, e-mail:

[GitHub] spark issue #21471: [SPARK-24444][DOCS][PYTHON] Improve Pandas UDF docs to e...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21471 Right, let's go ahead then. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mai

[GitHub] spark issue #21471: [SPARK-24444][DOCS][PYTHON] Improve Pandas UDF docs to e...

Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21471 Well, I'd like to get it in 2.3 before the the next RC is cut, which could be pretty soon I think. If by chance we decide the other PR can be backported, I won't mind fixing up the wording

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total size of states in HDFSBac...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21469 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91375/ Test PASSed. ---

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total size of states in HDFSBac...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21469 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21469: [SPARK-24441][SS] Expose total size of states in HDFSBac...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21469 **[Test build #91375 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91375/testReport)** for PR 21469 at commit [`6c1c30b`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #21378: [SPARK-24326][Mesos] add support for local:// sch...

Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21378 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91376/ Test PASSed. ---

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19602 **[Test build #91376 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91376/testReport)** for PR 19602 at commit [`1504c66`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #21471: [SPARK-24444][DOCS][PYTHON] Improve Pandas UDF docs to e...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21471 Makes sense but mind if I do that after we see how it goes in the other PR? Will not forget this one. --- - To unsubscribe,

[GitHub] spark issue #21378: [SPARK-24326][Mesos] add support for local:// scheme for...

Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21378 merged to master. thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21467: [SPARK-23754][PYTHON][FOLLOWUP] Move UDF stop iteration ...

Github user viirya commented on the issue: https://github.com/apache/spark/pull/21467 Looks like not all changes in #21383 are reverted. Can you check it again? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark

[GitHub] spark pull request #21474: [SPARK-24297][CORE] Fetch-to-disk by default for ...

Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21474#discussion_r192295318 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -429,7 +429,11 @@ package object config { "external shu

[GitHub] spark pull request #21476: [SPARK-24446][yarn] Properly quote library path f...

Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21476#discussion_r192295158 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -1485,6 +1486,22 @@ private object Client extends Logg

[GitHub] spark issue #21471: [SPARK-24444][DOCS][PYTHON] Improve Pandas UDF docs to e...

Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21471 Thanks @HyukjinKwon! Can this be merged to branch-2.3? On Thu, May 31, 2018, 9:04 PM asfgit wrote: > Closed #21471 via b2d0226 >

[GitHub] spark issue #21452: [MINOR][CORE] Log committer class used by HadoopMapRedCo...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21452 **[Test build #91379 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91379/testReport)** for PR 21452 at commit [`9881d9c`](https://github.com/apache/spark/commit/98

[GitHub] spark issue #21452: [MINOR][CORE] Log committer class used by HadoopMapRedCo...

Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21452 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mai

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91374/ Test PASSed. ---

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21092 **[Test build #91374 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91374/testReport)** for PR 21092 at commit [`1801e96`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #21471: [SPARK-24444][DOCS][PYTHON] Improve Pandas UDF do...

Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21471 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21471: [SPARK-24444][DOCS][PYTHON] Improve Pandas UDF docs to e...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21471 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #21471: [SPARK-24444][DOCS][PYTHON] Improve Pandas UDF docs to e...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21471 This is in progress. I'm just going to merge this. Let's talk about more in that PR (and fix some if needed). --- - To unsub

[GitHub] spark pull request #21467: [SPARK-23754][PYTHON][FOLLOWUP] Move UDF stop ite...

Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21467#discussion_r192279606 --- Diff: python/pyspark/worker.py --- @@ -140,15 +139,18 @@ def read_single_udf(pickleSer, infile, eval_type): else: row_func =

[GitHub] spark pull request #21467: [SPARK-23754][PYTHON][FOLLOWUP] Move UDF stop ite...

Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21467#discussion_r192284083 --- Diff: python/pyspark/worker.py --- @@ -140,15 +139,18 @@ def read_single_udf(pickleSer, infile, eval_type): else: row_func =

[GitHub] spark pull request #21472: [SPARK-24445][SQL] Schema in json format for from...

Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21472#discussion_r192293089 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -747,8 +748,13 @@ case class StructsToJson

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r192292200 --- Diff: python/pyspark/sql/dataframe.py --- @@ -78,6 +78,7 @@ def __init__(self, jdf, sql_ctx): self.is_cached = False self._

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r192292453 --- Diff: docs/configuration.md --- @@ -456,6 +456,29 @@ Apart from these, the following properties are also available, and may be useful from

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r192292278 --- Diff: docs/configuration.md --- @@ -456,6 +456,29 @@ Apart from these, the following properties are also available, and may be useful from

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r192291854 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -291,37 +310,30 @@ class Dataset[T] private[sql]( } }

[GitHub] spark pull request #21473: [SPARK-21896][SQL] Fix StackOverflow caused by wi...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21473#discussion_r192292014 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1744,11 +1744,14 @@ class Analyzer( *it i

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91373/ Test PASSed. ---

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r192291498 --- Diff: python/pyspark/sql/dataframe.py --- @@ -351,8 +352,70 @@ def show(self, n=20, truncate=True, vertical=False): else: pr

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21092 **[Test build #91373 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91373/testReport)** for PR 21092 at commit [`7bedeb6`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3752/ Tes

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3751/ Tes

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19602 **[Test build #91378 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91378/testReport)** for PR 19602 at commit [`8febfa9`](https://github.com/apache/spark/commit/8f

[GitHub] spark pull request #19602: [SPARK-22384][SQL] Refine partition pruning when ...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19602#discussion_r192286696 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/HiveClientSuite.scala --- @@ -74,23 +86,23 @@ class HiveClientSuite(version: String)

[GitHub] spark pull request #19602: [SPARK-22384][SQL] Refine partition pruning when ...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19602#discussion_r192286502 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/client/HiveClientSuite.scala --- @@ -66,6 +65,19 @@ class HiveClientSuite(version: String)

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19602 **[Test build #91377 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91377/testReport)** for PR 19602 at commit [`8d5057a`](https://github.com/apache/spark/commit/8d

[GitHub] spark pull request #19602: [SPARK-22384][SQL] Refine partition pruning when ...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19602#discussion_r192286303 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala --- @@ -657,18 +656,30 @@ private[client] class Shim_v0_13 extends Shim_

[GitHub] spark issue #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured Stream...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21477 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91371/ Test FAILed. ---

[GitHub] spark issue #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured Stream...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21477 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21428: [SPARK-24235][SS] Implement continuous shuffle writer fo...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21428 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21477: [WIP] [SPARK-24396] [SS] [PYSPARK] Add Structured Stream...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21477 **[Test build #91371 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91371/testReport)** for PR 21477 at commit [`701a455`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #21428: [SPARK-24235][SS] Implement continuous shuffle writer fo...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21428 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91368/ Test PASSed. ---

[GitHub] spark issue #21069: [SPARK-23920][SQL]add array_remove to remove all element...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21069 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21069: [SPARK-23920][SQL]add array_remove to remove all element...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21069 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91369/ Test PASSed. ---

[GitHub] spark issue #21428: [SPARK-24235][SS] Implement continuous shuffle writer fo...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21428 **[Test build #91368 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91368/testReport)** for PR 21428 at commit [`629455b`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #21069: [SPARK-23920][SQL]add array_remove to remove all element...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21069 **[Test build #91369 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91369/testReport)** for PR 21069 at commit [`52d2308`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3750/ Tes

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21473: [SPARK-21896][SQL] Fix StackOverflow caused by wi...

Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21473#discussion_r192284247 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala --- @@ -687,4 +687,29 @@ class DataFrameAggregateSuite extends QueryT

[GitHub] spark issue #21474: [SPARK-24297][CORE] Fetch-to-disk by default for > 2gb

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21474 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21474: [SPARK-24297][CORE] Fetch-to-disk by default for > 2gb

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21474 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91365/ Test PASSed. ---

[GitHub] spark issue #21474: [SPARK-24297][CORE] Fetch-to-disk by default for > 2gb

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21474 **[Test build #91365 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91365/testReport)** for PR 21474 at commit [`6b7c4f7`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #21381: [SPARK-24330][SQL]Refactor ExecuteWriteTask and U...

Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21381 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19602 **[Test build #91376 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91376/testReport)** for PR 19602 at commit [`1504c66`](https://github.com/apache/spark/commit/15

[GitHub] spark issue #21381: [SPARK-24330][SQL]Refactor ExecuteWriteTask and Use `whi...

Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21381 +1 for moving these write tasks to an individual file and abstract more common code, merging to master! --- - To unsubscribe,

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r192282041 --- Diff: python/pyspark/sql/dataframe.py --- @@ -351,8 +352,70 @@ def show(self, n=20, truncate=True, vertical=False): else:

  1   2   3   4   5   6   >