[GitHub] [spark] SparkQA commented on issue #23984: [SPARK-24360][FOLLOW-UP][SQL] Fix NoSuchFieldError when read Hive materialized views

SparkQA commented on issue #23984: [SPARK-24360][FOLLOW-UP][SQL] Fix NoSuchFieldError when read Hive materialized views URL: https://github.com/apache/spark/pull/23984#issuecomment-469954333 **[Test build #103076 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuild

[GitHub] [spark] AmplabJenkins removed a comment on issue #23871: [SPARK-23433][SPARK-25250] [CORE] Later created TaskSet should learn about the finished partitions

AmplabJenkins removed a comment on issue #23871: [SPARK-23433][SPARK-25250] [CORE] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/23871#issuecomment-469954063 Test PASSed. Refer to this link for build results (access rights to

[GitHub] [spark] AmplabJenkins commented on issue #23871: [SPARK-23433][SPARK-25250] [CORE] Later created TaskSet should learn about the finished partitions

AmplabJenkins commented on issue #23871: [SPARK-23433][SPARK-25250] [CORE] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/23871#issuecomment-469954063 Test PASSed. Refer to this link for build results (access rights to CI serve

[GitHub] [spark] AmplabJenkins commented on issue #23984: [SPARK-24360][FOLLOW-UP][SQL] Fix NoSuchFieldError when read Hive materialized views

AmplabJenkins commented on issue #23984: [SPARK-24360][FOLLOW-UP][SQL] Fix NoSuchFieldError when read Hive materialized views URL: https://github.com/apache/spark/pull/23984#issuecomment-469954047 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #23871: [SPARK-23433][SPARK-25250] [CORE] Later created TaskSet should learn about the finished partitions

AmplabJenkins removed a comment on issue #23871: [SPARK-23433][SPARK-25250] [CORE] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/23871#issuecomment-469954060 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins commented on issue #23871: [SPARK-23433][SPARK-25250] [CORE] Later created TaskSet should learn about the finished partitions

AmplabJenkins commented on issue #23871: [SPARK-23433][SPARK-25250] [CORE] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/23871#issuecomment-469954060 Merged build finished. Test PASSed. --

[GitHub] [spark] SparkQA commented on issue #23871: [SPARK-23433][SPARK-25250] [CORE] Later created TaskSet should learn about the finished partitions

SparkQA commented on issue #23871: [SPARK-23433][SPARK-25250] [CORE] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/23871#issuecomment-469953177 **[Test build #103075 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkP

[GitHub] [spark] wangyum opened a new pull request #23984: [SPARK-24360][FOLLOW-UP][SQL] Fix NoSuchFieldError when read Hive materialized views

wangyum opened a new pull request #23984: [SPARK-24360][FOLLOW-UP][SQL] Fix NoSuchFieldError when read Hive materialized views URL: https://github.com/apache/spark/pull/23984 ## What changes were proposed in this pull request? This pr fix `NoSuchFieldError` when reading Hive material

[GitHub] [spark] SparkQA commented on issue #23943: [SPARK-27034][SQL] Nested schema pruning for ORC

SparkQA commented on issue #23943: [SPARK-27034][SQL] Nested schema pruning for ORC URL: https://github.com/apache/spark/pull/23943#issuecomment-469953158 **[Test build #103074 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103074/testReport)** for PR 2394

[GitHub] [spark] Ngone51 commented on a change in pull request #23871: [SPARK-23433][SPARK-25250] [CORE] Later created TaskSet should learn about the finished partitions

Ngone51 commented on a change in pull request #23871: [SPARK-23433][SPARK-25250] [CORE] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/23871#discussion_r262779778 ## File path: core/src/test/scala/org/apache/spark/schedu

[GitHub] [spark] SparkQA commented on issue #23932: [SPARK-27026] [Build] Upgrade Docker image for release build to Ubuntu 18.04LTS

SparkQA commented on issue #23932: [SPARK-27026] [Build] Upgrade Docker image for release build to Ubuntu 18.04LTS URL: https://github.com/apache/spark/pull/23932#issuecomment-469953130 **[Test build #103068 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/1

[GitHub] [spark] AmplabJenkins removed a comment on issue #23943: [SPARK-27034][SQL] Nested schema pruning for ORC

AmplabJenkins removed a comment on issue #23943: [SPARK-27034][SQL] Nested schema pruning for ORC URL: https://github.com/apache/spark/pull/23943#issuecomment-469952805 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] Ngone51 commented on a change in pull request #23871: [SPARK-23433][SPARK-25250] [CORE] Later created TaskSet should learn about the finished partitions

Ngone51 commented on a change in pull request #23871: [SPARK-23433][SPARK-25250] [CORE] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/23871#discussion_r262779599 ## File path: core/src/main/scala/org/apache/spark/schedul

[GitHub] [spark] AmplabJenkins commented on issue #23943: [SPARK-27034][SQL] Nested schema pruning for ORC

AmplabJenkins commented on issue #23943: [SPARK-27034][SQL] Nested schema pruning for ORC URL: https://github.com/apache/spark/pull/23943#issuecomment-469952805 Merged build finished. Test PASSed. This is an automated message

[GitHub] [spark] AmplabJenkins removed a comment on issue #23943: [SPARK-27034][SQL] Nested schema pruning for ORC

AmplabJenkins removed a comment on issue #23943: [SPARK-27034][SQL] Nested schema pruning for ORC URL: https://github.com/apache/spark/pull/23943#issuecomment-469952808 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.

[GitHub] [spark] AmplabJenkins commented on issue #23943: [SPARK-27034][SQL] Nested schema pruning for ORC

AmplabJenkins commented on issue #23943: [SPARK-27034][SQL] Nested schema pruning for ORC URL: https://github.com/apache/spark/pull/23943#issuecomment-469952808 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenk

[GitHub] [spark] SparkQA commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

SparkQA commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469951897 **[Test build #103073 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103073/

[GitHub] [spark] AmplabJenkins removed a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

AmplabJenkins removed a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469951415 Test PASSed. Refer to this link for build results (access rights to CI server needed): h

[GitHub] [spark] AmplabJenkins commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

AmplabJenkins commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469951415 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://a

[GitHub] [spark] AmplabJenkins removed a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

AmplabJenkins removed a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469951410 Merged build finished. Test PASSed. ---

[GitHub] [spark] AmplabJenkins commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

AmplabJenkins commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469951410 Merged build finished. Test PASSed. Th

[GitHub] [spark] dilipbiswal edited a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

dilipbiswal edited a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469927252 @maropu Thank you. Very few systems i could find that expose the semi and anti join types to users

[GitHub] [spark] hddong commented on issue #23837: [SPARK-26929][SQL]fix table owner use user instead of principal when create table through spark-sql or beeline

hddong commented on issue #23837: [SPARK-26929][SQL]fix table owner use user instead of principal when create table through spark-sql or beeline URL: https://github.com/apache/spark/pull/23837#issuecomment-469948528 @HyukjinKwon , could you help to review this again at new PR https://gith

[GitHub] [spark] AmplabJenkins removed a comment on issue #23927: [SPARK-27065][CORE] avoid more than one active task set managers for a stage

AmplabJenkins removed a comment on issue #23927: [SPARK-27065][CORE] avoid more than one active task set managers for a stage URL: https://github.com/apache/spark/pull/23927#issuecomment-469946878 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #23927: [SPARK-27065][CORE] avoid more than one active task set managers for a stage

AmplabJenkins removed a comment on issue #23927: [SPARK-27065][CORE] avoid more than one active task set managers for a stage URL: https://github.com/apache/spark/pull/23927#issuecomment-469946874 Merged build finished. Test PASSed. -

[GitHub] [spark] AmplabJenkins commented on issue #23927: [SPARK-27065][CORE] avoid more than one active task set managers for a stage

AmplabJenkins commented on issue #23927: [SPARK-27065][CORE] avoid more than one active task set managers for a stage URL: https://github.com/apache/spark/pull/23927#issuecomment-469946878 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://

[GitHub] [spark] AmplabJenkins commented on issue #23927: [SPARK-27065][CORE] avoid more than one active task set managers for a stage

AmplabJenkins commented on issue #23927: [SPARK-27065][CORE] avoid more than one active task set managers for a stage URL: https://github.com/apache/spark/pull/23927#issuecomment-469946874 Merged build finished. Test PASSed.

[GitHub] [spark] SparkQA commented on issue #23927: [SPARK-27065][CORE] avoid more than one active task set managers for a stage

SparkQA commented on issue #23927: [SPARK-27065][CORE] avoid more than one active task set managers for a stage URL: https://github.com/apache/spark/pull/23927#issuecomment-469946488 **[Test build #103069 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/1030

[GitHub] [spark] SparkQA removed a comment on issue #23927: [SPARK-27065][CORE] avoid more than one active task set managers for a stage

SparkQA removed a comment on issue #23927: [SPARK-27065][CORE] avoid more than one active task set managers for a stage URL: https://github.com/apache/spark/pull/23927#issuecomment-469871064 **[Test build #103069 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuild

[GitHub] [spark] AmplabJenkins removed a comment on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system

AmplabJenkins removed a comment on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system URL: https://github.com/apache/spark/pull/23915#issuecomment-469945259 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.ed

[GitHub] [spark] AmplabJenkins removed a comment on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system

AmplabJenkins removed a comment on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system URL: https://github.com/apache/spark/pull/23915#issuecomment-469945256 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] AmplabJenkins commented on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system

AmplabJenkins commented on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system URL: https://github.com/apache/spark/pull/23915#issuecomment-469945259 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkin

[GitHub] [spark] AmplabJenkins commented on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system

AmplabJenkins commented on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system URL: https://github.com/apache/spark/pull/23915#issuecomment-469945256 Merged build finished. Test PASSed. This is an automated message

[GitHub] [spark] SparkQA removed a comment on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system

SparkQA removed a comment on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system URL: https://github.com/apache/spark/pull/23915#issuecomment-469885055 **[Test build #103071 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103071/testReport)** for

[GitHub] [spark] SparkQA commented on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system

SparkQA commented on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system URL: https://github.com/apache/spark/pull/23915#issuecomment-469944819 **[Test build #103071 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103071/testReport)** for PR 2391

[GitHub] [spark] srowen commented on issue #23931: [SPARK-26048][build] Enable flume profile when creating 2.x releases.

srowen commented on issue #23931: [SPARK-26048][build] Enable flume profile when creating 2.x releases. URL: https://github.com/apache/spark/pull/23931#issuecomment-469941576 Locally, or in the bundle that was pushed to Sonatype, do you see the hash / signature files for this module, or ot

[GitHub] [spark] jagadesh-kiran commented on a change in pull request #23946: [SPARK-26860][PySpark] [SparkR] Fix for RangeBetween and RowsBetween docs to be in sync with spark documentation

jagadesh-kiran commented on a change in pull request #23946: [SPARK-26860][PySpark] [SparkR] Fix for RangeBetween and RowsBetween docs to be in sync with spark documentation URL: https://github.com/apache/spark/pull/23946#discussion_r262769820 ## File path: python/pyspark/sql/windo

[GitHub] [spark] HeartSaVioR commented on a change in pull request #23956: [SPARK-27042][SS] Close cached Kafka producer in case of task retry

HeartSaVioR commented on a change in pull request #23956: [SPARK-27042][SS] Close cached Kafka producer in case of task retry URL: https://github.com/apache/spark/pull/23956#discussion_r262768038 ## File path: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/Cac

[GitHub] [spark] 10110346 commented on issue #23975: [SPARK-27056][MESOS]Remove start-shuffle-service.sh

10110346 commented on issue #23975: [SPARK-27056][MESOS]Remove start-shuffle-service.sh URL: https://github.com/apache/spark/pull/23975#issuecomment-469935278 > > > Isn't this used by standalone? I admit I haven't kept up with how the shuffle service is or isn't launched by variou

[GitHub] [spark] haiboself commented on a change in pull request #23944: [SPARK-26932][DOC] Add a warning for Hive 2.1.1 ORC reader issue

haiboself commented on a change in pull request #23944: [SPARK-26932][DOC] Add a warning for Hive 2.1.1 ORC reader issue URL: https://github.com/apache/spark/pull/23944#discussion_r262759756 ## File path: docs/sql-migration-guide-upgrade.md ## @@ -169,7 +169,7 @@ displayTi

[GitHub] [spark] dilipbiswal commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

dilipbiswal commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469927252 @maropu Thank you. Very few systems i could find that expose the semi and anti join types to users. I che

[GitHub] [spark] AmplabJenkins removed a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

AmplabJenkins removed a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469925218 Test FAILed. Refer to this link for build results (access rights to CI server needed): ht

[GitHub] [spark] dbtsai commented on issue #23931: [SPARK-26048][build] Enable flume profile when creating 2.x releases.

dbtsai commented on issue #23931: [SPARK-26048][build] Enable flume profile when creating 2.x releases. URL: https://github.com/apache/spark/pull/23931#issuecomment-469925630 It's weird that with flume enabled, I got an error on signature validation on flume test jar when I published the j

[GitHub] [spark] AmplabJenkins commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

AmplabJenkins commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469925218 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://am

[GitHub] [spark] AmplabJenkins commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

AmplabJenkins commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469925215 Merged build finished. Test FAILed. Th

[GitHub] [spark] AmplabJenkins removed a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

AmplabJenkins removed a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469925215 Merged build finished. Test FAILed. ---

[GitHub] [spark] SparkQA removed a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

SparkQA removed a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469878347 **[Test build #103070 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder

[GitHub] [spark] SparkQA commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

SparkQA commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469925020 **[Test build #103070 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103070

[GitHub] [spark] nikitagkonda commented on a change in pull request #21149: [SPARK-24076][SQL] Use different seed in HashAggregate to avoid hash conflict

nikitagkonda commented on a change in pull request #21149: [SPARK-24076][SQL] Use different seed in HashAggregate to avoid hash conflict URL: https://github.com/apache/spark/pull/21149#discussion_r262755515 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggreg

[GitHub] [spark] maropu commented on issue #23748: [SPARK-23619][SQL] Added default column names for explode & posexplode in comments

maropu commented on issue #23748: [SPARK-23619][SQL] Added default column names for explode & posexplode in comments URL: https://github.com/apache/spark/pull/23748#issuecomment-469921751 Could you update the PR description; plz list up all the function you addressed in this PR? -

[GitHub] [spark] maropu commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

maropu commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469920751 Any other dbms-like system making this optional? -

[GitHub] [spark] mwlon edited a comment on issue #23967: [SPARK-27015] [Mesos] properly escape mesos scheduler arguments

mwlon edited a comment on issue #23967: [SPARK-27015] [Mesos] properly escape mesos scheduler arguments URL: https://github.com/apache/spark/pull/23967#issuecomment-469917584 Master only is what I would recommend anyway. Thanks! -

[GitHub] [spark] mwlon commented on issue #23967: [SPARK-27015] [Mesos] properly escape mesos scheduler arguments

mwlon commented on issue #23967: [SPARK-27015] [Mesos] properly escape mesos scheduler arguments URL: https://github.com/apache/spark/pull/23967#issuecomment-469917584 Master only is what I would recommend anyway This is an a

[GitHub] [spark] AmplabJenkins removed a comment on issue #23939: [SPARK-27019] [SQL][WEBUI]onJobStart happens after onExecutionEnd shouldn't overwrite kvstore

AmplabJenkins removed a comment on issue #23939: [SPARK-27019] [SQL][WEBUI]onJobStart happens after onExecutionEnd shouldn't overwrite kvstore URL: https://github.com/apache/spark/pull/23939#issuecomment-469912300 Test PASSed. Refer to this link for build results (access rights to CI ser

[GitHub] [spark] AmplabJenkins removed a comment on issue #23939: [SPARK-27019] [SQL][WEBUI]onJobStart happens after onExecutionEnd shouldn't overwrite kvstore

AmplabJenkins removed a comment on issue #23939: [SPARK-27019] [SQL][WEBUI]onJobStart happens after onExecutionEnd shouldn't overwrite kvstore URL: https://github.com/apache/spark/pull/23939#issuecomment-469912294 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #23939: [SPARK-27019] [SQL][WEBUI]onJobStart happens after onExecutionEnd shouldn't overwrite kvstore

AmplabJenkins commented on issue #23939: [SPARK-27019] [SQL][WEBUI]onJobStart happens after onExecutionEnd shouldn't overwrite kvstore URL: https://github.com/apache/spark/pull/23939#issuecomment-469912300 Test PASSed. Refer to this link for build results (access rights to CI server need

[GitHub] [spark] AmplabJenkins commented on issue #23939: [SPARK-27019] [SQL][WEBUI]onJobStart happens after onExecutionEnd shouldn't overwrite kvstore

AmplabJenkins commented on issue #23939: [SPARK-27019] [SQL][WEBUI]onJobStart happens after onExecutionEnd shouldn't overwrite kvstore URL: https://github.com/apache/spark/pull/23939#issuecomment-469912294 Merged build finished. Test PASSed.

[GitHub] [spark] SparkQA removed a comment on issue #23939: [SPARK-27019] [SQL][WEBUI]onJobStart happens after onExecutionEnd shouldn't overwrite kvstore

SparkQA removed a comment on issue #23939: [SPARK-27019] [SQL][WEBUI]onJobStart happens after onExecutionEnd shouldn't overwrite kvstore URL: https://github.com/apache/spark/pull/23939#issuecomment-469831277 **[Test build #103067 has started](https://amplab.cs.berkeley.edu/jenkins/job/Spar

[GitHub] [spark] AmplabJenkins removed a comment on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition

AmplabJenkins removed a comment on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition URL: https://github.com/apache/spark/pull/23964#issuecomment-469911751 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition

AmplabJenkins removed a comment on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition URL: https://github.com/apache/spark/pull/23964#issuecomment-469911748 Merged build finished. Test PASSed. ---

[GitHub] [spark] SparkQA commented on issue #23939: [SPARK-27019] [SQL][WEBUI]onJobStart happens after onExecutionEnd shouldn't overwrite kvstore

SparkQA commented on issue #23939: [SPARK-27019] [SQL][WEBUI]onJobStart happens after onExecutionEnd shouldn't overwrite kvstore URL: https://github.com/apache/spark/pull/23939#issuecomment-469911879 **[Test build #103067 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRe

[GitHub] [spark] AmplabJenkins commented on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition

AmplabJenkins commented on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition URL: https://github.com/apache/spark/pull/23964#issuecomment-469911751 Test PASSed. Refer to this link for build results (access rights to CI server needed): https:

[GitHub] [spark] AmplabJenkins commented on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition

AmplabJenkins commented on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition URL: https://github.com/apache/spark/pull/23964#issuecomment-469911748 Merged build finished. Test PASSed. ---

[GitHub] [spark] SparkQA removed a comment on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition

SparkQA removed a comment on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition URL: https://github.com/apache/spark/pull/23964#issuecomment-469831264 **[Test build #103066 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBui

[GitHub] [spark] SparkQA commented on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition

SparkQA commented on issue #23964: [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition URL: https://github.com/apache/spark/pull/23964#issuecomment-469911337 **[Test build #103066 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/10

[GitHub] [spark] dilipbiswal edited a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

dilipbiswal edited a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469910613 @dongjoon-hyun Thanks.. I totally understand :-). So here are the options : 1) Do nothing 2)

[GitHub] [spark] dilipbiswal commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

dilipbiswal commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469910613 @dongjoon-hyun Thanks.. I totally understand :-). So here are the options : 1) Do nothing 2) Make L

[GitHub] [spark] dongjoon-hyun commented on issue #23958: [SPARK-27045] SQL tab in UI shows callsite instead of actual SQL

dongjoon-hyun commented on issue #23958: [SPARK-27045] SQL tab in UI shows callsite instead of actual SQL URL: https://github.com/apache/spark/pull/23958#issuecomment-469909914 Hi, @ajithme . This is an improvement instead of a bug fix. Could you update the title of this PR and JIRA?

[GitHub] [spark] HyukjinKwon closed pull request #23971: [SPARK-26922][R] Set socket timeout consistently in Arrow optimization

HyukjinKwon closed pull request #23971: [SPARK-26922][R] Set socket timeout consistently in Arrow optimization URL: https://github.com/apache/spark/pull/23971 This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] AmplabJenkins removed a comment on issue #23978: [MINOR] Spelling mistake: forword -> forward

AmplabJenkins removed a comment on issue #23978: [MINOR] Spelling mistake: forword -> forward URL: https://github.com/apache/spark/pull/23978#issuecomment-469908942 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/

[GitHub] [spark] AmplabJenkins removed a comment on issue #23978: [MINOR] Spelling mistake: forword -> forward

AmplabJenkins removed a comment on issue #23978: [MINOR] Spelling mistake: forword -> forward URL: https://github.com/apache/spark/pull/23978#issuecomment-469908937 Merged build finished. Test PASSed. This is an automated mes

[GitHub] [spark] AmplabJenkins commented on issue #23978: [MINOR] Spelling mistake: forword -> forward

AmplabJenkins commented on issue #23978: [MINOR] Spelling mistake: forword -> forward URL: https://github.com/apache/spark/pull/23978#issuecomment-469908937 Merged build finished. Test PASSed. This is an automated message fro

[GitHub] [spark] AmplabJenkins commented on issue #23978: [MINOR] Spelling mistake: forword -> forward

AmplabJenkins commented on issue #23978: [MINOR] Spelling mistake: forword -> forward URL: https://github.com/apache/spark/pull/23978#issuecomment-469908942 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/

[GitHub] [spark] HyukjinKwon commented on issue #23971: [SPARK-26922][R] Set socket timeout consistently in Arrow optimization

HyukjinKwon commented on issue #23971: [SPARK-26922][R] Set socket timeout consistently in Arrow optimization URL: https://github.com/apache/spark/pull/23971#issuecomment-469908752 Merged to master. This is an automated messa

[GitHub] [spark] HyukjinKwon commented on issue #23971: [SPARK-26922][R] Set socket timeout consistently in Arrow optimization

HyukjinKwon commented on issue #23971: [SPARK-26922][R] Set socket timeout consistently in Arrow optimization URL: https://github.com/apache/spark/pull/23971#issuecomment-469908693 Let me just get this in. I am pretty sure this is the only last instance left. -

[GitHub] [spark] HyukjinKwon commented on issue #23900: [SPARK-23836][PYTHON] Add support for StructType return in Scalar Pandas UDF

HyukjinKwon commented on issue #23900: [SPARK-23836][PYTHON] Add support for StructType return in Scalar Pandas UDF URL: https://github.com/apache/spark/pull/23900#issuecomment-469908213 Yea, no objection. I will take a look when I find some time whether it's merged or not anyway. ---

[GitHub] [spark] SparkQA commented on issue #23978: [MINOR] Spelling mistake: forword -> forward

SparkQA commented on issue #23978: [MINOR] Spelling mistake: forword -> forward URL: https://github.com/apache/spark/pull/23978#issuecomment-469907834 **[Test build #103072 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103072/testReport)** for PR 23978 at

[GitHub] [spark] dongjoon-hyun edited a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

dongjoon-hyun edited a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469907224 Ur, I understand the intention, but this is a breaking change which may make the existing users

[GitHub] [spark] HyukjinKwon commented on issue #23978: [MINOR] Spelling mistake: forword -> forward

HyukjinKwon commented on issue #23978: [MINOR] Spelling mistake: forword -> forward URL: https://github.com/apache/spark/pull/23978#issuecomment-469907657 retest this please This is an automated message from the Apache Git Se

[GitHub] [spark] dongjoon-hyun commented on issue #23978: [MINOR] Spelling mistake: forword -> forward

dongjoon-hyun commented on issue #23978: [MINOR] Spelling mistake: forword -> forward URL: https://github.com/apache/spark/pull/23978#issuecomment-469907647 Retest this please. This is an automated message from the Apache Git

[GitHub] [spark] HyukjinKwon commented on issue #23978: [MINOR] Spelling mistake: forword -> forward

HyukjinKwon commented on issue #23978: [MINOR] Spelling mistake: forword -> forward URL: https://github.com/apache/spark/pull/23978#issuecomment-469907690 Oops This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] dongjoon-hyun commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

dongjoon-hyun commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469907224 Ur, I understand the intention, but this is a breaking change which may make the existing users upset w

[GitHub] [spark] dongjoon-hyun edited a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

dongjoon-hyun edited a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-469907224 Ur, I understand the intention, but this is a breaking change which may make the existing users

[GitHub] [spark] HyukjinKwon commented on a change in pull request #23946: [SPARK-26860][PySpark] [SparkR] Fix for RangeBetween and RowsBetween docs to be in sync with spark documentation

HyukjinKwon commented on a change in pull request #23946: [SPARK-26860][PySpark] [SparkR] Fix for RangeBetween and RowsBetween docs to be in sync with spark documentation URL: https://github.com/apache/spark/pull/23946#discussion_r262740577 ## File path: python/pyspark/sql/window.p

[GitHub] [spark] dilipbiswal commented on a change in pull request #23905: [SPARK-24669][SQL] Refresh table before drop database cascade

dilipbiswal commented on a change in pull request #23905: [SPARK-24669][SQL] Refresh table before drop database cascade URL: https://github.com/apache/spark/pull/23905#discussion_r262728934 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatal

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #23905: [SPARK-24669][SQL] Refresh table before drop database cascade

dongjoon-hyun commented on a change in pull request #23905: [SPARK-24669][SQL] Refresh table before drop database cascade URL: https://github.com/apache/spark/pull/23905#discussion_r262728476 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCat

[GitHub] [spark] gagafunctor commented on issue #23983: [SPARK-26881][core] Heuristic for tree aggregate depth

gagafunctor commented on issue #23983: [SPARK-26881][core] Heuristic for tree aggregate depth URL: https://github.com/apache/spark/pull/23983#issuecomment-469894473 I think I messed up with scalafmt. Will fix that tomorrow and clean the PR. -

[GitHub] [spark] AmplabJenkins removed a comment on issue #23983: [SPARK-26881][core] Heuristic for tree aggregate depth

AmplabJenkins removed a comment on issue #23983: [SPARK-26881][core] Heuristic for tree aggregate depth URL: https://github.com/apache/spark/pull/23983#issuecomment-469892999 Can one of the admins verify this patch? This is a

[GitHub] [spark] AmplabJenkins commented on issue #23983: [SPARK-26881][core] Heuristic for tree aggregate depth

AmplabJenkins commented on issue #23983: [SPARK-26881][core] Heuristic for tree aggregate depth URL: https://github.com/apache/spark/pull/23983#issuecomment-469893623 Can one of the admins verify this patch? This is an automa

[GitHub] [spark] AmplabJenkins removed a comment on issue #23983: [SPARK-26881][core] Heuristic for tree aggregate depth

AmplabJenkins removed a comment on issue #23983: [SPARK-26881][core] Heuristic for tree aggregate depth URL: https://github.com/apache/spark/pull/23983#issuecomment-469892857 Can one of the admins verify this patch? This is a

[GitHub] [spark] AmplabJenkins commented on issue #23983: [SPARK-26881][core] Heuristic for tree aggregate depth

AmplabJenkins commented on issue #23983: [SPARK-26881][core] Heuristic for tree aggregate depth URL: https://github.com/apache/spark/pull/23983#issuecomment-469892999 Can one of the admins verify this patch? This is an automa

[GitHub] [spark] AmplabJenkins commented on issue #23983: [SPARK-26881][core] Heuristic for tree aggregate depth

AmplabJenkins commented on issue #23983: [SPARK-26881][core] Heuristic for tree aggregate depth URL: https://github.com/apache/spark/pull/23983#issuecomment-469892857 Can one of the admins verify this patch? This is an automa

[GitHub] [spark] gagafunctor opened a new pull request #23983: [SPARK-26881][core] Heuristic for tree aggregate depth

gagafunctor opened a new pull request #23983: [SPARK-26881][core] Heuristic for tree aggregate depth URL: https://github.com/apache/spark/pull/23983 Changes proposed: - Adding method to compute treeAggregate depth required to avoid exceeding driver max result size (first commit) - Us

[GitHub] [spark] AmplabJenkins removed a comment on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system

AmplabJenkins removed a comment on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system URL: https://github.com/apache/spark/pull/23915#issuecomment-469886689 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.e

[GitHub] [spark] AmplabJenkins removed a comment on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system

AmplabJenkins removed a comment on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system URL: https://github.com/apache/spark/pull/23915#issuecomment-469886682 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] AmplabJenkins commented on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system

AmplabJenkins commented on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system URL: https://github.com/apache/spark/pull/23915#issuecomment-469886689 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenki

[GitHub] [spark] AmplabJenkins commented on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system

AmplabJenkins commented on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system URL: https://github.com/apache/spark/pull/23915#issuecomment-469886682 Merged build finished. Test PASSed. This is an automated message

[GitHub] [spark] SparkQA commented on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system

SparkQA commented on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system URL: https://github.com/apache/spark/pull/23915#issuecomment-469885055 **[Test build #103071 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103071/testReport)** for PR 23915

[GitHub] [spark] rdblue commented on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system

rdblue commented on issue #23915: [SPARK-24252][SQL] Add v2 catalog plugin system URL: https://github.com/apache/spark/pull/23915#issuecomment-469884489 Retest this please. This is an automated message from the Apache Git Ser

[GitHub] [spark] wypoon commented on a change in pull request #23767: [SPARK-26329][CORE][WIP] Faster polling of executor memory metrics.

wypoon commented on a change in pull request #23767: [SPARK-26329][CORE][WIP] Faster polling of executor memory metrics. URL: https://github.com/apache/spark/pull/23767#discussion_r262716263 ## File path: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ##

<    1   2   3   4   5   6   7   8   9   >