[GitHub] [spark] imback82 commented on a change in pull request #32854: [SPARK-34320][SQL] Migrate ALTER TABLE DROP COLUMNS commands to use UnresolvedTable to resolve the identifier

2021-06-10 Thread GitBox
imback82 commented on a change in pull request #32854: URL: https://github.com/apache/spark/pull/32854#discussion_r648877170 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -298,6 +298,7 @@ class Analyzer(override val

[GitHub] [spark] imback82 commented on pull request #32854: [SPARK-34320][SQL] Migrate ALTER TABLE DROP COLUMNS commands to use UnresolvedTable to resolve the identifier

2021-06-10 Thread GitBox
imback82 commented on pull request #32854: URL: https://github.com/apache/spark/pull/32854#issuecomment-858337156 @cloud-fan this is an alternative approach suggested in https://github.com/apache/spark/pull/32542#issuecomment-843759217. We can migrate the alter table commands one by one

[GitHub] [spark] imback82 opened a new pull request #32855: [SPARK-34524][SQL][FOLLOWUP] Remove unused checkAlterTablePartition in CheckAnalysis.scala

2021-06-10 Thread GitBox
imback82 opened a new pull request #32855: URL: https://github.com/apache/spark/pull/32855 ### What changes were proposed in this pull request? #31637 removed the usage of `CheckAnalysis.checkAlterTablePartition` but didn't remove the function. ### Why are the changes

[GitHub] [spark] SparkQA commented on pull request #32815: [SPARK-35675][SQL] EnsureRequirements remove shuffle should respect PartitioningCollection

2021-06-10 Thread GitBox
SparkQA commented on pull request #32815: URL: https://github.com/apache/spark/pull/32815#issuecomment-858353315 **[Test build #139605 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139605/testReport)** for PR 32815 at commit

[GitHub] [spark] Ngone51 commented on pull request #32136: [SPARK-35022][CORE] Task Scheduling Plugin in Spark

2021-06-10 Thread GitBox
Ngone51 commented on pull request #32136: URL: https://github.com/apache/spark/pull/32136#issuecomment-858375487 > The mapping could be between specific resources (e.g. PVC) and task (i.e. state store). Your rephrase looks good except for one point here. "task (i.e. state

[GitHub] [spark] SparkQA commented on pull request #32857: [SPARK-35707][ML] optimize sparse GEMM by skipping bound checking

2021-06-10 Thread GitBox
SparkQA commented on pull request #32857: URL: https://github.com/apache/spark/pull/32857#issuecomment-858386706 **[Test build #139622 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139622/testReport)** for PR 32857 at commit

[GitHub] [spark] SparkQA commented on pull request #32852: [SPARK-35283][SQL] Support query some DDL with CTES

2021-06-10 Thread GitBox
SparkQA commented on pull request #32852: URL: https://github.com/apache/spark/pull/32852#issuecomment-858386662 **[Test build #139623 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139623/testReport)** for PR 32852 at commit

[GitHub] [spark] SparkQA commented on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

2021-06-10 Thread GitBox
SparkQA commented on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-858386822 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44148/ -- This is an automated message from the Apache

[GitHub] [spark] eejbyfeldt commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values

2021-06-10 Thread GitBox
eejbyfeldt commented on pull request #32783: URL: https://github.com/apache/spark/pull/32783#issuecomment-858394340 > `StructConverter` converts from Catalyst struct to a `Row`. Will it be a behavior change? Although it is Catalyst expression. From my understanding this is exactly

[GitHub] [spark] SparkQA commented on pull request #32828: [SPARK-35689][SS] Add log warn when keyWithIndexToValue returns null value

2021-06-10 Thread GitBox
SparkQA commented on pull request #32828: URL: https://github.com/apache/spark/pull/32828#issuecomment-858394064 **[Test build #139625 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139625/testReport)** for PR 32828 at commit

[GitHub] [spark] SparkQA commented on pull request #32470: [WIP] Simplify ResolveAggregateFunctions

2021-06-10 Thread GitBox
SparkQA commented on pull request #32470: URL: https://github.com/apache/spark/pull/32470#issuecomment-858471292 **[Test build #139629 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139629/testReport)** for PR 32470 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32470: [WIP] Simplify ResolveAggregateFunctions

2021-06-10 Thread GitBox
AmplabJenkins commented on pull request #32470: URL: https://github.com/apache/spark/pull/32470#issuecomment-858471325 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139629/ -- This

[GitHub] [spark] beliefer commented on pull request #32852: [SPARK-35283][SQL] Support query some DDL with CTES

2021-06-10 Thread GitBox
beliefer commented on pull request #32852: URL: https://github.com/apache/spark/pull/32852#issuecomment-858500113 > https://dev.mysql.com/doc/refman/8.0/en/extended-show.html @wangyum There is a discussion below. https://github.com/apache/spark/pull/31548#issuecomment-814170427

[GitHub] [spark] zhouyejoe commented on a change in pull request #32007: [SPARK-33350][SHUFFLE] Add support to DiskBlockManager to create merge directory and to get the local shuffle merged data

2021-06-10 Thread GitBox
zhouyejoe commented on a change in pull request #32007: URL: https://github.com/apache/spark/pull/32007#discussion_r648874848 ## File path: core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala ## @@ -153,6 +185,82 @@ private[spark] class DiskBlockManager(conf:

[GitHub] [spark] imback82 opened a new pull request #32854: [SPARK-34320][SQL] Migrate ALTER TABLE DROP COLUMNS commands to use UnresolvedTable to resolve the identifier

2021-06-10 Thread GitBox
imback82 opened a new pull request #32854: URL: https://github.com/apache/spark/pull/32854 ### What changes were proposed in this pull request? This PR proposes to migrate the following `ALTER TABLE ... DROP COLUMNS` command to use `UnresolvedTable` as a `child` to resolve

[GitHub] [spark] zhouyejoe commented on a change in pull request #32007: [SPARK-33350][SHUFFLE] Add support to DiskBlockManager to create merge directory and to get the local shuffle merged data

2021-06-10 Thread GitBox
zhouyejoe commented on a change in pull request #32007: URL: https://github.com/apache/spark/pull/32007#discussion_r648874572 ## File path: core/src/main/scala/org/apache/spark/util/Utils.scala ## @@ -2566,11 +2601,28 @@ private[spark] object Utils extends Logging { }

[GitHub] [spark] SparkQA commented on pull request #32841: [SPARK-35673][SQL] Fix user-defined hint and unrecognized hint in subquery.

2021-06-10 Thread GitBox
SparkQA commented on pull request #32841: URL: https://github.com/apache/spark/pull/32841#issuecomment-858343681 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44142/ --

[GitHub] [spark] jerqi closed pull request #32856: [SPARK-35706][SQL]Consider making the ':' in STRUCT data type definition optional

2021-06-10 Thread GitBox
jerqi closed pull request #32856: URL: https://github.com/apache/spark/pull/32856 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [spark] SparkQA commented on pull request #32821: [SPARK-35342][PYTHON] Introduce DecimalOps

2021-06-10 Thread GitBox
SparkQA commented on pull request #32821: URL: https://github.com/apache/spark/pull/32821#issuecomment-858350174 **[Test build #139620 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139620/testReport)** for PR 32821 at commit

[GitHub] [spark] SparkQA commented on pull request #32855: [SPARK-34524][SQL][FOLLOWUP] Remove unused checkAlterTablePartition in CheckAnalysis.scala

2021-06-10 Thread GitBox
SparkQA commented on pull request #32855: URL: https://github.com/apache/spark/pull/32855#issuecomment-858350181 **[Test build #139618 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139618/testReport)** for PR 32855 at commit

[GitHub] [spark] SparkQA commented on pull request #32854: [SPARK-34320][SQL] Migrate ALTER TABLE DROP COLUMNS commands to use UnresolvedTable to resolve the identifier

2021-06-10 Thread GitBox
SparkQA commented on pull request #32854: URL: https://github.com/apache/spark/pull/32854#issuecomment-858350117 **[Test build #139619 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139619/testReport)** for PR 32854 at commit

[GitHub] [spark] SparkQA commented on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

2021-06-10 Thread GitBox
SparkQA commented on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-858350225 **[Test build #139621 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139621/testReport)** for PR 32816 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32815: [SPARK-35675][SQL] EnsureRequirements remove shuffle should respect PartitioningCollection

2021-06-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32815: URL: https://github.com/apache/spark/pull/32815#issuecomment-858296344 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44132/

[GitHub] [spark] SparkQA removed a comment on pull request #32815: [SPARK-35675][SQL] EnsureRequirements remove shuffle should respect PartitioningCollection

2021-06-10 Thread GitBox
SparkQA removed a comment on pull request #32815: URL: https://github.com/apache/spark/pull/32815#issuecomment-858231569 **[Test build #139605 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139605/testReport)** for PR 32815 at commit

[GitHub] [spark] zhengruifeng commented on pull request #32822: [SPARK-35678][ML] add a common softmax function

2021-06-10 Thread GitBox
zhengruifeng commented on pull request #32822: URL: https://github.com/apache/spark/pull/32822#issuecomment-858370600 ping @srowen -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] cloud-fan commented on pull request #32841: [SPARK-35673][SQL] Fix user-defined hint and unrecognized hint in subquery.

2021-06-10 Thread GitBox
cloud-fan commented on pull request #32841: URL: https://github.com/apache/spark/pull/32841#issuecomment-858386217 thanks, merging to master/3.1/3.0! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] cloud-fan closed pull request #32841: [SPARK-35673][SQL] Fix user-defined hint and unrecognized hint in subquery.

2021-06-10 Thread GitBox
cloud-fan closed pull request #32841: URL: https://github.com/apache/spark/pull/32841 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] SparkQA commented on pull request #32470: [WIP] Simplify ResolveAggregateFunctions

2021-06-10 Thread GitBox
SparkQA commented on pull request #32470: URL: https://github.com/apache/spark/pull/32470#issuecomment-858396906 **[Test build #139612 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139612/testReport)** for PR 32470 at commit

[GitHub] [spark] SparkQA commented on pull request #32842: [MINOR][SQL] No need to normolize name for built-in functions

2021-06-10 Thread GitBox
SparkQA commented on pull request #32842: URL: https://github.com/apache/spark/pull/32842#issuecomment-858396774 **[Test build #139607 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139607/testReport)** for PR 32842 at commit

[GitHub] [spark] SparkQA commented on pull request #32857: [SPARK-35707][ML] optimize sparse GEMM by skipping bound checking

2021-06-10 Thread GitBox
SparkQA commented on pull request #32857: URL: https://github.com/apache/spark/pull/32857#issuecomment-858415585 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44149/ --

[GitHub] [spark] SparkQA commented on pull request #32821: [SPARK-35342][PYTHON] Introduce DecimalOps

2021-06-10 Thread GitBox
SparkQA commented on pull request #32821: URL: https://github.com/apache/spark/pull/32821#issuecomment-858427341 **[Test build #139628 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139628/testReport)** for PR 32821 at commit

[GitHub] [spark] yos1p edited a comment on pull request #32795: [SPARK-35588][PYTHON][DOCS] Update quickstart.ipynb to use pyspark.pandas

2021-06-10 Thread GitBox
yos1p edited a comment on pull request #32795: URL: https://github.com/apache/spark/pull/32795#issuecomment-856936411 > You might need to push some changes at https://github.com/apache/spark/blob/master/binder/postBuild to build Spark, install extra dependencies, etc. It seems

[GitHub] [spark] SparkQA commented on pull request #32859: [SPARK-35708][PYTHON][TEST] Add BaseTest for DataTypeOps

2021-06-10 Thread GitBox
SparkQA commented on pull request #32859: URL: https://github.com/apache/spark/pull/32859#issuecomment-858427215 **[Test build #139626 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139626/testReport)** for PR 32859 at commit

[GitHub] [spark] SparkQA commented on pull request #32828: [SPARK-35689][SS] Add log warn when keyWithIndexToValue returns null value

2021-06-10 Thread GitBox
SparkQA commented on pull request #32828: URL: https://github.com/apache/spark/pull/32828#issuecomment-858427294 **[Test build #139627 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139627/testReport)** for PR 32828 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32857: [SPARK-35707][ML] optimize sparse GEMM by skipping bound checking

2021-06-10 Thread GitBox
AmplabJenkins commented on pull request #32857: URL: https://github.com/apache/spark/pull/32857#issuecomment-858432871 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139622/ -- This

[GitHub] [spark] cloud-fan commented on a change in pull request #32849: [WIP][SPARK-35704][SQL] Support fields by the day-time interval type

2021-06-10 Thread GitBox
cloud-fan commented on a change in pull request #32849: URL: https://github.com/apache/spark/pull/32849#discussion_r648978557 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/IntervalUtils.scala ## @@ -158,7 +158,10 @@ object IntervalUtils {

[GitHub] [spark] AmplabJenkins commented on pull request #32821: [SPARK-35342][PYTHON] Introduce DecimalOps

2021-06-10 Thread GitBox
AmplabJenkins commented on pull request #32821: URL: https://github.com/apache/spark/pull/32821#issuecomment-858466240 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32859: [SPARK-35708][PYTHON][TEST] Add BaseTest for DataTypeOps

2021-06-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32859: URL: https://github.com/apache/spark/pull/32859#issuecomment-858466246 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139626/

[GitHub] [spark] SparkQA commented on pull request #32821: [SPARK-35342][PYTHON] Introduce DecimalOps

2021-06-10 Thread GitBox
SparkQA commented on pull request #32821: URL: https://github.com/apache/spark/pull/32821#issuecomment-858464559 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44155/ --

[GitHub] [spark] SparkQA commented on pull request #32854: [SPARK-34320][SQL] Migrate ALTER TABLE DROP COLUMNS commands to use UnresolvedTable to resolve the identifier

2021-06-10 Thread GitBox
SparkQA commented on pull request #32854: URL: https://github.com/apache/spark/pull/32854#issuecomment-858486749 **[Test build #139619 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139619/testReport)** for PR 32854 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #32854: [SPARK-34320][SQL] Migrate ALTER TABLE DROP COLUMNS commands to use UnresolvedTable to resolve the identifier

2021-06-10 Thread GitBox
SparkQA removed a comment on pull request #32854: URL: https://github.com/apache/spark/pull/32854#issuecomment-858350117 **[Test build #139619 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139619/testReport)** for PR 32854 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32815: [SPARK-35675][SQL] EnsureRequirements remove shuffle should respect PartitioningCollection

2021-06-10 Thread GitBox
AmplabJenkins commented on pull request #32815: URL: https://github.com/apache/spark/pull/32815#issuecomment-858354919 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139605/ -- This

[GitHub] [spark] mridulm commented on pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

2021-06-10 Thread GitBox
mridulm commented on pull request #30691: URL: https://github.com/apache/spark/pull/30691#issuecomment-858362869 jenkins, test this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] SparkQA commented on pull request #32821: [SPARK-35342][PYTHON] Introduce DecimalOps

2021-06-10 Thread GitBox
SparkQA commented on pull request #32821: URL: https://github.com/apache/spark/pull/32821#issuecomment-858376468 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44143/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

2021-06-10 Thread GitBox
SparkQA commented on pull request #30691: URL: https://github.com/apache/spark/pull/30691#issuecomment-858387633 **[Test build #139624 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139624/testReport)** for PR 30691 at commit

[GitHub] [spark] SparkQA commented on pull request #32821: [SPARK-35342][PYTHON] Introduce DecimalOps

2021-06-10 Thread GitBox
SparkQA commented on pull request #32821: URL: https://github.com/apache/spark/pull/32821#issuecomment-858402837 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44147/ -- This is an automated message from the

[GitHub] [spark] jerqi opened a new pull request #32858: [SPARK-35706][SQL]Consider making the ':' in STRUCT data type definit…

2021-06-10 Thread GitBox
jerqi opened a new pull request #32858: URL: https://github.com/apache/spark/pull/32858 ### What changes were proposed in this pull request? The STRUCT type syntax is defined like this: STRUCT(fieldNmae: fileType [NOT NULL][COMMENT stringLiteral][,.]) So the field

[GitHub] [spark] Yikun opened a new pull request #32859: [SPARK-35708][PYTHON][TEST] Add BaseTest for DataTypeOps

2021-06-10 Thread GitBox
Yikun opened a new pull request #32859: URL: https://github.com/apache/spark/pull/32859 ### What changes were proposed in this pull request? This patch adds DataTypeOps test to check the ops is loaded as expected. ### Why are the changes needed? When complete

[GitHub] [spark] SparkQA commented on pull request #32828: [SPARK-35689][SS] Add log warn when keyWithIndexToValue returns null value

2021-06-10 Thread GitBox
SparkQA commented on pull request #32828: URL: https://github.com/apache/spark/pull/32828#issuecomment-858438256 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44152/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on a change in pull request #32849: [WIP][SPARK-35704][SQL] Support fields by the day-time interval type

2021-06-10 Thread GitBox
cloud-fan commented on a change in pull request #32849: URL: https://github.com/apache/spark/pull/32849#discussion_r648984100 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/DayTimeIntervalType.scala ## @@ -57,16 +62,61 @@ class DayTimeIntervalType

[GitHub] [spark] SparkQA commented on pull request #32852: [SPARK-35283][SQL] Support query some DDL with CTES

2021-06-10 Thread GitBox
SparkQA commented on pull request #32852: URL: https://github.com/apache/spark/pull/32852#issuecomment-858451052 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44150/ -- This is an automated message from the

[GitHub] [spark] SparkQA removed a comment on pull request #32470: [WIP] Simplify ResolveAggregateFunctions

2021-06-10 Thread GitBox
SparkQA removed a comment on pull request #32470: URL: https://github.com/apache/spark/pull/32470#issuecomment-858467500 **[Test build #139629 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139629/testReport)** for PR 32470 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32470: [WIP] Simplify ResolveAggregateFunctions

2021-06-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32470: URL: https://github.com/apache/spark/pull/32470#issuecomment-858471325 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139629/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32776: [SPARK-35639][SQL] Add metrics about coalesced partitions to CustomShuffleReader in AQE

2021-06-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32776: URL: https://github.com/apache/spark/pull/32776#issuecomment-858347806 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44140/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32841: [SPARK-35673][SQL] Fix user-defined hint and unrecognized hint in subquery.

2021-06-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32841: URL: https://github.com/apache/spark/pull/32841#issuecomment-858347803 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44142/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32852: [SPARK-35283][SQL] Support query some DDL with CTES

2021-06-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32852: URL: https://github.com/apache/spark/pull/32852#issuecomment-858347804 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32853: [SPARK-35683][PYTHON] Fix Index.difference to avoid collect 'other' to driver side

2021-06-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32853: URL: https://github.com/apache/spark/pull/32853#issuecomment-858347805 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139614/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32470: [WIP] Simplify ResolveAggregateFunctions

2021-06-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32470: URL: https://github.com/apache/spark/pull/32470#issuecomment-858347812 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44137/

[GitHub] [spark] AmplabJenkins commented on pull request #32470: [WIP] Simplify ResolveAggregateFunctions

2021-06-10 Thread GitBox
AmplabJenkins commented on pull request #32470: URL: https://github.com/apache/spark/pull/32470#issuecomment-858347812 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44137/ --

[GitHub] [spark] SparkQA removed a comment on pull request #32470: [WIP] Simplify ResolveAggregateFunctions

2021-06-10 Thread GitBox
SparkQA removed a comment on pull request #32470: URL: https://github.com/apache/spark/pull/32470#issuecomment-858290812 **[Test build #139612 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139612/testReport)** for PR 32470 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #32842: [MINOR][SQL] No need to normolize name for built-in functions

2021-06-10 Thread GitBox
SparkQA removed a comment on pull request #32842: URL: https://github.com/apache/spark/pull/32842#issuecomment-858252089 **[Test build #139607 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139607/testReport)** for PR 32842 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #32828: [SPARK-35689][SS] Add log warn when keyWithIndexToValue returns null value

2021-06-10 Thread GitBox
SparkQA removed a comment on pull request #32828: URL: https://github.com/apache/spark/pull/32828#issuecomment-858394064 **[Test build #139625 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139625/testReport)** for PR 32828 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #32849: [WIP][SPARK-35704][SQL] Support fields by the day-time interval type

2021-06-10 Thread GitBox
cloud-fan commented on a change in pull request #32849: URL: https://github.com/apache/spark/pull/32849#discussion_r648970057 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Average.scala ## @@ -95,7 +95,7 @@ case class

[GitHub] [spark] SparkQA removed a comment on pull request #32821: [SPARK-35342][PYTHON] Introduce DecimalOps

2021-06-10 Thread GitBox
SparkQA removed a comment on pull request #32821: URL: https://github.com/apache/spark/pull/32821#issuecomment-858427341 **[Test build #139628 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139628/testReport)** for PR 32821 at commit

[GitHub] [spark] SparkQA commented on pull request #32776: [SPARK-35639][SQL] Add metrics about coalesced partitions to CustomShuffleReader in AQE

2021-06-10 Thread GitBox
SparkQA commented on pull request #32776: URL: https://github.com/apache/spark/pull/32776#issuecomment-858463477 **[Test build #139613 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139613/testReport)** for PR 32776 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #32776: [SPARK-35639][SQL] Add metrics about coalesced partitions to CustomShuffleReader in AQE

2021-06-10 Thread GitBox
SparkQA removed a comment on pull request #32776: URL: https://github.com/apache/spark/pull/32776#issuecomment-858294837 **[Test build #139613 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139613/testReport)** for PR 32776 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #32470: [WIP] Simplify ResolveAggregateFunctions

2021-06-10 Thread GitBox
cloud-fan commented on a change in pull request #32470: URL: https://github.com/apache/spark/pull/32470#discussion_r649008968 ## File path: sql/core/src/test/resources/tpcds-plan-stability/approved-plans-v1_4/q35.sf100/explain.txt ## @@ -306,11 +306,11 @@ Input [19]:

[GitHub] [spark] tanelk commented on pull request #32862: [SPARK-35695][SQL] Collect observed metrics from cached sub-tree

2021-06-10 Thread GitBox
tanelk commented on pull request #32862: URL: https://github.com/apache/spark/pull/32862#issuecomment-858494892 Pinging @cloud-fan , @sarutak -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] SparkQA commented on pull request #32853: [SPARK-35683][PYTHON] Fix Index.difference to avoid collect 'other' to driver side

2021-06-10 Thread GitBox
SparkQA commented on pull request #32853: URL: https://github.com/apache/spark/pull/32853#issuecomment-858335817 **[Test build #139614 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139614/testReport)** for PR 32853 at commit

[GitHub] [spark] SparkQA commented on pull request #32776: [SPARK-35639][SQL] Add metrics about coalesced partitions to CustomShuffleReader in AQE

2021-06-10 Thread GitBox
SparkQA commented on pull request #32776: URL: https://github.com/apache/spark/pull/32776#issuecomment-858335819 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44140/ -- This is an automated message from the

[GitHub] [spark] SparkQA removed a comment on pull request #32853: [SPARK-35683][PYTHON] Fix Index.difference to avoid collect 'other' to driver side

2021-06-10 Thread GitBox
SparkQA removed a comment on pull request #32853: URL: https://github.com/apache/spark/pull/32853#issuecomment-858316127 **[Test build #139614 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139614/testReport)** for PR 32853 at commit

[GitHub] [spark] SparkQA commented on pull request #32821: [SPARK-35342][PYTHON] Introduce DecimalOps

2021-06-10 Thread GitBox
SparkQA commented on pull request #32821: URL: https://github.com/apache/spark/pull/32821#issuecomment-858351005 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44143/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32821: [SPARK-35342][PYTHON] Introduce DecimalOps

2021-06-10 Thread GitBox
SparkQA commented on pull request #32821: URL: https://github.com/apache/spark/pull/32821#issuecomment-858350628 **[Test build #139620 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139620/testReport)** for PR 32821 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32821: [SPARK-35342][PYTHON] Introduce DecimalOps

2021-06-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32821: URL: https://github.com/apache/spark/pull/32821#issuecomment-858350651 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139620/

[GitHub] [spark] AmplabJenkins commented on pull request #32821: [SPARK-35342][PYTHON] Introduce DecimalOps

2021-06-10 Thread GitBox
AmplabJenkins commented on pull request #32821: URL: https://github.com/apache/spark/pull/32821#issuecomment-858350651 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139620/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #32821: [SPARK-35342][PYTHON] Introduce DecimalOps

2021-06-10 Thread GitBox
SparkQA removed a comment on pull request #32821: URL: https://github.com/apache/spark/pull/32821#issuecomment-858350174 **[Test build #139620 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139620/testReport)** for PR 32821 at commit

[GitHub] [spark] dgd-contributor commented on pull request #32839: [SPARK-35679][SQL] instantToMicros overflow

2021-06-10 Thread GitBox
dgd-contributor commented on pull request #32839: URL: https://github.com/apache/spark/pull/32839#issuecomment-858366444 Thanks @MaxGekk @cloud-fan @gengliangwang for supporting -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] zhengruifeng commented on pull request #32857: [SPARK-35707][ML] optimize sparse GEMM by skipping bound checking

2021-06-10 Thread GitBox
zhengruifeng commented on pull request #32857: URL: https://github.com/apache/spark/pull/32857#issuecomment-858366739 ![image](https://user-images.githubusercontent.com/7322292/121479250-6a762000-c9fc-11eb-9ca2-1331ec8fffb8.png) -- This is an automated message from the Apache Git

[GitHub] [spark] SparkQA commented on pull request #32855: [SPARK-34524][SQL][FOLLOWUP] Remove unused checkAlterTablePartition in CheckAnalysis.scala

2021-06-10 Thread GitBox
SparkQA commented on pull request #32855: URL: https://github.com/apache/spark/pull/32855#issuecomment-858388065 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44145/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

2021-06-10 Thread GitBox
SparkQA commented on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-858409096 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44148/ -- This is an automated message from the

[GitHub] [spark] wangyum commented on pull request #32852: [SPARK-35283][SQL] Support query some DDL with CTES

2021-06-10 Thread GitBox
wangyum commented on pull request #32852: URL: https://github.com/apache/spark/pull/32852#issuecomment-858417308 This is MySQL syntax: ``` mysql> SHOW CHARACTER SET WHERE `Default collation` LIKE '%japanese%'; +-+---+-++

[GitHub] [spark] SparkQA commented on pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

2021-06-10 Thread GitBox
SparkQA commented on pull request #30691: URL: https://github.com/apache/spark/pull/30691#issuecomment-858426166 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44151/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #32858: [SPARK-35706][SQL]Consider making the ':' in STRUCT data type definit…

2021-06-10 Thread GitBox
AmplabJenkins commented on pull request #32858: URL: https://github.com/apache/spark/pull/32858#issuecomment-858425281 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] SparkQA commented on pull request #32852: [SPARK-35283][SQL] Support query some DDL with CTES

2021-06-10 Thread GitBox
SparkQA commented on pull request #32852: URL: https://github.com/apache/spark/pull/32852#issuecomment-858425912 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44150/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32821: [SPARK-35342][PYTHON] Introduce DecimalOps

2021-06-10 Thread GitBox
SparkQA commented on pull request #32821: URL: https://github.com/apache/spark/pull/32821#issuecomment-858454242 **[Test build #139628 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139628/testReport)** for PR 32821 at commit

[GitHub] [spark] SparkQA commented on pull request #32859: [SPARK-35708][PYTHON][TEST] Add BaseTest for DataTypeOps

2021-06-10 Thread GitBox
SparkQA commented on pull request #32859: URL: https://github.com/apache/spark/pull/32859#issuecomment-858490830 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44153/ -- This is an automated message from the

[GitHub] [spark] jerqi opened a new pull request #32856: [SPARK-35706][SQL]Consider making the ':' in STRUCT data type definition optional

2021-06-10 Thread GitBox
jerqi opened a new pull request #32856: URL: https://github.com/apache/spark/pull/32856 ### What changes were proposed in this pull request? The STRUCT type syntax is defined like this: STRUCT(fieldNmae: fileType [NOT NULL][COMMENT stringLiteral][,.]) So the field list

[GitHub] [spark] AmplabJenkins commented on pull request #32856: [SPARK-35706][SQL]Consider making the ':' in STRUCT data type definition optional

2021-06-10 Thread GitBox
AmplabJenkins commented on pull request #32856: URL: https://github.com/apache/spark/pull/32856#issuecomment-858349898 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] SparkQA commented on pull request #32854: [SPARK-34320][SQL] Migrate ALTER TABLE DROP COLUMNS commands to use UnresolvedTable to resolve the identifier

2021-06-10 Thread GitBox
SparkQA commented on pull request #32854: URL: https://github.com/apache/spark/pull/32854#issuecomment-858378455 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44146/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #32853: [SPARK-35683][PYTHON] Fix Index.difference to avoid collect 'other' to driver side

2021-06-10 Thread GitBox
AmplabJenkins commented on pull request #32853: URL: https://github.com/apache/spark/pull/32853#issuecomment-858384277 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44141/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32821: [SPARK-35342][PYTHON] Introduce DecimalOps

2021-06-10 Thread GitBox
AmplabJenkins commented on pull request #32821: URL: https://github.com/apache/spark/pull/32821#issuecomment-858384275 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44143/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32821: [SPARK-35342][PYTHON] Introduce DecimalOps

2021-06-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32821: URL: https://github.com/apache/spark/pull/32821#issuecomment-858384275 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44143/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32853: [SPARK-35683][PYTHON] Fix Index.difference to avoid collect 'other' to driver side

2021-06-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32853: URL: https://github.com/apache/spark/pull/32853#issuecomment-858384277 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44141/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

2021-06-10 Thread GitBox
AmplabJenkins removed a comment on pull request #30691: URL: https://github.com/apache/spark/pull/30691#issuecomment-858391673 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139624/

[GitHub] [spark] SparkQA removed a comment on pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

2021-06-10 Thread GitBox
SparkQA removed a comment on pull request #30691: URL: https://github.com/apache/spark/pull/30691#issuecomment-858387633 **[Test build #139624 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139624/testReport)** for PR 30691 at commit

[GitHub] [spark] viirya commented on pull request #32828: [SPARK-35689][SS] Add log warn when keyWithIndexToValue returns null value

2021-06-10 Thread GitBox
viirya commented on pull request #32828: URL: https://github.com/apache/spark/pull/32828#issuecomment-858400846 Added test. Please take look. Thanks. @HeartSaVioR @dongjoon-hyun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #32855: [SPARK-34524][SQL][FOLLOWUP] Remove unused checkAlterTablePartition in CheckAnalysis.scala

2021-06-10 Thread GitBox
SparkQA commented on pull request #32855: URL: https://github.com/apache/spark/pull/32855#issuecomment-858410582 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44145/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #32855: [SPARK-34524][SQL][FOLLOWUP] Remove unused checkAlterTablePartition in CheckAnalysis.scala

2021-06-10 Thread GitBox
AmplabJenkins commented on pull request #32855: URL: https://github.com/apache/spark/pull/32855#issuecomment-858424084 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44145/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32821: [SPARK-35342][PYTHON] Introduce DecimalOps

2021-06-10 Thread GitBox
AmplabJenkins commented on pull request #32821: URL: https://github.com/apache/spark/pull/32821#issuecomment-858424088 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44147/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32821: [SPARK-35342][PYTHON] Introduce DecimalOps

2021-06-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32821: URL: https://github.com/apache/spark/pull/32821#issuecomment-858424088 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44147/

  1   2   3   4   5   6   7   >