[GitHub] [spark] viirya commented on a change in pull request #29587: [SPARK-32376][SQL] Make unionByName null-filling behavior work with struct columns

2020-08-31 Thread GitBox
viirya commented on a change in pull request #29587: URL: https://github.com/apache/spark/pull/29587#discussion_r480793907 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala ## @@ -546,7 +547,8 @@ case class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29502: [WIP][SPARK-32677][SQL] Load function resource before create

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #29502: URL: https://github.com/apache/spark/pull/29502#issuecomment-684319575 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29502: [WIP][SPARK-32677][SQL] Load function resource before create

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29502: URL: https://github.com/apache/spark/pull/29502#issuecomment-684319575 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29607: [SPARK-32761][SQL] Allow aggregating multiple foldable distinct expressions

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29607: URL: https://github.com/apache/spark/pull/29607#issuecomment-684323910 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
SparkQA commented on pull request #28490: URL: https://github.com/apache/spark/pull/28490#issuecomment-684325215 **[Test build #128132 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128132/testReport)** for PR 28490 at commit

[GitHub] [spark] cloud-fan closed pull request #29595: [SPARK-32659][SQL][FOLLOWUP] Improve test for pruning DPP on non-atomic type

2020-08-31 Thread GitBox
cloud-fan closed pull request #29595: URL: https://github.com/apache/spark/pull/29595 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on pull request #29595: [SPARK-32659][SQL][FOLLOWUP] Improve test for pruning DPP on non-atomic type

2020-08-31 Thread GitBox
cloud-fan commented on pull request #29595: URL: https://github.com/apache/spark/pull/29595#issuecomment-684378625 thanks, merging to master/3.0! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #28490: URL: https://github.com/apache/spark/pull/28490#issuecomment-684371491 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-684183985 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29605: [SPARK-31511][SQL][2.4] Make BytesToBytesMap iterators thread-safe

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29605: URL: https://github.com/apache/spark/pull/29605#issuecomment-684193792 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] HyukjinKwon commented on pull request #29596: [SPARK-32190][PYTHON][DOCS] Development - Contribution Guide in PySpark

2020-08-31 Thread GitBox
HyukjinKwon commented on pull request #29596: URL: https://github.com/apache/spark/pull/29596#issuecomment-684193996 > Maybe mention in testing releases to use a throw away virtual environment? Okay, I think it's better to add a link once the env setup page is complete. I will do it

[GitHub] [spark] AmplabJenkins commented on pull request #29605: [SPARK-31511][SQL][2.4] Make BytesToBytesMap iterators thread-safe

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29605: URL: https://github.com/apache/spark/pull/29605#issuecomment-684192191 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29606: [SPARK-32191][FOLLOW-UP][PYTHON][DOCS] Indent the table and reword the main page in migration guide

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29606: URL: https://github.com/apache/spark/pull/29606#issuecomment-684253813 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29606: [SPARK-32191][FOLLOW-UP][PYTHON][DOCS] Indent the table and reword the main page in migration guide

2020-08-31 Thread GitBox
SparkQA commented on pull request #29606: URL: https://github.com/apache/spark/pull/29606#issuecomment-684251336 **[Test build #128131 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128131/testReport)** for PR 29606 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29606: [SPARK-32191][FOLLOW-UP][PYTHON][DOCS] Indent the table and reword the main page in migration guide

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #29606: URL: https://github.com/apache/spark/pull/29606#issuecomment-684241267 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] dbtsai commented on a change in pull request #29565: [SPARK-24994][SQL] Add UnwrapCastInBinaryComparison optimizer to simplify literal types

2020-08-31 Thread GitBox
dbtsai commented on a change in pull request #29565: URL: https://github.com/apache/spark/pull/29565#discussion_r480819992 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparison.scala ## @@ -0,0 +1,216 @@ +/* + *

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29607: [SPARK-32761][SQL] Allow aggregating multiple foldable distinct expressions

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #29607: URL: https://github.com/apache/spark/pull/29607#issuecomment-684327565 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #28490: URL: https://github.com/apache/spark/pull/28490#issuecomment-684329349 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HyukjinKwon commented on pull request #29607: [SPARK-32761][SQL] Allow aggregating multiple foldable distinct expressions

2020-08-31 Thread GitBox
HyukjinKwon commented on pull request #29607: URL: https://github.com/apache/spark/pull/29607#issuecomment-68433 add to whitelist This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins commented on pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #28490: URL: https://github.com/apache/spark/pull/28490#issuecomment-684390648 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #28490: URL: https://github.com/apache/spark/pull/28490#issuecomment-684390648 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] sunchao commented on a change in pull request #29565: [SPARK-24994][SQL] Add UnwrapCastInBinaryComparison optimizer to simplify literal types

2020-08-31 Thread GitBox
sunchao commented on a change in pull request #29565: URL: https://github.com/apache/spark/pull/29565#discussion_r480850745 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparison.scala ## @@ -0,0 +1,216 @@ +/* + *

[GitHub] [spark] SparkQA commented on pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
SparkQA commented on pull request #28490: URL: https://github.com/apache/spark/pull/28490#issuecomment-684390480 **[Test build #128135 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128135/testReport)** for PR 28490 at commit

[GitHub] [spark] SparkQA commented on pull request #29574: [MINOR][R] Fix a R style in try and finally at DataFrame.R

2020-08-31 Thread GitBox
SparkQA commented on pull request #29574: URL: https://github.com/apache/spark/pull/29574#issuecomment-684013200 **[Test build #128115 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128115/testReport)** for PR 29574 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29574: [MINOR][R] Fix a R style in try and finally at DataFrame.R

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29574: URL: https://github.com/apache/spark/pull/29574#issuecomment-684013343 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #26953: [SPARK-30306][CORE][PYTHON] Instrument Python UDF execution time and throughput metrics using Spark Metrics system

2020-08-31 Thread GitBox
SparkQA commented on pull request #26953: URL: https://github.com/apache/spark/pull/26953#issuecomment-684018253 **[Test build #128117 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128117/testReport)** for PR 26953 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #28490: URL: https://github.com/apache/spark/pull/28490#issuecomment-684023594 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-684023154 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
SparkQA removed a comment on pull request #28490: URL: https://github.com/apache/spark/pull/28490#issuecomment-683873997 **[Test build #128104 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128104/testReport)** for PR 28490 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` when there is only one branch

2020-08-31 Thread GitBox
SparkQA removed a comment on pull request #21850: URL: https://github.com/apache/spark/pull/21850#issuecomment-683919007 **[Test build #128112 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128112/testReport)** for PR 21850 at commit

[GitHub] [spark] SparkQA commented on pull request #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` when there is only one branch

2020-08-31 Thread GitBox
SparkQA commented on pull request #21850: URL: https://github.com/apache/spark/pull/21850#issuecomment-684036492 **[Test build #128112 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128112/testReport)** for PR 21850 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29595: [SPARK-32659][SQL][FOLLOW-UP] Improve test for pruning DPP on non-atomic type

2020-08-31 Thread GitBox
SparkQA removed a comment on pull request #29595: URL: https://github.com/apache/spark/pull/29595#issuecomment-683907489 **[Test build #128108 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128108/testReport)** for PR 29595 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29601: [SPARK-32757][SQL] Physical InSubqueryExec should be consistent with logical InSubquery

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29601: URL: https://github.com/apache/spark/pull/29601#issuecomment-684082340 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29601: [SPARK-32757][SQL] Physical InSubqueryExec should be consistent with logical InSubquery

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #29601: URL: https://github.com/apache/spark/pull/29601#issuecomment-684082340 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-684101151 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-684101151 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dbtsai commented on a change in pull request #29565: [SPARK-24994][SQL] Add UnwrapCastInBinaryComparison optimizer to simplify literal types

2020-08-31 Thread GitBox
dbtsai commented on a change in pull request #29565: URL: https://github.com/apache/spark/pull/29565#discussion_r480476764 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparison.scala ## @@ -0,0 +1,214 @@ +/* + *

[GitHub] [spark] dbtsai commented on a change in pull request #29565: [SPARK-24994][SQL] Add UnwrapCastInBinaryComparison optimizer to simplify literal types

2020-08-31 Thread GitBox
dbtsai commented on a change in pull request #29565: URL: https://github.com/apache/spark/pull/29565#discussion_r480476764 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparison.scala ## @@ -0,0 +1,214 @@ +/* + *

[GitHub] [spark] sunchao commented on a change in pull request #29565: [SPARK-24994][SQL] Add UnwrapCastInBinaryComparison optimizer to simplify literal types

2020-08-31 Thread GitBox
sunchao commented on a change in pull request #29565: URL: https://github.com/apache/spark/pull/29565#discussion_r480477785 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparison.scala ## @@ -0,0 +1,214 @@ +/* + *

[GitHub] [spark] SparkQA commented on pull request #29502: [WIP][SPARK-32677][SQL] Load function resource before create

2020-08-31 Thread GitBox
SparkQA commented on pull request #29502: URL: https://github.com/apache/spark/pull/29502#issuecomment-684116675 **[Test build #128122 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128122/testReport)** for PR 29502 at commit

[GitHub] [spark] dbtsai commented on a change in pull request #29565: [SPARK-24994][SQL] Add UnwrapCastInBinaryComparison optimizer to simplify literal types

2020-08-31 Thread GitBox
dbtsai commented on a change in pull request #29565: URL: https://github.com/apache/spark/pull/29565#discussion_r480491875 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparison.scala ## @@ -0,0 +1,214 @@ +/* + *

[GitHub] [spark] AmplabJenkins commented on pull request #29502: [WIP][SPARK-32677][SQL] Load function resource before create

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29502: URL: https://github.com/apache/spark/pull/29502#issuecomment-684117127 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29603: [SPARK-32721][SQL][FOLLOWUP] Simplify if clauses with null and boolean

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #29603: URL: https://github.com/apache/spark/pull/29603#issuecomment-684127875 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] sunchao commented on a change in pull request #29565: [SPARK-24994][SQL] Add UnwrapCastInBinaryComparison optimizer to simplify literal types

2020-08-31 Thread GitBox
sunchao commented on a change in pull request #29565: URL: https://github.com/apache/spark/pull/29565#discussion_r480525850 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparison.scala ## @@ -0,0 +1,214 @@ +/* + *

[GitHub] [spark] AmplabJenkins commented on pull request #29567: [SPARK-32721][SQL] Simplify if clauses with null and boolean

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29567: URL: https://github.com/apache/spark/pull/29567#issuecomment-684133830 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] xuanyuanking commented on pull request #28781: [SPARK-31953][SS] Add Spark Structured Streaming History Server Support

2020-08-31 Thread GitBox
xuanyuanking commented on pull request #28781: URL: https://github.com/apache/spark/pull/28781#issuecomment-684146375 @uncleGen Sorry for the late reply, could you please resolve the conflicts? Let's continue this work.

[GitHub] [spark] HyukjinKwon commented on pull request #29596: [SPARK-32190][PYTHON][DOCS] Development - Contribution Guide in PySpark

2020-08-31 Thread GitBox
HyukjinKwon commented on pull request #29596: URL: https://github.com/apache/spark/pull/29596#issuecomment-684146590 It is planned to document development environment setup separately with Conda separately so it should be fine. I could try to add some more words about that while I am here

[GitHub] [spark] LantaoJin commented on pull request #29558: [SPARK-32715][CORE] Fix memory leak when failed to store pieces of broadcast

2020-08-31 Thread GitBox
LantaoJin commented on pull request #29558: URL: https://github.com/apache/spark/pull/29558#issuecomment-684161707 Ping @cloud-fan @dongjoon-hyun This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] KevinSmile closed pull request #29599: [MINOR] fix confusing val names

2020-08-31 Thread GitBox
KevinSmile closed pull request #29599: URL: https://github.com/apache/spark/pull/29599 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] AmplabJenkins commented on pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #28490: URL: https://github.com/apache/spark/pull/28490#issuecomment-684181119 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29396: [SPARK-32579][SQL] Implement JDBCScan/ScanBuilder/WriteBuilder

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #29396: URL: https://github.com/apache/spark/pull/29396#issuecomment-684183985 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cxzl25 opened a new pull request #29605: [SPARK-31511][SQL][2.4] Make BytesToBytesMap iterators thread-safe

2020-08-31 Thread GitBox
cxzl25 opened a new pull request #29605: URL: https://github.com/apache/spark/pull/29605 ### What changes were proposed in this pull request? This PR increases the thread safety of the `BytesToBytesMap`: - It makes the `iterator()` and `destructiveIterator()` methods used their own

[GitHub] [spark] cloud-fan commented on a change in pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
cloud-fan commented on a change in pull request #28490: URL: https://github.com/apache/spark/pull/28490#discussion_r480801106 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1326,7 +1326,11 @@ class Analyzer( * Note

[GitHub] [spark] cloud-fan commented on a change in pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
cloud-fan commented on a change in pull request #28490: URL: https://github.com/apache/spark/pull/28490#discussion_r480799325 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1326,7 +1326,11 @@ class Analyzer( * Note

[GitHub] [spark] cloud-fan commented on a change in pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
cloud-fan commented on a change in pull request #28490: URL: https://github.com/apache/spark/pull/28490#discussion_r480799758 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1428,8 +1428,36 @@ class Analyzer( //

[GitHub] [spark] cloud-fan commented on a change in pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
cloud-fan commented on a change in pull request #28490: URL: https://github.com/apache/spark/pull/28490#discussion_r480801309 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1425,11 +1433,48 @@ class Analyzer( //

[GitHub] [spark] dbtsai commented on a change in pull request #29565: [SPARK-24994][SQL] Add UnwrapCastInBinaryComparison optimizer to simplify literal types

2020-08-31 Thread GitBox
dbtsai commented on a change in pull request #29565: URL: https://github.com/apache/spark/pull/29565#discussion_r480824929 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparison.scala ## @@ -0,0 +1,216 @@ +/* + *

[GitHub] [spark] dbtsai commented on a change in pull request #29565: [SPARK-24994][SQL] Add UnwrapCastInBinaryComparison optimizer to simplify literal types

2020-08-31 Thread GitBox
dbtsai commented on a change in pull request #29565: URL: https://github.com/apache/spark/pull/29565#discussion_r480823284 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparison.scala ## @@ -0,0 +1,216 @@ +/* + *

[GitHub] [spark] SparkQA removed a comment on pull request #29502: [WIP][SPARK-32677][SQL] Load function resource before create

2020-08-31 Thread GitBox
SparkQA removed a comment on pull request #29502: URL: https://github.com/apache/spark/pull/29502#issuecomment-684119050 **[Test build #128123 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128123/testReport)** for PR 29502 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29606: [SPARK-32191][FOLLOW-UP][PYTHON][DOCS] Indent the table and reword the main page in migration guide

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #29606: URL: https://github.com/apache/spark/pull/29606#issuecomment-684313135 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29603: [SPARK-32721][SQL][FOLLOWUP] Simplify if clauses with null and boolean

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #29603: URL: https://github.com/apache/spark/pull/29603#issuecomment-684367170 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29603: [SPARK-32721][SQL][FOLLOWUP] Simplify if clauses with null and boolean

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29603: URL: https://github.com/apache/spark/pull/29603#issuecomment-684367170 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
SparkQA commented on pull request #28490: URL: https://github.com/apache/spark/pull/28490#issuecomment-684367153 **[Test build #128135 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128135/testReport)** for PR 28490 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29587: [SPARK-32376][SQL] Make unionByName null-filling behavior work with struct columns

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #29587: URL: https://github.com/apache/spark/pull/29587#issuecomment-684363229 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #28490: URL: https://github.com/apache/spark/pull/28490#issuecomment-684371491 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dbtsai commented on a change in pull request #29603: [SPARK-32721][SQL][FOLLOWUP] Simplify if clauses with null and boolean

2020-08-31 Thread GitBox
dbtsai commented on a change in pull request #29603: URL: https://github.com/apache/spark/pull/29603#discussion_r480777332 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala ## @@ -465,6 +465,8 @@ object SimplifyConditionals

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29605: [SPARK-31511][SQL][2.4] Make BytesToBytesMap iterators thread-safe

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #29605: URL: https://github.com/apache/spark/pull/29605#issuecomment-684192191 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] dbtsai commented on a change in pull request #29603: [SPARK-32721][SQL][FOLLOWUP] Simplify if clauses with null and boolean

2020-08-31 Thread GitBox
dbtsai commented on a change in pull request #29603: URL: https://github.com/apache/spark/pull/29603#discussion_r480779334 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/SimplifyConditionalSuite.scala ## @@ -166,29 +166,37 @@ class

[GitHub] [spark] AmplabJenkins commented on pull request #29596: [SPARK-32190][PYTHON][DOCS] Development - Contribution Guide in PySpark

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29596: URL: https://github.com/apache/spark/pull/29596#issuecomment-684247694 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29596: [SPARK-32190][PYTHON][DOCS] Development - Contribution Guide in PySpark

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #29596: URL: https://github.com/apache/spark/pull/29596#issuecomment-684247694 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29607: [SPARK-32761][SQL] Allow aggregating multiple foldable distinct expressions

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #29607: URL: https://github.com/apache/spark/pull/29607#issuecomment-684323910 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins commented on pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #28490: URL: https://github.com/apache/spark/pull/28490#issuecomment-684329349 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29607: [SPARK-32761][SQL] Allow aggregating multiple foldable distinct expressions

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29607: URL: https://github.com/apache/spark/pull/29607#issuecomment-684327565 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29596: [SPARK-32190][PYTHON][DOCS] Development - Contribution Guide in PySpark

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #29596: URL: https://github.com/apache/spark/pull/29596#issuecomment-684206525 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29596: [SPARK-32190][PYTHON][DOCS] Development - Contribution Guide in PySpark

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29596: URL: https://github.com/apache/spark/pull/29596#issuecomment-684206525 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dbtsai commented on a change in pull request #29565: [SPARK-24994][SQL] Add UnwrapCastInBinaryComparison optimizer to simplify literal types

2020-08-31 Thread GitBox
dbtsai commented on a change in pull request #29565: URL: https://github.com/apache/spark/pull/29565#discussion_r480807290 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparison.scala ## @@ -0,0 +1,214 @@ +/* + *

[GitHub] [spark] HyukjinKwon opened a new pull request #29606: [SPARK-32191][FOLLOW-UP][PYTHON][DOCS] Indent the table and reword the main page in migration guide

2020-08-31 Thread GitBox
HyukjinKwon opened a new pull request #29606: URL: https://github.com/apache/spark/pull/29606 ### What changes were proposed in this pull request? This PR is a minor followup to fix: 1. Slightly reword the wording in the main page. 2. The indentation in the table at the

[GitHub] [spark] cloud-fan commented on a change in pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
cloud-fan commented on a change in pull request #28490: URL: https://github.com/apache/spark/pull/28490#discussion_r480805872 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1425,11 +1433,48 @@ class Analyzer( //

[GitHub] [spark] HyukjinKwon commented on pull request #29606: [SPARK-32191][FOLLOW-UP][PYTHON][DOCS] Indent the table and reword the main page in migration guide

2020-08-31 Thread GitBox
HyukjinKwon commented on pull request #29606: URL: https://github.com/apache/spark/pull/29606#issuecomment-684240166 cc @viirya, can you take a quick look when you're available? This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #29606: [SPARK-32191][FOLLOW-UP][PYTHON][DOCS] Indent the table and reword the main page in migration guide

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29606: URL: https://github.com/apache/spark/pull/29606#issuecomment-684241267 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dbtsai commented on a change in pull request #29565: [SPARK-24994][SQL] Add UnwrapCastInBinaryComparison optimizer to simplify literal types

2020-08-31 Thread GitBox
dbtsai commented on a change in pull request #29565: URL: https://github.com/apache/spark/pull/29565#discussion_r480829926 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparison.scala ## @@ -0,0 +1,216 @@ +/* + *

[GitHub] [spark] linhongliu-db opened a new pull request #29607: Allow aggregating multiple foldable distinct expressions

2020-08-31 Thread GitBox
linhongliu-db opened a new pull request #29607: URL: https://github.com/apache/spark/pull/29607 ### What changes were proposed in this pull request? For queries with multiple foldable distinct columns, since they will be eliminated during execution, it's not mandatory to let

[GitHub] [spark] AmplabJenkins commented on pull request #29606: [SPARK-32191][FOLLOW-UP][PYTHON][DOCS] Indent the table and reword the main page in migration guide

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29606: URL: https://github.com/apache/spark/pull/29606#issuecomment-684313135 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dbtsai commented on a change in pull request #29565: [SPARK-24994][SQL] Add UnwrapCastInBinaryComparison optimizer to simplify literal types

2020-08-31 Thread GitBox
dbtsai commented on a change in pull request #29565: URL: https://github.com/apache/spark/pull/29565#discussion_r480831353 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparison.scala ## @@ -0,0 +1,216 @@ +/* + *

[GitHub] [spark] SparkQA commented on pull request #29606: [SPARK-32191][FOLLOW-UP][PYTHON][DOCS] Indent the table and reword the main page in migration guide

2020-08-31 Thread GitBox
SparkQA commented on pull request #29606: URL: https://github.com/apache/spark/pull/29606#issuecomment-684310589 **[Test build #128131 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128131/testReport)** for PR 29606 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29606: [SPARK-32191][FOLLOW-UP][PYTHON][DOCS] Indent the table and reword the main page in migration guide

2020-08-31 Thread GitBox
SparkQA removed a comment on pull request #29606: URL: https://github.com/apache/spark/pull/29606#issuecomment-684251336 **[Test build #128131 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128131/testReport)** for PR 29606 at commit

[GitHub] [spark] SparkQA commented on pull request #29502: [WIP][SPARK-32677][SQL] Load function resource before create

2020-08-31 Thread GitBox
SparkQA commented on pull request #29502: URL: https://github.com/apache/spark/pull/29502#issuecomment-684313452 **[Test build #128123 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128123/testReport)** for PR 29502 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #28490: URL: https://github.com/apache/spark/pull/28490#issuecomment-684349635 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #28490: URL: https://github.com/apache/spark/pull/28490#issuecomment-684349635 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29607: [SPARK-32761][SQL] Allow aggregating multiple foldable distinct expressions

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #29607: URL: https://github.com/apache/spark/pull/29607#issuecomment-684349755 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29603: [SPARK-32721][SQL][FOLLOWUP] Simplify if clauses with null and boolean

2020-08-31 Thread GitBox
SparkQA commented on pull request #29603: URL: https://github.com/apache/spark/pull/29603#issuecomment-684386343 **[Test build #128136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128136/testReport)** for PR 29603 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
cloud-fan commented on a change in pull request #28490: URL: https://github.com/apache/spark/pull/28490#discussion_r480848629 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1325,24 +1325,43 @@ class Analyzer( *

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29603: [SPARK-32721][SQL][FOLLOWUP] Simplify if clauses with null and boolean

2020-08-31 Thread GitBox
AmplabJenkins removed a comment on pull request #29603: URL: https://github.com/apache/spark/pull/29603#issuecomment-684388347 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29603: [SPARK-32721][SQL][FOLLOWUP] Simplify if clauses with null and boolean

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29603: URL: https://github.com/apache/spark/pull/29603#issuecomment-684388347 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HyukjinKwon commented on pull request #29596: [SPARK-32190][PYTHON][DOCS] Development - Contribution Guide in PySpark

2020-08-31 Thread GitBox
HyukjinKwon commented on pull request #29596: URL: https://github.com/apache/spark/pull/29596#issuecomment-684290970 Merged to master. Thank you guys. This is an automated message from the Apache Git Service. To

[GitHub] [spark] HyukjinKwon closed pull request #29596: [SPARK-32190][PYTHON][DOCS] Development - Contribution Guide in PySpark

2020-08-31 Thread GitBox
HyukjinKwon closed pull request #29596: URL: https://github.com/apache/spark/pull/29596 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
AngersZh commented on a change in pull request #28490: URL: https://github.com/apache/spark/pull/28490#discussion_r480838692 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1326,7 +1326,11 @@ class Analyzer( *

[GitHub] [spark] AmplabJenkins commented on pull request #29607: [SPARK-32761][SQL] Allow aggregating multiple foldable distinct expressions

2020-08-31 Thread GitBox
AmplabJenkins commented on pull request #29607: URL: https://github.com/apache/spark/pull/29607#issuecomment-684349755 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

2020-08-31 Thread GitBox
AngersZh commented on a change in pull request #28490: URL: https://github.com/apache/spark/pull/28490#discussion_r480838768 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1425,11 +1433,48 @@ class Analyzer(

[GitHub] [spark] SparkQA commented on pull request #29607: [SPARK-32761][SQL] Allow aggregating multiple foldable distinct expressions

2020-08-31 Thread GitBox
SparkQA commented on pull request #29607: URL: https://github.com/apache/spark/pull/29607#issuecomment-684346142 **[Test build #128133 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/128133/testReport)** for PR 29607 at commit

  1   2   3   4   5   6   7   8   >