[jira] [Assigned] (SPARK-32762) Enhance the verification of sql-expression-schema.md

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32762: Assignee: (was: Apache Spark) > Enhance the verification of sql-expression-schema.md

[jira] [Commented] (SPARK-32762) Enhance the verification of sql-expression-schema.md

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188149#comment-17188149 ] Apache Spark commented on SPARK-32762: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-32762) Enhance the verification of sql-expression-schema.md

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32762: Assignee: Apache Spark > Enhance the verification of sql-expression-schema.md >

[jira] [Created] (SPARK-32762) Enhance the verification of sql-expression-schema.md

2020-08-31 Thread Yang Jie (Jira)
Yang Jie created SPARK-32762: Summary: Enhance the verification of sql-expression-schema.md Key: SPARK-32762 URL: https://issues.apache.org/jira/browse/SPARK-32762 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-32187) User Guide - Shipping Python Package

2020-08-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32187: Assignee: Fabian Höring > User Guide - Shipping Python Package >

[jira] [Assigned] (SPARK-32761) Planner error when aggregating multiple distinct Constant columns

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32761: Assignee: Apache Spark > Planner error when aggregating multiple distinct Constant

[jira] [Assigned] (SPARK-32761) Planner error when aggregating multiple distinct Constant columns

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32761: Assignee: (was: Apache Spark) > Planner error when aggregating multiple distinct

[jira] [Commented] (SPARK-32761) Planner error when aggregating multiple distinct Constant columns

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188146#comment-17188146 ] Apache Spark commented on SPARK-32761: -- User 'linhongliu-db' has created a pull request for this

[jira] [Resolved] (SPARK-32190) Development - Contribution Guide

2020-08-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32190. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29596

[jira] [Assigned] (SPARK-32190) Development - Contribution Guide

2020-08-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32190: Assignee: Hyukjin Kwon > Development - Contribution Guide >

[jira] [Updated] (SPARK-32761) Planner error when aggregating multiple distinct Constant columns

2020-08-31 Thread Linhong Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Linhong Liu updated SPARK-32761: Description: SELECT COUNT(DISTINCT 2), COUNT(DISTINCT 2, 3) will trigger this bug. The

[jira] [Created] (SPARK-32761) Planner error when aggregating multiple distinct Constant columns

2020-08-31 Thread Linhong Liu (Jira)
Linhong Liu created SPARK-32761: --- Summary: Planner error when aggregating multiple distinct Constant columns Key: SPARK-32761 URL: https://issues.apache.org/jira/browse/SPARK-32761 Project: Spark

[jira] [Commented] (SPARK-32191) Migration Guide

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188139#comment-17188139 ] Apache Spark commented on SPARK-32191: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-32760) Support for INET data type

2020-08-31 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188138#comment-17188138 ] Xiao Li commented on SPARK-32760: - supporting a new data type is very expensive. I suggested to close it

[jira] [Commented] (SPARK-31511) Make BytesToBytesMap iterator() thread-safe

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188131#comment-17188131 ] Apache Spark commented on SPARK-31511: -- User 'cxzl25' has created a pull request for this issue:

[jira] [Commented] (SPARK-31511) Make BytesToBytesMap iterator() thread-safe

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188132#comment-17188132 ] Apache Spark commented on SPARK-31511: -- User 'cxzl25' has created a pull request for this issue:

[jira] [Resolved] (SPARK-32759) Support for INET data type

2020-08-31 Thread Ruslan Dautkhanov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruslan Dautkhanov resolved SPARK-32759. --- Resolution: Duplicate > Support for INET data type > -- > >

[jira] [Created] (SPARK-32760) Support for INET data type

2020-08-31 Thread Ruslan Dautkhanov (Jira)
Ruslan Dautkhanov created SPARK-32760: - Summary: Support for INET data type Key: SPARK-32760 URL: https://issues.apache.org/jira/browse/SPARK-32760 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-32759) Support for INET data type

2020-08-31 Thread Ruslan Dautkhanov (Jira)
Ruslan Dautkhanov created SPARK-32759: - Summary: Support for INET data type Key: SPARK-32759 URL: https://issues.apache.org/jira/browse/SPARK-32759 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-32758) Spark ignores limit(1) and starts tasks for all partition

2020-08-31 Thread Ivan Tsukanov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Tsukanov updated SPARK-32758: -- Description: If we run the following code {code:scala} val sparkConf = new SparkConf()

[jira] [Updated] (SPARK-32758) Spark ignores limit(1) and starts tasks for all partition

2020-08-31 Thread Ivan Tsukanov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Tsukanov updated SPARK-32758: -- Environment: (was:     должен  ) > Spark ignores limit(1) and starts tasks for all

[jira] [Updated] (SPARK-32758) Spark ignores limit(1) and starts tasks for all partition

2020-08-31 Thread Ivan Tsukanov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Tsukanov updated SPARK-32758: -- Description: If we run the following code {code:scala} val sparkConf = new SparkConf()

[jira] [Updated] (SPARK-32758) Spark ignores limit(1) and starts tasks for all partition

2020-08-31 Thread Ivan Tsukanov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Tsukanov updated SPARK-32758: -- Attachment: image-2020-09-01-10-51-09-417.png > Spark ignores limit(1) and starts tasks for

[jira] [Created] (SPARK-32758) Spark ignores limit(1) and starts tasks for all partition

2020-08-31 Thread Ivan Tsukanov (Jira)
Ivan Tsukanov created SPARK-32758: - Summary: Spark ignores limit(1) and starts tasks for all partition Key: SPARK-32758 URL: https://issues.apache.org/jira/browse/SPARK-32758 Project: Spark

[jira] [Commented] (SPARK-32691) Test org.apache.spark.DistributedSuite failed on arm64 jenkins

2020-08-31 Thread huangtianhua (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188113#comment-17188113 ] huangtianhua commented on SPARK-32691: -- > testOnly *DistributedSuite -- -z "caching in memory and

[jira] [Updated] (SPARK-32691) Test org.apache.spark.DistributedSuite failed on arm64 jenkins

2020-08-31 Thread huangtianhua (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huangtianhua updated SPARK-32691: - Attachment: failure.log > Test org.apache.spark.DistributedSuite failed on arm64 jenkins >

[jira] [Updated] (SPARK-32691) Test org.apache.spark.DistributedSuite failed on arm64 jenkins

2020-08-31 Thread huangtianhua (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huangtianhua updated SPARK-32691: - Attachment: success.log > Test org.apache.spark.DistributedSuite failed on arm64 jenkins >

[jira] [Updated] (SPARK-32715) Broadcast block pieces may memory leak

2020-08-31 Thread Lantao Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-32715: --- Affects Version/s: 2.4.6 3.0.0 > Broadcast block pieces may memory leak >

[jira] [Commented] (SPARK-27951) ANSI SQL: NTH_VALUE function

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188105#comment-17188105 ] Apache Spark commented on SPARK-27951: -- User 'beliefer' has created a pull request for this issue:

[jira] [Commented] (SPARK-27951) ANSI SQL: NTH_VALUE function

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188104#comment-17188104 ] Apache Spark commented on SPARK-27951: -- User 'beliefer' has created a pull request for this issue:

[jira] [Commented] (SPARK-32752) Alias breaks for interval typed literals

2020-08-31 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188103#comment-17188103 ] Kent Yao commented on SPARK-32752: -- two many forms of interval representation that we support and some

[jira] [Commented] (SPARK-32312) Upgrade Apache Arrow to 1.0.0

2020-08-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188079#comment-17188079 ] Hyukjin Kwon commented on SPARK-32312: -- [~bryanc] just out of curiosity, are you still working on

[jira] [Commented] (SPARK-32721) Simplify if clauses with null and boolean

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188075#comment-17188075 ] Apache Spark commented on SPARK-32721: -- User 'sunchao' has created a pull request for this issue:

[jira] [Commented] (SPARK-32721) Simplify if clauses with null and boolean

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188073#comment-17188073 ] Apache Spark commented on SPARK-32721: -- User 'sunchao' has created a pull request for this issue:

[jira] [Commented] (SPARK-32750) Add code-gen for SortAggregateExec

2020-08-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188071#comment-17188071 ] Takeshi Yamamuro commented on SPARK-32750: -- Yea, I don't remember the impl. details in the PR,

[jira] [Resolved] (SPARK-32728) Using groupby with rand creates different values when joining table with itself

2020-08-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-32728. -- Resolution: Invalid > Using groupby with rand creates different values when joining

[jira] [Commented] (SPARK-32728) Using groupby with rand creates different values when joining table with itself

2020-08-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188070#comment-17188070 ] Takeshi Yamamuro commented on SPARK-32728: -- +1 on the Sean comment, so I'll close this. >

[jira] [Commented] (SPARK-32693) Compare two dataframes with same schema except nullable property

2020-08-31 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188057#comment-17188057 ] L. C. Hsieh commented on SPARK-32693: - Ok. Thanks [~maropu] > Compare two dataframes with same

[jira] [Commented] (SPARK-32693) Compare two dataframes with same schema except nullable property

2020-08-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17188055#comment-17188055 ] Takeshi Yamamuro commented on SPARK-32693: -- I changed 3.0.2 to 3.0.1 in "Fix Version/s" because

[jira] [Updated] (SPARK-32693) Compare two dataframes with same schema except nullable property

2020-08-31 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-32693: - Fix Version/s: (was: 3.0.2) 3.0.1 > Compare two dataframes with

[jira] [Resolved] (SPARK-32721) Simplify if clauses with null and boolean

2020-08-31 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-32721. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29567

[jira] [Assigned] (SPARK-32721) Simplify if clauses with null and boolean

2020-08-31 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai reassigned SPARK-32721: --- Assignee: Chao Sun > Simplify if clauses with null and boolean >

[jira] [Commented] (SPARK-32119) ExecutorPlugin doesn't work with Standalone Cluster and Kubernetes with --jars

2020-08-31 Thread Luca Canali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187980#comment-17187980 ] Luca Canali commented on SPARK-32119: - I believe it would be good to have this fix also in the

[jira] [Commented] (SPARK-32624) Replace getClass.getName with getClass.getCanonicalName in CodegenContext.addReferenceObj

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187974#comment-17187974 ] Apache Spark commented on SPARK-32624: -- User 'rednaxelafx' has created a pull request for this

[jira] [Updated] (SPARK-32756) Fix CaseInsensitiveMap in Scala 2.13

2020-08-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-32756: - Priority: Minor (was: Major) > Fix CaseInsensitiveMap in Scala 2.13 >

[jira] [Commented] (SPARK-32756) Fix CaseInsensitiveMap in Scala 2.13

2020-08-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187934#comment-17187934 ] Sean R. Owen commented on SPARK-32756: -- Meh, maybe not. We aren't going to make this change for

[jira] [Commented] (SPARK-32756) Fix CaseInsensitiveMap in Scala 2.13

2020-08-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187933#comment-17187933 ] Sean R. Owen commented on SPARK-32756: -- This should just be a followup for SPARK-32364 > Fix

[jira] [Commented] (SPARK-32728) Using groupby with rand creates different values when joining table with itself

2020-08-31 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187932#comment-17187932 ] Sean R. Owen commented on SPARK-32728: -- I think this is to be expected: you are going to get

[jira] [Assigned] (SPARK-32757) Physical InSubqueryExec should be consistent with logical InSubquery

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32757: Assignee: Wenchen Fan (was: Apache Spark) > Physical InSubqueryExec should be

[jira] [Commented] (SPARK-32757) Physical InSubqueryExec should be consistent with logical InSubquery

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187929#comment-17187929 ] Apache Spark commented on SPARK-32757: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32757) Physical InSubqueryExec should be consistent with logical InSubquery

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32757: Assignee: Apache Spark (was: Wenchen Fan) > Physical InSubqueryExec should be

[jira] [Commented] (SPARK-32757) Physical InSubqueryExec should be consistent with logical InSubquery

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187931#comment-17187931 ] Apache Spark commented on SPARK-32757: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-32752) Alias breaks for interval typed literals

2020-08-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187928#comment-17187928 ] Wenchen Fan commented on SPARK-32752: - [~Qin Yao] do you have any idea to fix it? > Alias breaks

[jira] [Created] (SPARK-32757) Physical InSubqueryExec should be consistent with logical InSubquery

2020-08-31 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-32757: --- Summary: Physical InSubqueryExec should be consistent with logical InSubquery Key: SPARK-32757 URL: https://issues.apache.org/jira/browse/SPARK-32757 Project: Spark

[jira] [Commented] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187902#comment-17187902 ] Apache Spark commented on SPARK-32317: -- User 'izchen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32317: Assignee: Apache Spark > Parquet file loading with different schema(Decimal(N, P)) in

[jira] [Assigned] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32317: Assignee: (was: Apache Spark) > Parquet file loading with different

[jira] [Commented] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187900#comment-17187900 ] Apache Spark commented on SPARK-32317: -- User 'izchen' has created a pull request for this issue:

[jira] [Updated] (SPARK-32756) Fix CaseInsensitiveMap in Scala 2.13

2020-08-31 Thread Karol Chmist (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karol Chmist updated SPARK-32756: - Description:   "Spark SQL" module doesn't compile in Scala 2.13: {code:java} [info] Compiling

[jira] [Commented] (SPARK-18067) SortMergeJoin adds shuffle if join predicates have non partitioned columns

2020-08-31 Thread Andy Van Yperen-De Deyne (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187897#comment-17187897 ] Andy Van Yperen-De Deyne commented on SPARK-18067: -- I am facing this performance issue

[jira] [Updated] (SPARK-32756) Fix CaseInsensitiveMap in Scala 2.13

2020-08-31 Thread Karol Chmist (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karol Chmist updated SPARK-32756: - Description: [info] Compiling 26 Scala sources to

[jira] [Assigned] (SPARK-32756) Fix CaseInsensitiveMap in Scala 2.13

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32756: Assignee: (was: Apache Spark) > Fix CaseInsensitiveMap in Scala 2.13 >

[jira] [Assigned] (SPARK-32756) Fix CaseInsensitiveMap in Scala 2.13

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32756: Assignee: Apache Spark > Fix CaseInsensitiveMap in Scala 2.13 >

[jira] [Commented] (SPARK-32756) Fix CaseInsensitiveMap in Scala 2.13

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187865#comment-17187865 ] Apache Spark commented on SPARK-32756: -- User 'karolchmist' has created a pull request for this

[jira] [Commented] (SPARK-32756) Fix CaseInsensitiveMap in Scala 2.13

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187864#comment-17187864 ] Apache Spark commented on SPARK-32756: -- User 'karolchmist' has created a pull request for this

[jira] [Created] (SPARK-32756) Fix CaseInsensitiveMap in Scala 2.13

2020-08-31 Thread Karol Chmist (Jira)
Karol Chmist created SPARK-32756: Summary: Fix CaseInsensitiveMap in Scala 2.13 Key: SPARK-32756 URL: https://issues.apache.org/jira/browse/SPARK-32756 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-32750) Add code-gen for SortAggregateExec

2020-08-31 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187860#comment-17187860 ] Cheng Su commented on SPARK-32750: -- [~maropu] - thanks a lot for pointing out existing context. Will

[jira] [Assigned] (SPARK-32542) Add an optimizer rule to split an Expand into multiple Expands for aggregates

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32542: Assignee: Apache Spark > Add an optimizer rule to split an Expand into multiple Expands

[jira] [Assigned] (SPARK-32542) Add an optimizer rule to split an Expand into multiple Expands for aggregates

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32542: Assignee: (was: Apache Spark) > Add an optimizer rule to split an Expand into

[jira] [Commented] (SPARK-27733) Upgrade to Avro 1.10.0

2020-08-31 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187849#comment-17187849 ] Chao Sun commented on SPARK-27733: -- bq. BTW, is Hive community will to introduce the version upgrade of

[jira] [Commented] (SPARK-32691) Test org.apache.spark.DistributedSuite failed on arm64 jenkins

2020-08-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187843#comment-17187843 ] Dongjoon Hyun commented on SPARK-32691: --- Yes. That failure looks like that, but it's still

[jira] [Reopened] (SPARK-32542) Add an optimizer rule to split an Expand into multiple Expands for aggregates

2020-08-31 Thread karl wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] karl wang reopened SPARK-32542: --- > Add an optimizer rule to split an Expand into multiple Expands for aggregates >

[jira] [Resolved] (SPARK-32542) Add an optimizer rule to split an Expand into multiple Expands for aggregates

2020-08-31 Thread karl wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] karl wang resolved SPARK-32542. --- Resolution: Fixed > Add an optimizer rule to split an Expand into multiple Expands for aggregates >

[jira] [Assigned] (SPARK-32755) Maintain the order of expressions in AttributeSet and ExpressionSet

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32755: Assignee: (was: Apache Spark) > Maintain the order of expressions in AttributeSet

[jira] [Assigned] (SPARK-32755) Maintain the order of expressions in AttributeSet and ExpressionSet

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32755: Assignee: Apache Spark > Maintain the order of expressions in AttributeSet and

[jira] [Commented] (SPARK-32755) Maintain the order of expressions in AttributeSet and ExpressionSet

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187779#comment-17187779 ] Apache Spark commented on SPARK-32755: -- User 'dbaliafroozeh' has created a pull request for this

[jira] [Created] (SPARK-32755) Maintain the order of expressions in AttributeSet and ExpressionSet

2020-08-31 Thread Ali Afroozeh (Jira)
Ali Afroozeh created SPARK-32755: Summary: Maintain the order of expressions in AttributeSet and ExpressionSet Key: SPARK-32755 URL: https://issues.apache.org/jira/browse/SPARK-32755 Project: Spark

[jira] [Commented] (SPARK-32742) FileOutputCommitter warns "No Output found for attempt"

2020-08-31 Thread Ryan Luo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187751#comment-17187751 ] Ryan Luo commented on SPARK-32742: -- [~hyukjin.kwon] Thanks for advising. > FileOutputCommitter warns

[jira] [Resolved] (SPARK-32592) Make `DataFrameReader.table` take the specified options

2020-08-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32592. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29535

[jira] [Assigned] (SPARK-32592) Make `DataFrameReader.table` take the specified options

2020-08-31 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32592: --- Assignee: Huaxin Gao > Make `DataFrameReader.table` take the specified options >

[jira] [Commented] (SPARK-32752) Alias breaks for interval typed literals

2020-08-31 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187692#comment-17187692 ] Kent Yao commented on SPARK-32752: -- cc [~cloud_fan] > Alias breaks for interval typed literals >

[jira] [Commented] (SPARK-32190) Development - Contribution Guide

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187686#comment-17187686 ] Apache Spark commented on SPARK-32190: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-32190) Development - Contribution Guide

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32190: Assignee: (was: Apache Spark) > Development - Contribution Guide >

[jira] [Assigned] (SPARK-32190) Development - Contribution Guide

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32190: Assignee: Apache Spark > Development - Contribution Guide >

[jira] [Commented] (SPARK-32190) Development - Contribution Guide

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187684#comment-17187684 ] Apache Spark commented on SPARK-32190: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-32659) Fix the data issue of inserted DPP on non-atomic type

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187631#comment-17187631 ] Apache Spark commented on SPARK-32659: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-32746) Not able to run Pandas UDF

2020-08-31 Thread Rahul Bhatia (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187617#comment-17187617 ] Rahul Bhatia commented on SPARK-32746: -- [~hyukjin.kwon] Here are the logs, and more symptoms as

[jira] [Commented] (SPARK-32752) Alias breaks for interval typed literals

2020-08-31 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187613#comment-17187613 ] Kent Yao commented on SPARK-32752: -- this cases will be captured by grammar rule `multiUnitsInterval`,

[jira] [Assigned] (SPARK-32754) Unify `assertEqualPlans` for join reorder suites

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32754: Assignee: (was: Apache Spark) > Unify `assertEqualPlans` for join reorder suites >

[jira] [Assigned] (SPARK-32754) Unify `assertEqualPlans` for join reorder suites

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32754: Assignee: Apache Spark > Unify `assertEqualPlans` for join reorder suites >

[jira] [Commented] (SPARK-32754) Unify `assertEqualPlans` for join reorder suites

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187591#comment-17187591 ] Apache Spark commented on SPARK-32754: -- User 'wzhfy' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32754) Unify `assertEqualPlans` for join reorder suites

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32754: Assignee: Apache Spark > Unify `assertEqualPlans` for join reorder suites >

[jira] [Updated] (SPARK-32754) Unify `assertEqualPlans` for join reorder suites

2020-08-31 Thread Zhenhua Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-32754: - Summary: Unify `assertEqualPlans` for join reorder suites (was: Unify `assertEqualPlans` for

[jira] [Created] (SPARK-32754) Unify `assertEqualPlans` for all join reorder suites

2020-08-31 Thread Zhenhua Wang (Jira)
Zhenhua Wang created SPARK-32754: Summary: Unify `assertEqualPlans` for all join reorder suites Key: SPARK-32754 URL: https://issues.apache.org/jira/browse/SPARK-32754 Project: Spark Issue

[jira] [Updated] (SPARK-32748) Support local property propagation in SubqueryBroadcastExec

2020-08-31 Thread Zhenhua Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-32748: - Description: Since SPARK-22590, local property propagation is supported through

[jira] [Updated] (SPARK-32748) Support local property propagation in SubqueryBroadcastExec

2020-08-31 Thread Zhenhua Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-32748: - Description: Since

[jira] [Updated] (SPARK-32748) Support local property propagation in SubqueryBroadcastExec

2020-08-31 Thread Zhenhua Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-32748: - Description: Currently local property propagation is supported through

[jira] [Assigned] (SPARK-32747) Deduplicate configuration set/unset in test_sparkSQL_arrow.R

2020-08-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32747: Assignee: Hyukjin Kwon > Deduplicate configuration set/unset in test_sparkSQL_arrow.R >

[jira] [Resolved] (SPARK-32747) Deduplicate configuration set/unset in test_sparkSQL_arrow.R

2020-08-31 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32747. -- Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved by pull

[jira] [Commented] (SPARK-32753) Deduplicating and repartitioning the same column create duplicate rows with AQE

2020-08-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17187507#comment-17187507 ] Apache Spark commented on SPARK-32753: -- User 'manuzhang' has created a pull request for this issue:

  1   2   >