[jira] [Assigned] (SPARK-35783) Set the list of read columns in the task configuration to reduce reading of ORC data.

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35783: Assignee: (was: Apache Spark) > Set the list of read columns in the task

[jira] [Assigned] (SPARK-35783) Set the list of read columns in the task configuration to reduce reading of ORC data.

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35783: Assignee: Apache Spark > Set the list of read columns in the task configuration to

[jira] [Assigned] (SPARK-35622) DataFrame's count function do not need groupBy and avoid shuffle

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35622: Assignee: Apache Spark > DataFrame's count function do not need groupBy and avoid

[jira] [Commented] (SPARK-35622) DataFrame's count function do not need groupBy and avoid shuffle

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364074#comment-17364074 ] Apache Spark commented on SPARK-35622: -- User 'StefanXiepj' has created a pull request for this

[jira] [Assigned] (SPARK-35622) DataFrame's count function do not need groupBy and avoid shuffle

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35622: Assignee: (was: Apache Spark) > DataFrame's count function do not need groupBy and

[jira] [Commented] (SPARK-35783) Set the list of read columns in the task configuration to reduce reading of ORC data.

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364065#comment-17364065 ] Apache Spark commented on SPARK-35783: -- User 'weixiuli' has created a pull request for this issue:

[jira] [Commented] (SPARK-35771) Format year-month intervals using type fields

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364069#comment-17364069 ] Apache Spark commented on SPARK-35771: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35771) Format year-month intervals using type fields

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35771: Assignee: Kousuke Saruta (was: Apache Spark) > Format year-month intervals using type

[jira] [Commented] (SPARK-35771) Format year-month intervals using type fields

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364068#comment-17364068 ] Apache Spark commented on SPARK-35771: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35771) Format year-month intervals using type fields

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35771: Assignee: Apache Spark (was: Kousuke Saruta) > Format year-month intervals using type

[jira] [Commented] (SPARK-35783) Set the list of read columns in the task configuration to reduce reading of ORC data.

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364066#comment-17364066 ] Apache Spark commented on SPARK-35783: -- User 'weixiuli' has created a pull request for this issue:

[jira] [Updated] (SPARK-35783) Set the list of read columns in the task configuration to reduce reading of ORC data.

2021-06-15 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-35783: - Summary: Set the list of read columns in the task configuration to reduce reading of ORC data. (was:

[jira] [Updated] (SPARK-35783) Set the list of read columns in the task configuration to reduce read data for ORC

2021-06-15 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-35783: - Description: Now, if the read column list is not set in the task configuration, it will read all

[jira] [Assigned] (SPARK-35774) Parse any year-month interval types in SQL

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35774: Assignee: Kousuke Saruta (was: Apache Spark) > Parse any year-month interval types in

[jira] [Commented] (SPARK-35774) Parse any year-month interval types in SQL

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364064#comment-17364064 ] Apache Spark commented on SPARK-35774: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35774) Parse any year-month interval types in SQL

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35774: Assignee: Apache Spark (was: Kousuke Saruta) > Parse any year-month interval types in

[jira] [Updated] (SPARK-35783) Set the list of read columns in the task configuration to reduce read data for ORC

2021-06-15 Thread weixiuli (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-35783: - Summary: Set the list of read columns in the task configuration to reduce read data for ORC (was: Set

[jira] [Commented] (SPARK-35774) Parse any year-month interval types in SQL

2021-06-15 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364063#comment-17364063 ] Kousuke Saruta commented on SPARK-35774: [~maxgekk]Thank you for letting me know. Actually, I'm

[jira] [Comment Edited] (SPARK-35774) Parse any year-month interval types in SQL

2021-06-15 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364063#comment-17364063 ] Kousuke Saruta edited comment on SPARK-35774 at 6/16/21, 5:13 AM: --

[jira] [Created] (SPARK-35783) Set the list of read columns in the task configuration for reducing read data for ORC

2021-06-15 Thread weixiuli (Jira)
weixiuli created SPARK-35783: Summary: Set the list of read columns in the task configuration for reducing read data for ORC Key: SPARK-35783 URL: https://issues.apache.org/jira/browse/SPARK-35783

[jira] [Commented] (SPARK-35774) Parse any year-month interval types in SQL

2021-06-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364060#comment-17364060 ] Max Gekk commented on SPARK-35774: -- [~sarutak] Would you like to work on this? > Parse any year-month

[jira] [Commented] (SPARK-35773) Parse year-month interval literals to tightest types

2021-06-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364059#comment-17364059 ] Max Gekk commented on SPARK-35773: -- [~sarutak] FYI, this is similar to what you have done already.

[jira] [Commented] (SPARK-35771) Format year-month intervals using type fields

2021-06-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364058#comment-17364058 ] Max Gekk commented on SPARK-35771: -- [~sarutak] Would you like to work on this? > Format year-month

[jira] [Updated] (SPARK-35782) leveldbjni doesn't work in Apple Silicon on macOS

2021-06-15 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-35782: Description: leveldbjni doesn't contain the native library for Apple Silicon on macOS. We will need to

[jira] [Created] (SPARK-35782) leveldbjni doesn't work in Apple Silicon on macOS

2021-06-15 Thread DB Tsai (Jira)
DB Tsai created SPARK-35782: --- Summary: leveldbjni doesn't work in Apple Silicon on macOS Key: SPARK-35782 URL: https://issues.apache.org/jira/browse/SPARK-35782 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-35781) Support Spark on Apple Silicon on macOS natively

2021-06-15 Thread DB Tsai (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-35781: Description: This is an umbrella JIRA tracking the progress of supporting Apple Silicon on macOS

[jira] [Created] (SPARK-35781) Support Spark on Apple Silicon on macOS natively

2021-06-15 Thread DB Tsai (Jira)
DB Tsai created SPARK-35781: --- Summary: Support Spark on Apple Silicon on macOS natively Key: SPARK-35781 URL: https://issues.apache.org/jira/browse/SPARK-35781 Project: Spark Issue Type: New

[jira] [Resolved] (SPARK-35613) Cache commonly occurring strings from SQLMetrics, JsonProtocol and AccumulatorV2

2021-06-15 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-35613. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-35613) Cache commonly occurring strings from SQLMetrics, JsonProtocol and AccumulatorV2

2021-06-15 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-35613: --- Assignee: Venkata krishnan Sowrirajan > Cache commonly occurring strings

[jira] [Created] (SPARK-35780) Support DATE/TIMESTAMP literals across the full range

2021-06-15 Thread Linhong Liu (Jira)
Linhong Liu created SPARK-35780: --- Summary: Support DATE/TIMESTAMP literals across the full range Key: SPARK-35780 URL: https://issues.apache.org/jira/browse/SPARK-35780 Project: Spark Issue

[jira] [Resolved] (SPARK-35203) Improve Repartition statistics estimation

2021-06-15 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-35203. -- Fix Version/s: 3.2.0 Assignee: Yuming Wang Resolution: Fixed Resolved

[jira] [Updated] (SPARK-35707) optimize sparse GEMM by skipping bound checking

2021-06-15 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-35707: - Affects Version/s: (was: 3.1.3) (was: 3.0.3) > optimize sparse

[jira] [Resolved] (SPARK-35666) add new gemv to skip array shape checking

2021-06-15 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-35666. -- Assignee: zhengruifeng Resolution: Resolved > add new gemv to skip array shape checking

[jira] [Resolved] (SPARK-35707) optimize sparse GEMM by skipping bound checking

2021-06-15 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-35707. -- Assignee: zhengruifeng Resolution: Resolved > optimize sparse GEMM by skipping bound

[jira] [Resolved] (SPARK-35669) fix special char in CSV header with filter pushdown

2021-06-15 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-35669. -- Fix Version/s: 3.2.0 Assignee: Wenchen Fan Resolution: Fixed Resolved

[jira] [Updated] (SPARK-35669) fix special char in CSV header with filter pushdown

2021-06-15 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-35669: - Description: This is a followup of [#31964|https://github.com/apache/spark/pull/31964]

[jira] [Resolved] (SPARK-35691) addFile/addJar/addDirectory should put CanonicalFile

2021-06-15 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-35691. Fix Version/s: 3.2.0 Assignee: Kevin Su Resolution: Fixed Issue resolved

[jira] [Resolved] (SPARK-35760) Fix the max rows check for broadcast exchange

2021-06-15 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-35760. -- Fix Version/s: 3.2.0 Assignee: Cheng Su Resolution: Fixed Resolved by 

[jira] [Commented] (SPARK-34403) Remove dependency to commons-httpclient, is not used and has vulnerabilities.

2021-06-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363924#comment-17363924 ] Dongjoon Hyun commented on SPARK-34403: --- This is partially resolved via SPARK-35429 for Hadoop 3.2

[jira] [Commented] (SPARK-35429) Remove commons-httpclient due to EOL and CVEs

2021-06-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363922#comment-17363922 ] Dongjoon Hyun commented on SPARK-35429: --- This is resolved via

[jira] [Updated] (SPARK-35429) Remove commons-httpclient due to EOL and CVEs

2021-06-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35429: -- Fix Version/s: 3.2.0 > Remove commons-httpclient due to EOL and CVEs >

[jira] [Assigned] (SPARK-35429) Remove commons-httpclient due to EOL and CVEs

2021-06-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35429: - Assignee: Sumeet > Remove commons-httpclient due to EOL and CVEs >

[jira] [Commented] (SPARK-35779) Support dynamic filtering for v2 tables

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363920#comment-17363920 ] Apache Spark commented on SPARK-35779: -- User 'aokolnychyi' has created a pull request for this

[jira] [Commented] (SPARK-35779) Support dynamic filtering for v2 tables

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363919#comment-17363919 ] Apache Spark commented on SPARK-35779: -- User 'aokolnychyi' has created a pull request for this

[jira] [Assigned] (SPARK-35779) Support dynamic filtering for v2 tables

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35779: Assignee: (was: Apache Spark) > Support dynamic filtering for v2 tables >

[jira] [Assigned] (SPARK-35779) Support dynamic filtering for v2 tables

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35779: Assignee: Apache Spark > Support dynamic filtering for v2 tables >

[jira] [Updated] (SPARK-35344) Make conversion from/to literals data-type-based

2021-06-15 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35344: - Description: Aim to achieve {{F}}{{*.*}}{{lit(np}}{{*.*}}{{int64(1)).}} We can define {{def

[jira] [Commented] (SPARK-35344) Make conversion from/to literals data-type-based

2021-06-15 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363906#comment-17363906 ] Xinrong Meng commented on SPARK-35344: -- I'd like to work on this ticket. > Make conversion from/to

[jira] [Created] (SPARK-35779) Support dynamic filtering for v2 tables

2021-06-15 Thread Anton Okolnychyi (Jira)
Anton Okolnychyi created SPARK-35779: Summary: Support dynamic filtering for v2 tables Key: SPARK-35779 URL: https://issues.apache.org/jira/browse/SPARK-35779 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-35680) Support fields by year-month interval type

2021-06-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35680. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32909

[jira] [Resolved] (SPARK-35767) CoalesceExec can execute child plan twice

2021-06-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35767. --- Fix Version/s: 3.1.3 3.2.0 3.0.3 Assignee:

[jira] [Updated] (SPARK-35767) CoalesceExec can execute child plan twice

2021-06-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35767: -- Affects Version/s: 2.3.0 2.3.4 2.4.0

[jira] [Updated] (SPARK-35767) CoalesceExec can execute child plan twice

2021-06-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35767: -- Summary: CoalesceExec can execute child plan twice (was: [SQL] CoalesceExec can execute

[jira] [Updated] (SPARK-35767) [SQL] CoalesceExec can execute child plan twice

2021-06-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35767: -- Affects Version/s: 3.2.0 > [SQL] CoalesceExec can execute child plan twice >

[jira] [Resolved] (SPARK-34847) Simplify ResolveAggregateFunctions

2021-06-15 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34847. - Resolution: Duplicate > Simplify ResolveAggregateFunctions > --

[jira] [Created] (SPARK-35778) Check multiply/divide of year-month intervals of any fields by numeric

2021-06-15 Thread Max Gekk (Jira)
Max Gekk created SPARK-35778: Summary: Check multiply/divide of year-month intervals of any fields by numeric Key: SPARK-35778 URL: https://issues.apache.org/jira/browse/SPARK-35778 Project: Spark

[jira] [Updated] (SPARK-35778) Check multiply/divide of year-month intervals of any fields by numeric

2021-06-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-35778: - Description: Write tests that checks multiply/divide of the following intervals by numeric: # INTERVAL

[jira] [Created] (SPARK-35777) Check all year-month interval types in UDF

2021-06-15 Thread Max Gekk (Jira)
Max Gekk created SPARK-35777: Summary: Check all year-month interval types in UDF Key: SPARK-35777 URL: https://issues.apache.org/jira/browse/SPARK-35777 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-35777) Check all year-month interval types in UDF

2021-06-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-35777: - Description: Check all year-month interval types in UDF: # INTERVAL YEAR # INTERVAL YEAR TO MONTH #

[jira] [Updated] (SPARK-35776) Check all year-month interval types in arrow

2021-06-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-35776: - Description: Add tests to check that all year-month interval types are supported in (de-)serialization

[jira] [Created] (SPARK-35776) Check all year-month interval types in arrow

2021-06-15 Thread Max Gekk (Jira)
Max Gekk created SPARK-35776: Summary: Check all year-month interval types in arrow Key: SPARK-35776 URL: https://issues.apache.org/jira/browse/SPARK-35776 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-35775) Check all year-month interval types in aggregate expressions

2021-06-15 Thread Max Gekk (Jira)
Max Gekk created SPARK-35775: Summary: Check all year-month interval types in aggregate expressions Key: SPARK-35775 URL: https://issues.apache.org/jira/browse/SPARK-35775 Project: Spark Issue

[jira] [Updated] (SPARK-35775) Check all year-month interval types in aggregate expressions

2021-06-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-35775: - Description: Check all supported combination of YearMonthIntervalType fields in the aggregate

[jira] [Updated] (SPARK-35774) Parse any year-month interval types in SQL

2021-06-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-35774: - Description: Extend the parser rules to be able to parse the following types: # INTERVAL YEAR #

[jira] [Created] (SPARK-35774) Parse any year-month interval types in SQL

2021-06-15 Thread Max Gekk (Jira)
Max Gekk created SPARK-35774: Summary: Parse any year-month interval types in SQL Key: SPARK-35774 URL: https://issues.apache.org/jira/browse/SPARK-35774 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-35773) Parse year-month interval literals to tightest types

2021-06-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-35773: - Description: Modify AstBuilder.visitInterval to parse year-month interval literals to tightest types.

[jira] [Created] (SPARK-35773) Parse year-month interval literals to tightest types

2021-06-15 Thread Max Gekk (Jira)
Max Gekk created SPARK-35773: Summary: Parse year-month interval literals to tightest types Key: SPARK-35773 URL: https://issues.apache.org/jira/browse/SPARK-35773 Project: Spark Issue Type:

[jira] [Updated] (SPARK-35772) Check all year-month interval types in HiveInspectors tests

2021-06-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-35772: - Description: Check all year-month interval types are supported by HiveInspectors: # INTERVAL YEAR #

[jira] [Created] (SPARK-35772) Check all year-month interval types in HiveInspectors tests

2021-06-15 Thread Max Gekk (Jira)
Max Gekk created SPARK-35772: Summary: Check all year-month interval types in HiveInspectors tests Key: SPARK-35772 URL: https://issues.apache.org/jira/browse/SPARK-35772 Project: Spark Issue

[jira] [Updated] (SPARK-35771) Format year-month intervals using type fields

2021-06-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-35771: - Description: Format year-month interval to strings using the start and end fields of

[jira] [Created] (SPARK-35771) Format year-month intervals using type fields

2021-06-15 Thread Max Gekk (Jira)
Max Gekk created SPARK-35771: Summary: Format year-month intervals using type fields Key: SPARK-35771 URL: https://issues.apache.org/jira/browse/SPARK-35771 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-35770) Parse YearMonthIntervalType from JSON

2021-06-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-35770: - Description: Allow (de-)serialization of the YearMonthIntervalType class to JSON. See

[jira] [Created] (SPARK-35770) Parse YearMonthIntervalType from JSON

2021-06-15 Thread Max Gekk (Jira)
Max Gekk created SPARK-35770: Summary: Parse YearMonthIntervalType from JSON Key: SPARK-35770 URL: https://issues.apache.org/jira/browse/SPARK-35770 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-35769) Truncate java.time.Period by fields of year-month interval type

2021-06-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-35769: - Description: Truncate input java.time.Period instances using fields of YearMonthIntervalType. For

[jira] [Created] (SPARK-35769) Truncate java.time.Period by fields of year-month interval type

2021-06-15 Thread Max Gekk (Jira)
Max Gekk created SPARK-35769: Summary: Truncate java.time.Period by fields of year-month interval type Key: SPARK-35769 URL: https://issues.apache.org/jira/browse/SPARK-35769 Project: Spark

[jira] [Updated] (SPARK-35768) Take into account year-month interval fields in cast

2021-06-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-35768: - Description: Take into account year-month interval fields in casts of input values (strings for

[jira] [Created] (SPARK-35768) Take into account year-month interval fields in cast

2021-06-15 Thread Max Gekk (Jira)
Max Gekk created SPARK-35768: Summary: Take into account year-month interval fields in cast Key: SPARK-35768 URL: https://issues.apache.org/jira/browse/SPARK-35768 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-35767) [SQL] CoalesceExec can execute child plan twice

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35767: Assignee: (was: Apache Spark) > [SQL] CoalesceExec can execute child plan twice >

[jira] [Assigned] (SPARK-35767) [SQL] CoalesceExec can execute child plan twice

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35767: Assignee: Apache Spark > [SQL] CoalesceExec can execute child plan twice >

[jira] [Commented] (SPARK-35767) [SQL] CoalesceExec can execute child plan twice

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363756#comment-17363756 ] Apache Spark commented on SPARK-35767: -- User 'andygrove' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35129) Construct year-month interval column from integral fields

2021-06-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-35129: Assignee: angerszhu > Construct year-month interval column from integral fields >

[jira] [Resolved] (SPARK-35129) Construct year-month interval column from integral fields

2021-06-15 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35129. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32645

[jira] [Resolved] (SPARK-35766) Break down CastSuite/AnsiCastSuite into multiple files

2021-06-15 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-35766. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32918

[jira] [Updated] (SPARK-35767) [SQL] CoalesceExec can execute child plan twice

2021-06-15 Thread Andy Grove (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Grove updated SPARK-35767: --- Priority: Minor (was: Major) > [SQL] CoalesceExec can execute child plan twice >

[jira] [Created] (SPARK-35767) [SQL] CoalesceExec can execute child plan twice

2021-06-15 Thread Andy Grove (Jira)
Andy Grove created SPARK-35767: -- Summary: [SQL] CoalesceExec can execute child plan twice Key: SPARK-35767 URL: https://issues.apache.org/jira/browse/SPARK-35767 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-35378) Eagerly execute commands in QueryExecution instead of caller sides

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363730#comment-17363730 ] Apache Spark commented on SPARK-35378: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-35766) Break down CastSuite/AnsiCastSuite into multiple files

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363682#comment-17363682 ] Apache Spark commented on SPARK-35766: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-35766) Break down CastSuite/AnsiCastSuite into multiple files

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35766: Assignee: Apache Spark (was: Gengliang Wang) > Break down CastSuite/AnsiCastSuite into

[jira] [Assigned] (SPARK-35766) Break down CastSuite/AnsiCastSuite into multiple files

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35766: Assignee: Gengliang Wang (was: Apache Spark) > Break down CastSuite/AnsiCastSuite into

[jira] [Updated] (SPARK-35766) Break down CastSuite/AnsiCastSuite into multiple files

2021-06-15 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-35766: --- Priority: Minor (was: Major) > Break down CastSuite/AnsiCastSuite into multiple files >

[jira] [Created] (SPARK-35766) Break down CastSuite/AnsiCastSuite into multiple files

2021-06-15 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-35766: -- Summary: Break down CastSuite/AnsiCastSuite into multiple files Key: SPARK-35766 URL: https://issues.apache.org/jira/browse/SPARK-35766 Project: Spark

[jira] [Resolved] (SPARK-35722) wait until something does get queued

2021-06-15 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-35722. -- Fix Version/s: (was: 3.2.0) Resolution: Not A Problem > wait until something does

[jira] [Commented] (SPARK-35089) non consistent results running count for same dataset after filter and lead window function

2021-06-15 Thread Domagoj (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363657#comment-17363657 ] Domagoj commented on SPARK-35089: - I can create similar solution with creating snapshot after

[jira] [Assigned] (SPARK-35765) Distinct aggs are not duplicate sensitive

2021-06-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-35765: Assignee: Tanel Kiis > Distinct aggs are not duplicate sensitive >

[jira] [Resolved] (SPARK-35765) Distinct aggs are not duplicate sensitive

2021-06-15 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35765. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32904

[jira] [Commented] (SPARK-35622) DataFrame's count function do not need groupBy and avoid shuffle

2021-06-15 Thread xiepengjie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363605#comment-17363605 ] xiepengjie commented on SPARK-35622: [~hyukjin.kwon] [~dc-heros], Sorry, I was on vacation last week

[jira] [Commented] (SPARK-35765) Distinct aggs are not duplicate sensitive

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363590#comment-17363590 ] Apache Spark commented on SPARK-35765: -- User 'tanelk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35765) Distinct aggs are not duplicate sensitive

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35765: Assignee: (was: Apache Spark) > Distinct aggs are not duplicate sensitive >

[jira] [Commented] (SPARK-35765) Distinct aggs are not duplicate sensitive

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17363589#comment-17363589 ] Apache Spark commented on SPARK-35765: -- User 'tanelk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-35765) Distinct aggs are not duplicate sensitive

2021-06-15 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35765: Assignee: Apache Spark > Distinct aggs are not duplicate sensitive >

  1   2   >