[jira] [Updated] (SPARK-40474) Infer columns with mixed date and timestamp as String in CSV schema inference

2022-09-16 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40474: - Shepherd: Xiaonan Yang > Infer columns with mixed date and timestamp as String in CSV schema

[jira] [Updated] (SPARK-40474) Infer columns with mixed date and timestamp as String in CSV schema inference

2022-09-16 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40474: - Description: In this ticket, we introduced the support of date type in CSV schema inference.

[jira] [Updated] (SPARK-40474) Infer columns with mixed date and timestamp as String in CSV schema inference

2022-09-16 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40474: - Description: In this ticket, we introduced the support of date type in CSV schema inference.

[jira] [Updated] (SPARK-40474) Infer columns with mixed date and timestamp as String in CSV schema inference

2022-09-16 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40474: - Description: In this [ticket|https://issues.apache.org/jira/browse/SPARK-39469], we introduced

[jira] [Assigned] (SPARK-40473) Migrate parsing errors onto error classes

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40473: Assignee: Max Gekk (was: Apache Spark) > Migrate parsing errors onto error classes >

[jira] [Commented] (SPARK-40473) Migrate parsing errors onto error classes

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605874#comment-17605874 ] Apache Spark commented on SPARK-40473: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40473) Migrate parsing errors onto error classes

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40473: Assignee: Apache Spark (was: Max Gekk) > Migrate parsing errors onto error classes >

[jira] [Updated] (SPARK-40169) Fix the issue with Parquet column index and predicate pushdown in Data source V1

2022-09-16 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-40169: - Fix Version/s: 3.3.1 > Fix the issue with Parquet column index and predicate pushdown in Data source >

[jira] [Assigned] (SPARK-40169) Fix the issue with Parquet column index and predicate pushdown in Data source V1

2022-09-16 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-40169: Assignee: Chao Sun > Fix the issue with Parquet column index and predicate pushdown in Data

[jira] [Commented] (SPARK-39375) SPIP: Spark Connect - A client and server interface for Apache Spark

2022-09-16 Thread David Morin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605996#comment-17605996 ] David Morin commented on SPARK-39375: - Got it [~hyukjin.kwon]  

[jira] [Commented] (SPARK-40466) Improve the error message if the DSv2 source is disabled but DSv1 streaming source is not available

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605921#comment-17605921 ] Apache Spark commented on SPARK-40466: -- User 'huanliwang-db' has created a pull request for this

[jira] [Assigned] (SPARK-40466) Improve the error message if the DSv2 source is disabled but DSv1 streaming source is not available

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40466: Assignee: (was: Apache Spark) > Improve the error message if the DSv2 source is

[jira] [Assigned] (SPARK-40466) Improve the error message if the DSv2 source is disabled but DSv1 streaming source is not available

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40466: Assignee: Apache Spark > Improve the error message if the DSv2 source is disabled but

[jira] [Commented] (SPARK-40466) Improve the error message if the DSv2 source is disabled but DSv1 streaming source is not available

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605920#comment-17605920 ] Apache Spark commented on SPARK-40466: -- User 'huanliwang-db' has created a pull request for this

[jira] [Commented] (SPARK-40475) Allow job status tracking with jobGroupId

2022-09-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605981#comment-17605981 ] Dongjoon Hyun commented on SPARK-40475: --- Thank you for filing a JIRA, [~anuragmantri] . Go for it!

[jira] [Updated] (SPARK-40474) Infer columns with mixed date and timestamp as String in CSV schema inference

2022-09-16 Thread Xiaonan Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaonan Yang updated SPARK-40474: - Shepherd: (was: Xiaonan Yang) > Infer columns with mixed date and timestamp as String in CSV

[jira] [Comment Edited] (SPARK-39375) SPIP: Spark Connect - A client and server interface for Apache Spark

2022-09-16 Thread David Morin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605996#comment-17605996 ] David Morin edited comment on SPARK-39375 at 9/16/22 9:23 PM: --

[jira] [Comment Edited] (SPARK-39375) SPIP: Spark Connect - A client and server interface for Apache Spark

2022-09-16 Thread David Morin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605996#comment-17605996 ] David Morin edited comment on SPARK-39375 at 9/16/22 9:23 PM: --

[jira] [Resolved] (SPARK-40169) Fix the issue with Parquet column index and predicate pushdown in Data source V1

2022-09-16 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-40169. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37881

[jira] [Created] (SPARK-40475) Allow job status tracking with jobGroupId

2022-09-16 Thread Anurag Mantripragada (Jira)
Anurag Mantripragada created SPARK-40475: Summary: Allow job status tracking with jobGroupId Key: SPARK-40475 URL: https://issues.apache.org/jira/browse/SPARK-40475 Project: Spark

[jira] [Created] (SPARK-40473) Migrate parsing errors onto error classes

2022-09-16 Thread Max Gekk (Jira)
Max Gekk created SPARK-40473: Summary: Migrate parsing errors onto error classes Key: SPARK-40473 URL: https://issues.apache.org/jira/browse/SPARK-40473 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-40474) Infer columns with mixed date and timestamp as String in CSV schema inference

2022-09-16 Thread Xiaonan Yang (Jira)
Xiaonan Yang created SPARK-40474: Summary: Infer columns with mixed date and timestamp as String in CSV schema inference Key: SPARK-40474 URL: https://issues.apache.org/jira/browse/SPARK-40474

[jira] [Updated] (SPARK-40169) Fix the issue with Parquet column index and predicate pushdown in Data source V1

2022-09-16 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-40169: - Fix Version/s: 3.2.3 > Fix the issue with Parquet column index and predicate pushdown in Data source >

[jira] [Commented] (SPARK-39854) Catalyst 'ColumnPruning' Optimizer does not play well with sql function 'explode'

2022-09-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606016#comment-17606016 ] Dongjoon Hyun commented on SPARK-39854: --- Thank you, [~jiajiwu] . The recommended workaround would

[jira] [Resolved] (SPARK-32059) Nested Schema Pruning not Working in Window Functions

2022-09-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32059. --- Fix Version/s: 3.1.0 Assignee: Frank Yin Resolution: Fixed This was

[jira] [Assigned] (SPARK-40445) Refactor Resampler

2022-09-16 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-40445: - Assignee: Ruifeng Zheng > Refactor Resampler > -- > >

[jira] [Resolved] (SPARK-40445) Refactor Resampler

2022-09-16 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-40445. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37897

[jira] [Updated] (SPARK-39854) Catalyst 'ColumnPruning' Optimizer does not play well with sql function 'explode'

2022-09-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39854: -- Affects Version/s: 3.2.2 3.2.0 > Catalyst 'ColumnPruning' Optimizer

[jira] [Created] (SPARK-40476) Optimize the shuffle size of ALS

2022-09-16 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-40476: - Summary: Optimize the shuffle size of ALS Key: SPARK-40476 URL: https://issues.apache.org/jira/browse/SPARK-40476 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-40477) Support `NullType` in `ColumnarBatchRow`

2022-09-16 Thread Kazuyuki Tanimura (Jira)
Kazuyuki Tanimura created SPARK-40477: - Summary: Support `NullType` in `ColumnarBatchRow` Key: SPARK-40477 URL: https://issues.apache.org/jira/browse/SPARK-40477 Project: Spark Issue

[jira] [Updated] (SPARK-39656) Fix wrong namespace in DescribeNamespaceExec

2022-09-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-39656: Fix Version/s: (was: 3.1.4) (was: 3.4.0) (was:

[jira] [Resolved] (SPARK-40436) Upgrade Scala to 2.12.17

2022-09-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40436. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37892

[jira] [Assigned] (SPARK-40436) Upgrade Scala to 2.12.17

2022-09-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-40436: - Assignee: Yang Jie > Upgrade Scala to 2.12.17 > > >

[jira] [Resolved] (SPARK-40447) Implement `kendall` correlation in `DataFrame.corr`

2022-09-16 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-40447. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37913

[jira] [Assigned] (SPARK-40149) Star expansion after outer join asymmetrically includes joining key

2022-09-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-40149: --- Assignee: Wenchen Fan > Star expansion after outer join asymmetrically includes joining

[jira] [Updated] (SPARK-40476) Reduce the shuffle size of ALS

2022-09-16 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-40476: -- Summary: Reduce the shuffle size of ALS (was: Optimize the shuffle size of ALS) > Reduce

[jira] [Assigned] (SPARK-40476) Reduce the shuffle size of ALS

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40476: Assignee: Apache Spark > Reduce the shuffle size of ALS > --

[jira] [Assigned] (SPARK-40476) Reduce the shuffle size of ALS

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40476: Assignee: (was: Apache Spark) > Reduce the shuffle size of ALS >

[jira] [Commented] (SPARK-40476) Reduce the shuffle size of ALS

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606020#comment-17606020 ] Apache Spark commented on SPARK-40476: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-40471) Upgrade RoaringBitmap to 0.9.32

2022-09-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-40471: - Assignee: Yang Jie > Upgrade RoaringBitmap to 0.9.32 > ---

[jira] [Resolved] (SPARK-40471) Upgrade RoaringBitmap to 0.9.32

2022-09-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40471. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37914

[jira] [Commented] (SPARK-39854) Catalyst 'ColumnPruning' Optimizer does not play well with sql function 'explode'

2022-09-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17606023#comment-17606023 ] Dongjoon Hyun commented on SPARK-39854: --- It turns out SPARK-35194 caused this regression. I tested

[jira] [Assigned] (SPARK-40461) Set upperbound for pyzmq 24.0.0 for Python linter

2022-09-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-40461: --- Assignee: Hyukjin Kwon > Set upperbound for pyzmq 24.0.0 for Python linter >

[jira] [Reopened] (SPARK-40469) Upgrade Scala to 2.12.17

2022-09-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-40469: - > Upgrade Scala to 2.12.17 > > > Key: SPARK-40469 >

[jira] [Commented] (SPARK-40196) Consolidate `lit` function with NumPy scalar in sql and pandas module

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605658#comment-17605658 ] Apache Spark commented on SPARK-40196: -- User 'Yikun' has created a pull request for this issue:

[jira] [Updated] (SPARK-40469) Avoid creating directory failures

2022-09-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40469: Component/s: Spark Core (was: Build) > Avoid creating directory failures >

[jira] [Updated] (SPARK-40469) Avoid creating directory failures

2022-09-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40469: Description: {noformat} java.nio.file.NoSuchFileException:

[jira] [Updated] (SPARK-40469) Avoid creating directory failures

2022-09-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40469: Summary: Avoid creating directory failures (was: Upgrade Scala to 2.12.17) > Avoid creating

[jira] [Commented] (SPARK-39375) SPIP: Spark Connect - A client and server interface for Apache Spark

2022-09-16 Thread David Morin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605637#comment-17605637 ] David Morin commented on SPARK-39375: - Hi [~hyukjin.kwon] Good idea about the discussion on this

[jira] [Resolved] (SPARK-40463) Update gpg's keyserver

2022-09-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40463. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37906

[jira] [Commented] (SPARK-40447) Implement `kendall` correlation in `DataFrame.corr`

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605683#comment-17605683 ] Apache Spark commented on SPARK-40447: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-40447) Implement `kendall` correlation in `DataFrame.corr`

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40447: Assignee: Ruifeng Zheng (was: Apache Spark) > Implement `kendall` correlation in

[jira] [Assigned] (SPARK-40447) Implement `kendall` correlation in `DataFrame.corr`

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40447: Assignee: Apache Spark (was: Ruifeng Zheng) > Implement `kendall` correlation in

[jira] [Created] (SPARK-40471) Upgrade RoaringBitmap to 0.9.32

2022-09-16 Thread Yang Jie (Jira)
Yang Jie created SPARK-40471: Summary: Upgrade RoaringBitmap to 0.9.32 Key: SPARK-40471 URL: https://issues.apache.org/jira/browse/SPARK-40471 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-40469) Avoid creating directory failures

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605633#comment-17605633 ] Apache Spark commented on SPARK-40469: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40469) Avoid creating directory failures

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40469: Assignee: (was: Apache Spark) > Avoid creating directory failures >

[jira] [Assigned] (SPARK-40469) Avoid creating directory failures

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40469: Assignee: Apache Spark > Avoid creating directory failures >

[jira] [Updated] (SPARK-40470) arrays_zip output unexpected alias column names when using GetMapValue and GetArrayStructFields

2022-09-16 Thread Ivan Sadikov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Sadikov updated SPARK-40470: - Summary: arrays_zip output unexpected alias column names when using GetMapValue and

[jira] [Assigned] (SPARK-40470) arrays_zip output unexpected alias column names when using GetMapValue and GetArrayStructFields

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40470: Assignee: (was: Apache Spark) > arrays_zip output unexpected alias column names when

[jira] [Commented] (SPARK-40470) arrays_zip output unexpected alias column names when using GetMapValue and GetArrayStructFields

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605651#comment-17605651 ] Apache Spark commented on SPARK-40470: -- User 'sadikovi' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40470) arrays_zip output unexpected alias column names when using GetMapValue and GetArrayStructFields

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40470: Assignee: Apache Spark > arrays_zip output unexpected alias column names when using

[jira] [Commented] (SPARK-40196) Consolidate `lit` function with NumPy scalar in sql and pandas module

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605657#comment-17605657 ] Apache Spark commented on SPARK-40196: -- User 'Yikun' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40463) Update gpg's keyserver

2022-09-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40463: Assignee: Yuming Wang > Update gpg's keyserver > -- > >

[jira] [Assigned] (SPARK-40448) Prototype Implementation

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40448: Assignee: Apache Spark > Prototype Implementation > > >

[jira] [Commented] (SPARK-40448) Prototype Implementation

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605782#comment-17605782 ] Apache Spark commented on SPARK-40448: -- User 'grundprinzip' has created a pull request for this

[jira] [Commented] (SPARK-40448) Prototype Implementation

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605781#comment-17605781 ] Apache Spark commented on SPARK-40448: -- User 'grundprinzip' has created a pull request for this

[jira] [Assigned] (SPARK-40448) Prototype Implementation

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40448: Assignee: (was: Apache Spark) > Prototype Implementation >

[jira] [Updated] (SPARK-40449) Extend test coverage of Planner

2022-09-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-40449: - Summary: Extend test coverage of Planner (was: Extend Test Coverage of Planner) > Extend test

[jira] [Updated] (SPARK-40448) Prototype implementation

2022-09-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-40448: - Summary: Prototype implementation (was: Prototype Implementation) > Prototype implementation >

[jira] [Updated] (SPARK-39673) High-Level design doc for Spark Connect

2022-09-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39673: - Summary: High-Level design doc for Spark Connect (was: High-Level Design Doc for Spark

[jira] [Updated] (SPARK-40449) Extend test coverage of Catalyst optimizer

2022-09-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-40449: - Summary: Extend test coverage of Catalyst optimizer (was: Extend test coverage of Planner) >

[jira] [Updated] (SPARK-40449) Extend test coverage of Analyzer

2022-09-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-40449: - Summary: Extend test coverage of Analyzer (was: Extend test coverage of Catalyst optimizer) >

[jira] [Updated] (SPARK-40452) Developer documentation

2022-09-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-40452: - Summary: Developer documentation (was: Developer Documentation) > Developer documentation >

[jira] [Updated] (SPARK-40453) Improve error handling for GRPC server

2022-09-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-40453: - Summary: Improve error handling for GRPC server (was: Improve Error handling for GRPC server)

[jira] [Updated] (SPARK-39674) Initial protobuf definition for Spark Connect API

2022-09-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39674: - Summary: Initial protobuf definition for Spark Connect API (was: Initial Protobuf Definition

[jira] [Commented] (SPARK-38618) Implement JDBCDataSourceV2

2022-09-16 Thread shengkui leng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605700#comment-17605700 ] shengkui leng commented on SPARK-38618: --- Any update for this task? > Implement JDBCDataSourceV2 >

[jira] [Assigned] (SPARK-40465) Refactor Decimal so as we can use Int128 as underlying implementation

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40465: Assignee: Apache Spark > Refactor Decimal so as we can use Int128 as underlying

[jira] [Commented] (SPARK-40465) Refactor Decimal so as we can use Int128 as underlying implementation

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605749#comment-17605749 ] Apache Spark commented on SPARK-40465: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40465) Refactor Decimal so as we can use Int128 as underlying implementation

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40465: Assignee: (was: Apache Spark) > Refactor Decimal so as we can use Int128 as

[jira] [Commented] (SPARK-40441) With PANDAS_UDF, data from tasks on the same physical node is aggregated into one task execution, resulting in concurrency not being fully utilized

2022-09-16 Thread SimonAries (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605755#comment-17605755 ] SimonAries commented on SPARK-40441: Let me try that > With PANDAS_UDF, data from tasks on the same

[jira] [Assigned] (SPARK-40471) Upgrade RoaringBitmap to 0.9.32

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40471: Assignee: (was: Apache Spark) > Upgrade RoaringBitmap to 0.9.32 >

[jira] [Assigned] (SPARK-40471) Upgrade RoaringBitmap to 0.9.32

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40471: Assignee: Apache Spark > Upgrade RoaringBitmap to 0.9.32 >

[jira] [Commented] (SPARK-40471) Upgrade RoaringBitmap to 0.9.32

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605692#comment-17605692 ] Apache Spark commented on SPARK-40471: -- User 'LuciferYang' has created a pull request for this

[jira] [Commented] (SPARK-40471) Upgrade RoaringBitmap to 0.9.32

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605693#comment-17605693 ] Apache Spark commented on SPARK-40471: -- User 'LuciferYang' has created a pull request for this

[jira] [Resolved] (SPARK-40467) Split FlatMapGroupsWithState down to multiple test suites

2022-09-16 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-40467. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37907

[jira] [Commented] (SPARK-40465) Refactor Decimal so as we can use Int128 as underlying implementation

2022-09-16 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17605751#comment-17605751 ] Apache Spark commented on SPARK-40465: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40467) Split FlatMapGroupsWithState down to multiple test suites

2022-09-16 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-40467: Assignee: Jungtaek Lim > Split FlatMapGroupsWithState down to multiple test suites >

[jira] [Created] (SPARK-40472) Improve pyspark.sql.function example experience

2022-09-16 Thread deshanxiao (Jira)
deshanxiao created SPARK-40472: -- Summary: Improve pyspark.sql.function example experience Key: SPARK-40472 URL: https://issues.apache.org/jira/browse/SPARK-40472 Project: Spark Issue Type:

[jira] [Updated] (SPARK-40472) Improve pyspark.sql.function example experience

2022-09-16 Thread deshanxiao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deshanxiao updated SPARK-40472: --- Description: There are many exanple in pyspark.sql.function: {code:java}     Examples      

[jira] [Resolved] (SPARK-40470) arrays_zip output unexpected alias column names when using GetMapValue and GetArrayStructFields

2022-09-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40470. -- Fix Version/s: 3.3.1 3.2.3 3.4.0 Resolution:

[jira] [Assigned] (SPARK-40470) arrays_zip output unexpected alias column names when using GetMapValue and GetArrayStructFields

2022-09-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40470: Assignee: Ivan Sadikov > arrays_zip output unexpected alias column names when using

[jira] [Assigned] (SPARK-40398) Use Loop instead of Arrays.stream api

2022-09-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-40398: Assignee: Yang Jie > Use Loop instead of Arrays.stream api >

[jira] [Resolved] (SPARK-40398) Use Loop instead of Arrays.stream api

2022-09-16 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-40398. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37843