[jira] [Resolved] (SPARK-40401) Remove the support of deprecated `spark.akka.*` config

2022-09-09 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40401. - Fix Version/s: 3.4.0 Assignee: Dongjoon Hyun Resolution: Fixed > Remove the

[jira] [Assigned] (SPARK-40401) Remove the support of deprecated `spark.akka.*` config

2022-09-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40401: Assignee: Apache Spark > Remove the support of deprecated `spark.akka.*` config >

[jira] [Commented] (SPARK-40401) Remove the support of deprecated `spark.akka.*` config

2022-09-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602586#comment-17602586 ] Apache Spark commented on SPARK-40401: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-40401) Remove the support of deprecated `spark.akka.*` config

2022-09-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40401: Assignee: (was: Apache Spark) > Remove the support of deprecated `spark.akka.*`

[jira] [Commented] (SPARK-40401) Remove the support of deprecated `spark.akka.*` config

2022-09-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602585#comment-17602585 ] Apache Spark commented on SPARK-40401: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-40401) Remove the support of deprecated `spark.akka.*` config

2022-09-09 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-40401: - Summary: Remove the support of deprecated `spark.akka.*` config Key: SPARK-40401 URL: https://issues.apache.org/jira/browse/SPARK-40401 Project: Spark

[jira] [Resolved] (SPARK-39915) Dataset.repartition(N) may not create N partitions

2022-09-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-39915. --- Fix Version/s: 3.4.0 3.3.1 Assignee: XiDuo You

[jira] [Commented] (SPARK-39830) Add a test case to read ORC table that requires type promotion

2022-09-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602562#comment-17602562 ] Dongjoon Hyun commented on SPARK-39830: --- This is backported to branch-3.3 via

[jira] [Updated] (SPARK-39830) Add a test case to read ORC table that requires type promotion

2022-09-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-39830: -- Fix Version/s: 3.3.1 > Add a test case to read ORC table that requires type promotion >

[jira] [Assigned] (SPARK-40168) Handle FileNotFoundException when shuffle file deleted in decommissioner

2022-09-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-40168: - Assignee: Zhongwei Zhu > Handle FileNotFoundException when shuffle file deleted in

[jira] [Resolved] (SPARK-40168) Handle FileNotFoundException when shuffle file deleted in decommissioner

2022-09-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40168. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37603

[jira] [Updated] (SPARK-40269) Randomize the orders of peer in BlockManagerDecommissioner

2022-09-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40269: -- Affects Version/s: 3.4.0 (was: 3.3.0) > Randomize the orders of

[jira] [Updated] (SPARK-40269) Randomize the orders of peer in BlockManagerDecommissioner

2022-09-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40269: -- Component/s: Spark Core (was: Block Manager) > Randomize the orders of

[jira] [Assigned] (SPARK-40269) Randomize the orders of peer in BlockManagerDecommissioner

2022-09-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-40269: - Assignee: Zhongwei Zhu > Randomize the orders of peer in BlockManagerDecommissioner >

[jira] [Resolved] (SPARK-40269) Randomize the orders of peer in BlockManagerDecommissioner

2022-09-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40269. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37716

[jira] [Commented] (SPARK-40389) Decimals can't upcast as integral types if the cast can overflow

2022-09-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602508#comment-17602508 ] Apache Spark commented on SPARK-40389: -- User 'gengliangwang' has created a pull request for this

[jira] [Resolved] (SPARK-40034) PathOutputCommitters to work with dynamic partition overwrite

2022-09-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40034. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37468

[jira] [Assigned] (SPARK-40034) PathOutputCommitters to work with dynamic partition overwrite

2022-09-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-40034: - Assignee: Steve Loughran > PathOutputCommitters to work with dynamic partition

[jira] [Commented] (SPARK-40400) Pass error message parameters to exceptions as a map

2022-09-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602475#comment-17602475 ] Apache Spark commented on SPARK-40400: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40400) Pass error message parameters to exceptions as a map

2022-09-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40400: Assignee: Apache Spark (was: Max Gekk) > Pass error message parameters to exceptions as

[jira] [Commented] (SPARK-40400) Pass error message parameters to exceptions as a map

2022-09-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602473#comment-17602473 ] Apache Spark commented on SPARK-40400: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-40400) Pass error message parameters to exceptions as a map

2022-09-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40400: Assignee: Max Gekk (was: Apache Spark) > Pass error message parameters to exceptions as

[jira] [Created] (SPARK-40400) Pass error message parameters to exceptions as a map

2022-09-09 Thread Max Gekk (Jira)
Max Gekk created SPARK-40400: Summary: Pass error message parameters to exceptions as a map Key: SPARK-40400 URL: https://issues.apache.org/jira/browse/SPARK-40400 Project: Spark Issue Type:

[jira] [Commented] (SPARK-40280) Failure to create parquet predicate push down for ints and longs on some valid files

2022-09-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602323#comment-17602323 ] Apache Spark commented on SPARK-40280: -- User 'zzcclp' has created a pull request for this issue:

[jira] [Commented] (SPARK-40280) Failure to create parquet predicate push down for ints and longs on some valid files

2022-09-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602322#comment-17602322 ] Apache Spark commented on SPARK-40280: -- User 'zzcclp' has created a pull request for this issue:

[jira] [Commented] (SPARK-40280) Failure to create parquet predicate push down for ints and longs on some valid files

2022-09-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602317#comment-17602317 ] Apache Spark commented on SPARK-40280: -- User 'zzcclp' has created a pull request for this issue:

[jira] [Commented] (SPARK-40280) Failure to create parquet predicate push down for ints and longs on some valid files

2022-09-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602316#comment-17602316 ] Apache Spark commented on SPARK-40280: -- User 'zzcclp' has created a pull request for this issue:

[jira] [Commented] (SPARK-40388) SQL configuration spark.sql.mapKeyDedupPolicy not always applied

2022-09-09 Thread zzzzming95 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602307#comment-17602307 ] ming95 commented on SPARK-40388: Hi [~praetp] , can you provide a way to reproduce this bug? >

[jira] [Updated] (SPARK-40390) Spark Master UI - SSL implementation

2022-09-09 Thread Rhajvijay Manoharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rhajvijay Manoharan updated SPARK-40390: Priority: Major (was: Minor) > Spark Master UI - SSL implementation >

[jira] [Commented] (SPARK-39375) SPIP: Spark Connect - A client and server interface for Apache Spark.

2022-09-09 Thread David Morin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602218#comment-17602218 ] David Morin commented on SPARK-39375: - Amazing! I look forward to get some news about this proposal.

[jira] [Updated] (SPARK-40399) Make `pearson` correlation in `DataFrame.corr` support missing values and `min_periods`

2022-09-09 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-40399: -- Summary: Make `pearson` correlation in `DataFrame.corr` support missing values and

[jira] [Assigned] (SPARK-40399) DataFrame.corr `Pearson` support missing values and `min_periods `

2022-09-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40399: Assignee: Apache Spark > DataFrame.corr `Pearson` support missing values and

[jira] [Commented] (SPARK-40399) DataFrame.corr `Pearson` support missing values and `min_periods `

2022-09-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602217#comment-17602217 ] Apache Spark commented on SPARK-40399: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-40399) DataFrame.corr `Pearson` support missing values and `min_periods `

2022-09-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602216#comment-17602216 ] Apache Spark commented on SPARK-40399: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-40399) DataFrame.corr `Pearson` support missing values and `min_periods `

2022-09-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40399: Assignee: (was: Apache Spark) > DataFrame.corr `Pearson` support missing values and

[jira] [Updated] (SPARK-40399) DataFrame.corr `Pearson` support missing values and `min_periods `

2022-09-09 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-40399: -- Summary: DataFrame.corr `Pearson` support missing values and `min_periods ` (was:

[jira] [Created] (SPARK-40399) DataFrame.corr Pearson support missing values and `min_periods `

2022-09-09 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-40399: - Summary: DataFrame.corr Pearson support missing values and `min_periods ` Key: SPARK-40399 URL: https://issues.apache.org/jira/browse/SPARK-40399 Project: Spark

[jira] [Commented] (SPARK-39763) Executor memory footprint substantially increases while reading zstd compressed parquet files

2022-09-09 Thread Fengyu Cao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602154#comment-17602154 ] Fengyu Cao commented on SPARK-39763: [https://github.com/apache/parquet-mr/pull/982] > Executor