[jira] [Assigned] (SPARK-36681) Fail to load Snappy codec

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36681: Assignee: Apache Spark > Fail to load Snappy codec > - > >

[jira] [Assigned] (SPARK-36681) Fail to load Snappy codec

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36681: Assignee: (was: Apache Spark) > Fail to load Snappy codec >

[jira] [Commented] (SPARK-36681) Fail to load Snappy codec

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503362#comment-17503362 ] Apache Spark commented on SPARK-36681: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-38480) Remove spark.kubernetes.job.queue in favor of spark.kubernetes.driver.podGroupTemplateFile

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503351#comment-17503351 ] Apache Spark commented on SPARK-38480: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-38480) Remove spark.kubernetes.job.queue in favor of spark.kubernetes.driver.podGroupTemplateFile

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38480: Assignee: Apache Spark > Remove spark.kubernetes.job.queue in favor of >

[jira] [Assigned] (SPARK-38480) Remove spark.kubernetes.job.queue in favor of spark.kubernetes.driver.podGroupTemplateFile

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38480: Assignee: (was: Apache Spark) > Remove spark.kubernetes.job.queue in favor of >

[jira] [Comment Edited] (SPARK-38388) Repartition + Stage retries could lead to incorrect data

2022-03-08 Thread Jason Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503105#comment-17503105 ] Jason Xu edited comment on SPARK-38388 at 3/9/22, 7:34 AM: --- [~cloud_fan]

[jira] [Resolved] (SPARK-36679) Remove lz4 hadoop wrapper classes after Hadoop 3.3.2

2022-03-08 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-36679. -- Fix Version/s: 3.3.0 Resolution: Duplicate > Remove lz4 hadoop wrapper classes after Hadoop

[jira] [Created] (SPARK-38480) Remove spark.kubernetes.job.queue in favor of spark.kubernetes.driver.podGroupTemplateFile

2022-03-08 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-38480: - Summary: Remove spark.kubernetes.job.queue in favor of spark.kubernetes.driver.podGroupTemplateFile Key: SPARK-38480 URL: https://issues.apache.org/jira/browse/SPARK-38480

[jira] [Assigned] (SPARK-38479) Add `Series.duplicated` to indicate duplicate Series values.

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38479: Assignee: Apache Spark > Add `Series.duplicated` to indicate duplicate Series values. >

[jira] [Commented] (SPARK-38479) Add `Series.duplicated` to indicate duplicate Series values.

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503326#comment-17503326 ] Apache Spark commented on SPARK-38479: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-38479) Add `Series.duplicated` to indicate duplicate Series values.

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38479: Assignee: (was: Apache Spark) > Add `Series.duplicated` to indicate duplicate Series

[jira] [Updated] (SPARK-38479) Add `Series.duplicated` to indicate duplicate Series values.

2022-03-08 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-38479: - Issue Type: New Feature (was: Improvement) > Add `Series.duplicated` to indicate duplicate

[jira] [Created] (SPARK-38479) Add `Series.duplicated` to indicate duplicate Series values.

2022-03-08 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-38479: Summary: Add `Series.duplicated` to indicate duplicate Series values. Key: SPARK-38479 URL: https://issues.apache.org/jira/browse/SPARK-38479 Project: Spark

[jira] [Resolved] (SPARK-38458) Fix always false condition in LogDivertAppender#initLayout

2022-03-08 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-38458. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35780

[jira] [Assigned] (SPARK-38458) Fix always false condition in LogDivertAppender#initLayout

2022-03-08 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-38458: --- Assignee: Yang Jie > Fix always false condition in LogDivertAppender#initLayout >

[jira] [Created] (SPARK-38477) Use error classes in org.apache.spark.storage

2022-03-08 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38477: Summary: Use error classes in org.apache.spark.storage Key: SPARK-38477 URL: https://issues.apache.org/jira/browse/SPARK-38477 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-38474) Use error classes in org.apache.spark.security

2022-03-08 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38474: Summary: Use error classes in org.apache.spark.security Key: SPARK-38474 URL: https://issues.apache.org/jira/browse/SPARK-38474 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-38475) Use error classes in org.apache.spark.serializer

2022-03-08 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38475: Summary: Use error classes in org.apache.spark.serializer Key: SPARK-38475 URL: https://issues.apache.org/jira/browse/SPARK-38475 Project: Spark Issue Type:

[jira] [Created] (SPARK-38476) Use error classes in org.apache.spark.shuffle

2022-03-08 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38476: Summary: Use error classes in org.apache.spark.shuffle Key: SPARK-38476 URL: https://issues.apache.org/jira/browse/SPARK-38476 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-38478) Use error classes in org.apache.spark.ui

2022-03-08 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38478: Summary: Use error classes in org.apache.spark.ui Key: SPARK-38478 URL: https://issues.apache.org/jira/browse/SPARK-38478 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-38473) Use error classes in org.apache.spark.scheduler

2022-03-08 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38473: Summary: Use error classes in org.apache.spark.scheduler Key: SPARK-38473 URL: https://issues.apache.org/jira/browse/SPARK-38473 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-38472) Use error classes in org.apache.spark.rpc

2022-03-08 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38472: Summary: Use error classes in org.apache.spark.rpc Key: SPARK-38472 URL: https://issues.apache.org/jira/browse/SPARK-38472 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-38471) Use error classes in org.apache.spark.rdd

2022-03-08 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38471: Summary: Use error classes in org.apache.spark.rdd Key: SPARK-38471 URL: https://issues.apache.org/jira/browse/SPARK-38471 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-38470) Use error classes in org.apache.spark.partial

2022-03-08 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38470: Summary: Use error classes in org.apache.spark.partial Key: SPARK-38470 URL: https://issues.apache.org/jira/browse/SPARK-38470 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-38467) Use error classes in org.apache.spark.memory

2022-03-08 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38467: Summary: Use error classes in org.apache.spark.memory Key: SPARK-38467 URL: https://issues.apache.org/jira/browse/SPARK-38467 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-38468) Use error classes in org.apache.spark.metrics

2022-03-08 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38468: Summary: Use error classes in org.apache.spark.metrics Key: SPARK-38468 URL: https://issues.apache.org/jira/browse/SPARK-38468 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-38469) Use error classes in org.apache.spark.network

2022-03-08 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38469: Summary: Use error classes in org.apache.spark.network Key: SPARK-38469 URL: https://issues.apache.org/jira/browse/SPARK-38469 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-38465) Use error classes in org.apache.spark.launcher

2022-03-08 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38465: Summary: Use error classes in org.apache.spark.launcher Key: SPARK-38465 URL: https://issues.apache.org/jira/browse/SPARK-38465 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-38466) Use error classes in org.apache.spark.mapred

2022-03-08 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38466: Summary: Use error classes in org.apache.spark.mapred Key: SPARK-38466 URL: https://issues.apache.org/jira/browse/SPARK-38466 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-38464) Use error classes in org.apache.spark.io

2022-03-08 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38464: Summary: Use error classes in org.apache.spark.io Key: SPARK-38464 URL: https://issues.apache.org/jira/browse/SPARK-38464 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-38463) Use error classes in org.apache.spark.input

2022-03-08 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38463: Summary: Use error classes in org.apache.spark.input Key: SPARK-38463 URL: https://issues.apache.org/jira/browse/SPARK-38463 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-38462) Use error classes in org.apache.spark.executor

2022-03-08 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38462: Summary: Use error classes in org.apache.spark.executor Key: SPARK-38462 URL: https://issues.apache.org/jira/browse/SPARK-38462 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-38452) Support pyDockerfile and rDockerfile in SBT K8s IT

2022-03-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38452: - Assignee: Yikun Jiang > Support pyDockerfile and rDockerfile in SBT K8s IT >

[jira] [Resolved] (SPARK-38452) Support pyDockerfile and rDockerfile in SBT K8s IT

2022-03-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38452. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35772

[jira] [Created] (SPARK-38461) Use error classes in org.apache.spark.broadcast

2022-03-08 Thread Bo Zhang (Jira)
Bo Zhang created SPARK-38461: Summary: Use error classes in org.apache.spark.broadcast Key: SPARK-38461 URL: https://issues.apache.org/jira/browse/SPARK-38461 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-38312) Use error classes in org.apache.spark.metrics

2022-03-08 Thread Bo Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Zhang updated SPARK-38312: - Summary: Use error classes in org.apache.spark.metrics (was: Use error classes in spark-core) > Use

[jira] [Resolved] (SPARK-38449) Not call createTable when ifNotExist=true and table eixsts

2022-03-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38449. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35770

[jira] [Commented] (SPARK-38460) Clean up duplicate codes in get method of ColumnarBatchRow/ColumnarRow/MutableColumnarRow

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503303#comment-17503303 ] Apache Spark commented on SPARK-38460: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-38460) Clean up duplicate codes in get method of ColumnarBatchRow/ColumnarRow/MutableColumnarRow

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38460: Assignee: (was: Apache Spark) > Clean up duplicate codes in get method of >

[jira] [Commented] (SPARK-38460) Clean up duplicate codes in get method of ColumnarBatchRow/ColumnarRow/MutableColumnarRow

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503302#comment-17503302 ] Apache Spark commented on SPARK-38460: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-38460) Clean up duplicate codes in get method of ColumnarBatchRow/ColumnarRow/MutableColumnarRow

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38460: Assignee: Apache Spark > Clean up duplicate codes in get method of >

[jira] [Created] (SPARK-38460) Clean up duplicate codes in get method of ColumnarBatchRow/ColumnarRow/MutableColumnarRow

2022-03-08 Thread Yang Jie (Jira)
Yang Jie created SPARK-38460: Summary: Clean up duplicate codes in get method of ColumnarBatchRow/ColumnarRow/MutableColumnarRow Key: SPARK-38460 URL: https://issues.apache.org/jira/browse/SPARK-38460

[jira] [Created] (SPARK-38459) IsolatedClientLoader use build in hadoop version

2022-03-08 Thread angerszhu (Jira)
angerszhu created SPARK-38459: - Summary: IsolatedClientLoader use build in hadoop version Key: SPARK-38459 URL: https://issues.apache.org/jira/browse/SPARK-38459 Project: Spark Issue Type: Task

[jira] [Assigned] (SPARK-38187) Support resource reservation (Introduce minCPU/minMemory) with volcano implementations

2022-03-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38187: - Assignee: Dongjoon Hyun > Support resource reservation (Introduce minCPU/minMemory)

[jira] [Resolved] (SPARK-38187) Support resource reservation (Introduce minCPU/minMemory) with volcano implementations

2022-03-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38187. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35776

[jira] [Resolved] (SPARK-38455) Support driver/executor PodGroup templates

2022-03-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-38455. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35776

[jira] [Assigned] (SPARK-38455) Support driver/executor PodGroup templates

2022-03-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-38455: - Assignee: Dongjoon Hyun > Support driver/executor PodGroup templates >

[jira] [Assigned] (SPARK-37600) Upgrade to Hadoop 3.3.2

2022-03-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37600: - Assignee: Chao Sun > Upgrade to Hadoop 3.3.2 > --- > >

[jira] [Resolved] (SPARK-37600) Upgrade to Hadoop 3.3.2

2022-03-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37600. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34855

[jira] [Resolved] (SPARK-38407) ANSI Cast: loosen the limitation of casting non-null complex types

2022-03-08 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-38407. Fix Version/s: 3.3.0 3.2.2 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-38458) Fix always false condition in LogDivertAppender#initLayout

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38458: Assignee: (was: Apache Spark) > Fix always false condition in

[jira] [Assigned] (SPARK-38458) Fix always false condition in LogDivertAppender#initLayout

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38458: Assignee: Apache Spark > Fix always false condition in LogDivertAppender#initLayout >

[jira] [Commented] (SPARK-38458) Fix always false condition in LogDivertAppender#initLayout

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503283#comment-17503283 ] Apache Spark commented on SPARK-38458: -- User 'LuciferYang' has created a pull request for this

[jira] [Updated] (SPARK-38458) Fix always false condition in LogDivertAppender#initLayout

2022-03-08 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-38458: - Summary: Fix always false condition in LogDivertAppender#initLayout (was: Fix always false condition

[jira] [Commented] (SPARK-36194) Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503270#comment-17503270 ] Apache Spark commented on SPARK-36194: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-36194) Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503269#comment-17503269 ] Apache Spark commented on SPARK-36194: -- User 'wangyum' has created a pull request for this issue:

[jira] [Created] (SPARK-38458) Fix always false condition in LogDivertAppender#initLayout method

2022-03-08 Thread Yang Jie (Jira)
Yang Jie created SPARK-38458: Summary: Fix always false condition in LogDivertAppender#initLayout method Key: SPARK-38458 URL: https://issues.apache.org/jira/browse/SPARK-38458 Project: Spark

[jira] [Updated] (SPARK-38457) spark.sql.catalyst.analysis

2022-03-08 Thread liujintao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liujintao updated SPARK-38457: -- Description: create view step_29134910810  as select * from step_29134878417  limit 20(String),

[jira] [Resolved] (SPARK-38412) `from` and `to` is swapped in the StateSchemaCompatibilityChecker

2022-03-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-38412. -- Fix Version/s: 3.3.0 3.1.4 3.2.2 Assignee:

[jira] [Updated] (SPARK-38457) spark.sql.catalyst.analysis

2022-03-08 Thread liujintao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liujintao updated SPARK-38457: -- Description: select * from step_29134878417  limit 20 java.lang.RuntimeException:

[jira] [Created] (SPARK-38457) spark.sql.catalyst.analysis

2022-03-08 Thread liujintao (Jira)
liujintao created SPARK-38457: - Summary: spark.sql.catalyst.analysis Key: SPARK-38457 URL: https://issues.apache.org/jira/browse/SPARK-38457 Project: Spark Issue Type: Bug Components:

[jira] [Resolved] (SPARK-38450) Fix HiveQuerySuite//PushFoldableIntoBranchesSuite/TransposeWindowSuite

2022-03-08 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-38450. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35771

[jira] [Assigned] (SPARK-37865) Spark should not dedup the groupingExpressions when the first child of Union has duplicate columns

2022-03-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37865: --- Assignee: Karen Feng > Spark should not dedup the groupingExpressions when the first child

[jira] [Resolved] (SPARK-37865) Spark should not dedup the groupingExpressions when the first child of Union has duplicate columns

2022-03-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37865. - Fix Version/s: 3.3.0 3.0.4 3.2.2 3.1.3

[jira] [Resolved] (SPARK-38443) Document config STREAMING_SESSION_WINDOW_MERGE_SESSIONS_IN_LOCAL_PARTITION

2022-03-08 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-38443. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35762

[jira] [Assigned] (SPARK-38443) Document config STREAMING_SESSION_WINDOW_MERGE_SESSIONS_IN_LOCAL_PARTITION

2022-03-08 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-38443: --- Assignee: L. C. Hsieh > Document config

[jira] [Commented] (SPARK-38385) Improve error messages of 'mismatched input' cases from ANTLR

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503230#comment-17503230 ] Apache Spark commented on SPARK-38385: -- User 'anchovYu' has created a pull request for this issue:

[jira] [Commented] (SPARK-38379) Kubernetes: NoSuchElementException: spark.app.id when using PersistentVolumes

2022-03-08 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503229#comment-17503229 ] Thomas Graves commented on SPARK-38379: --- so the issue here is there is a race between when

[jira] [Commented] (SPARK-34960) Aggregate (Min/Max/Count) push down for ORC

2022-03-08 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503221#comment-17503221 ] Thomas Graves commented on SPARK-34960: --- if I'm reading the orc spec right the ColumnStatistics

[jira] [Commented] (SPARK-38385) Improve error messages of 'mismatched input' cases from ANTLR

2022-03-08 Thread Xinyi Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503219#comment-17503219 ] Xinyi Yu commented on SPARK-38385: -- Hi [~cloud_fan] , actually the PR only resolve the case 1 in this

[jira] [Updated] (SPARK-38456) Improve error messages of empty statement

2022-03-08 Thread Xinyi Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinyi Yu updated SPARK-38456: - Description: Please view the parent task description for the general idea: 

[jira] [Created] (SPARK-38456) Improve error messages of empty statement

2022-03-08 Thread Xinyi Yu (Jira)
Xinyi Yu created SPARK-38456: Summary: Improve error messages of empty statement Key: SPARK-38456 URL: https://issues.apache.org/jira/browse/SPARK-38456 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-36645) Aggregate (Min/Max/Count) push down for Parquet

2022-03-08 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-36645: -- Summary: Aggregate (Min/Max/Count) push down for Parquet (was: Aggregate (Count) push down

[jira] [Comment Edited] (SPARK-36645) Aggregate (Min/Max/Count) push down for Parquet

2022-03-08 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503212#comment-17503212 ] Thomas Graves edited comment on SPARK-36645 at 3/8/22, 10:52 PM: - Note

[jira] [Updated] (SPARK-36645) Aggregate (Count) push down for Parquet

2022-03-08 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-36645: -- Summary: Aggregate (Count) push down for Parquet (was: Aggregate (Min/Max/Count) push down

[jira] [Commented] (SPARK-36645) Aggregate (Min/Max/Count) push down for Parquet

2022-03-08 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503212#comment-17503212 ] Thomas Graves commented on SPARK-36645: --- Note it appears this only really pushes down count

[jira] [Commented] (SPARK-38187) Support resource reservation (Introduce minCPU/minMemory) with volcano implementations

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503201#comment-17503201 ] Apache Spark commented on SPARK-38187: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-38187) Support resource reservation (Introduce minCPU/minMemory) with volcano implementations

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503200#comment-17503200 ] Apache Spark commented on SPARK-38187: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-38455) Support driver/executor PodGroup templates

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503199#comment-17503199 ] Apache Spark commented on SPARK-38455: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-38455) Support driver/executor PodGroup templates

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503198#comment-17503198 ] Apache Spark commented on SPARK-38455: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-38455) Support driver/executor PodGroup templates

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38455: Assignee: (was: Apache Spark) > Support driver/executor PodGroup templates >

[jira] [Assigned] (SPARK-38455) Support driver/executor PodGroup templates

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38455: Assignee: Apache Spark > Support driver/executor PodGroup templates >

[jira] [Created] (SPARK-38455) Support driver/executor PodGroup templates

2022-03-08 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-38455: - Summary: Support driver/executor PodGroup templates Key: SPARK-38455 URL: https://issues.apache.org/jira/browse/SPARK-38455 Project: Spark Issue Type:

[jira] [Commented] (SPARK-38355) Change mktemp() to mkstemp()

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503191#comment-17503191 ] Apache Spark commented on SPARK-38355: -- User 'bjornjorgensen' has created a pull request for this

[jira] [Commented] (SPARK-38355) Change mktemp() to mkstemp()

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503190#comment-17503190 ] Apache Spark commented on SPARK-38355: -- User 'bjornjorgensen' has created a pull request for this

[jira] [Assigned] (SPARK-38355) Change mktemp() to mkstemp()

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38355: Assignee: Apache Spark > Change mktemp() to mkstemp() > > >

[jira] [Assigned] (SPARK-38355) Change mktemp() to mkstemp()

2022-03-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38355: Assignee: (was: Apache Spark) > Change mktemp() to mkstemp() >

[jira] [Updated] (SPARK-38438) Can't update spark.jars.packages on existing global/default context

2022-03-08 Thread Rafal Wojdyla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rafal Wojdyla updated SPARK-38438: -- Description: Reproduction: {code:python} from pyspark.sql import SparkSession # default

[jira] [Resolved] (SPARK-38428) Check the FetchShuffleBlocks message only once to improve iteration in external shuffle service

2022-03-08 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-38428. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Created] (SPARK-38454) Partition Data Type Prevents Filtering Sporadically

2022-03-08 Thread Christopher (Jira)
Christopher created SPARK-38454: --- Summary: Partition Data Type Prevents Filtering Sporadically Key: SPARK-38454 URL: https://issues.apache.org/jira/browse/SPARK-38454 Project: Spark Issue

[jira] [Commented] (SPARK-30788) Support `SimpleDateFormat` and `FastDateFormat` as legacy date/timestamp formatters

2022-03-08 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503127#comment-17503127 ] Max Gekk commented on SPARK-30788: -- It was merged to 3.0.0

[jira] [Comment Edited] (SPARK-30788) Support `SimpleDateFormat` and `FastDateFormat` as legacy date/timestamp formatters

2022-03-08 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503127#comment-17503127 ] Max Gekk edited comment on SPARK-30788 at 3/8/22, 6:55 PM: --- It was merged to

[jira] [Commented] (SPARK-30788) Support `SimpleDateFormat` and `FastDateFormat` as legacy date/timestamp formatters

2022-03-08 Thread Andy Grove (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503123#comment-17503123 ] Andy Grove commented on SPARK-30788: [~cloud_fan] [~maxgekk] I think the fix version on this issue

[jira] [Resolved] (SPARK-38309) SHS has incorrect percentiles for shuffle read bytes and shuffle total blocks metrics

2022-03-08 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-38309. - Fix Version/s: 3.3.0 3.2.2 3.1.3

[jira] [Assigned] (SPARK-38309) SHS has incorrect percentiles for shuffle read bytes and shuffle total blocks metrics

2022-03-08 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-38309: --- Assignee: Rob Reeves > SHS has incorrect percentiles for shuffle read

[jira] [Commented] (SPARK-38388) Repartition + Stage retries could lead to incorrect data

2022-03-08 Thread Jason Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503105#comment-17503105 ] Jason Xu commented on SPARK-38388: -- [~cloud_fan] [~mridul] have lots of insight on this issue in

[jira] [Updated] (SPARK-38388) Repartition + Stage retries could lead to incorrect data

2022-03-08 Thread Jason Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Xu updated SPARK-38388: - Description: Spark repartition uses RoundRobinPartitioning, the generated results is non-deterministic

[jira] [Commented] (SPARK-38388) Repartition + Stage retries could lead to incorrect data

2022-03-08 Thread Jason Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17503100#comment-17503100 ] Jason Xu commented on SPARK-38388: --  [~jiangxb1987] using `rand()` in the reproduce example is to

[jira] [Resolved] (SPARK-37193) DynamicJoinSelection.shouldDemoteBroadcastHashJoin should not apply to outer joins

2022-03-08 Thread Eugene Koifman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman resolved SPARK-37193. Fix Version/s: 3.3.0 Resolution: Abandoned this was addressed in SPARK-37753 >

  1   2   >