[jira] [Created] (SPARK-53415) Simply options for builtin FileFormats

2025-08-27 Thread Kent Yao (Jira)
Kent Yao created SPARK-53415: Summary: Simply options for builtin FileFormats Key: SPARK-53415 URL: https://issues.apache.org/jira/browse/SPARK-53415 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-53414) Add tests for Arrow UDF with profiler

2025-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53414: --- Labels: pull-request-available (was: ) > Add tests for Arrow UDF with profiler > --

[jira] [Updated] (SPARK-53412) Upgrade Volcano to 1.12.2

2025-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53412: --- Labels: pull-request-available (was: ) > Upgrade Volcano to 1.12.2 > --

[jira] [Created] (SPARK-53412) Upgrade Volcano to 1.12.2

2025-08-27 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-53412: - Summary: Upgrade Volcano to 1.12.2 Key: SPARK-53412 URL: https://issues.apache.org/jira/browse/SPARK-53412 Project: Spark Issue Type: Sub-task Co

[jira] [Updated] (SPARK-53413) Shuffle cleanup doesnot happen for commands

2025-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53413: --- Labels: pull-request-available (was: ) > Shuffle cleanup doesnot happen for commands >

[jira] [Created] (SPARK-53414) Add tests for Arrow UDF with profiler

2025-08-27 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-53414: - Summary: Add tests for Arrow UDF with profiler Key: SPARK-53414 URL: https://issues.apache.org/jira/browse/SPARK-53414 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-53413) Shuffle cleanup doesnot happen for commands

2025-08-27 Thread Karuppayya (Jira)
Karuppayya created SPARK-53413: -- Summary: Shuffle cleanup doesnot happen for commands Key: SPARK-53413 URL: https://issues.apache.org/jira/browse/SPARK-53413 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-53411) Upgrade the minimum K8s version to v1.32

2025-08-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-53411. --- Fix Version/s: kubernetes-operator-0.5.0 Resolution: Fixed Issue resolved by pull req

[jira] [Assigned] (SPARK-51168) Upgrade Hadoop to 3.4.2

2025-08-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-51168: - Assignee: Cheng Pan > Upgrade Hadoop to 3.4.2 > > >

[jira] [Updated] (SPARK-53411) Upgrade the minimum K8s version to v1.32

2025-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53411: --- Labels: pull-request-available (was: ) > Upgrade the minimum K8s version to v1.32 > ---

[jira] [Resolved] (SPARK-53410) Improve `merge_spark_pr.py` to accept PR numbers as a CLI argument

2025-08-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-53410. --- Fix Version/s: kubernetes-operator-0.5.0 Resolution: Fixed Issue resolved by pull req

[jira] [Assigned] (SPARK-53411) Upgrade the minimum K8s version to v1.32

2025-08-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53411: - Assignee: Dongjoon Hyun > Upgrade the minimum K8s version to v1.32 > --

[jira] [Resolved] (SPARK-53409) Upgrade `Gradle` to 9.0.0

2025-08-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-53409. --- Fix Version/s: kubernetes-operator-0.5.0 Resolution: Fixed Issue resolved by pull req

[jira] [Created] (SPARK-53411) Upgrade the minimum K8s version to v1.32

2025-08-27 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-53411: - Summary: Upgrade the minimum K8s version to v1.32 Key: SPARK-53411 URL: https://issues.apache.org/jira/browse/SPARK-53411 Project: Spark Issue Type: Sub-ta

[jira] [Assigned] (SPARK-53410) Improve `merge_spark_pr.py` to accept PR numbers as a CLI argument

2025-08-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53410: - Assignee: Dongjoon Hyun > Improve `merge_spark_pr.py` to accept PR numbers as a CLI arg

[jira] [Created] (SPARK-53410) Improve `merge_spark_pr.py` to accept PR numbers as a CLI argument

2025-08-27 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-53410: - Summary: Improve `merge_spark_pr.py` to accept PR numbers as a CLI argument Key: SPARK-53410 URL: https://issues.apache.org/jira/browse/SPARK-53410 Project: Spark

[jira] [Updated] (SPARK-53410) Improve `merge_spark_pr.py` to accept PR numbers as a CLI argument

2025-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53410: --- Labels: pull-request-available (was: ) > Improve `merge_spark_pr.py` to accept PR numbers a

[jira] [Assigned] (SPARK-53409) Upgrade `Gradle` to 9.0.0

2025-08-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53409: - Assignee: Dongjoon Hyun > Upgrade `Gradle` to 9.0.0 > - > >

[jira] [Updated] (SPARK-53409) Upgrade `Gradle` to 9.0.0

2025-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53409: --- Labels: pull-request-available (was: ) > Upgrade `Gradle` to 9.0.0 > --

[jira] [Created] (SPARK-53409) Upgrade `Gradle` to 9.0.0

2025-08-27 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-53409: - Summary: Upgrade `Gradle` to 9.0.0 Key: SPARK-53409 URL: https://issues.apache.org/jira/browse/SPARK-53409 Project: Spark Issue Type: Sub-task Co

[jira] [Resolved] (SPARK-53407) Use `baseDirectory` instead of `baseDir`

2025-08-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-53407. --- Fix Version/s: kubernetes-operator-0.5.0 Resolution: Fixed Issue resolved by pull req

[jira] [Updated] (SPARK-53408) Clean up unused functions from `QueryCompilationErrors`

2025-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53408: --- Labels: pull-request-available (was: ) > Clean up unused functions from `QueryCompilationEr

[jira] [Assigned] (SPARK-53407) Use `baseDirectory` instead of `baseDir`

2025-08-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53407: - Assignee: Dongjoon Hyun > Use `baseDirectory` instead of `baseDir` > --

[jira] [Updated] (SPARK-53408) Clean up unused functions from `QueryCompilationErrors`

2025-08-27 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-53408: - Summary: Clean up unused functions from `QueryCompilationErrors` (was: Clean up unused function from `Q

[jira] [Created] (SPARK-53408) Clean up unused function from `QueryCompilationErrors`

2025-08-27 Thread Yang Jie (Jira)
Yang Jie created SPARK-53408: Summary: Clean up unused function from `QueryCompilationErrors` Key: SPARK-53408 URL: https://issues.apache.org/jira/browse/SPARK-53408 Project: Spark Issue Type: Im

[jira] [Created] (SPARK-53407) Use `baseDirectory` instead of `baseDir`

2025-08-27 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-53407: - Summary: Use `baseDirectory` instead of `baseDir` Key: SPARK-53407 URL: https://issues.apache.org/jira/browse/SPARK-53407 Project: Spark Issue Type: Sub-ta

[jira] [Updated] (SPARK-53407) Use `baseDirectory` instead of `baseDir`

2025-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53407: --- Labels: pull-request-available (was: ) > Use `baseDirectory` instead of `baseDir` > ---

[jira] [Resolved] (SPARK-53397) Fix UDTF test with collations

2025-08-27 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-53397. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 52001 [https://

[jira] [Assigned] (SPARK-53397) Fix UDTF test with collations

2025-08-27 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-53397: - Assignee: Marko Ilic > Fix UDTF test with collations > - >

[jira] [Assigned] (SPARK-53245) Update Java Operator SDK version to 5.1.2

2025-08-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53245: - Assignee: Attila Mészáros > Update Java Operator SDK version to 5.1.2 > ---

[jira] [Resolved] (SPARK-53245) Update Java Operator SDK version to 5.1.2

2025-08-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-53245. --- Fix Version/s: kubernetes-operator-0.5.0 Resolution: Fixed Issue resolved by pull req

[jira] [Updated] (SPARK-53245) Update Java Operator SDK version to 5.1.2

2025-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53245: --- Labels: pull-request-available (was: ) > Update Java Operator SDK version to 5.1.2 > --

[jira] [Updated] (SPARK-53404) Make `Build` step to verify test compilation too

2025-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53404: --- Labels: pull-request-available (was: ) > Make `Build` step to verify test compilation too >

[jira] [Resolved] (SPARK-53404) Make `Build` step to verify test compilation too

2025-08-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-53404. --- Fix Version/s: connect-swift-0.4.0 Resolution: Fixed Issue resolved by pull request 2

[jira] [Assigned] (SPARK-53404) Make `Build` step to verify test compilation too

2025-08-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53404: - Assignee: Dongjoon Hyun > Make `Build` step to verify test compilation too > --

[jira] [Updated] (SPARK-53406) Support Shuffle Spec in Direct Partition ID Pass Through

2025-08-27 Thread Shujing Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shujing Yang updated SPARK-53406: - Parent: SPARK-53400 Issue Type: Sub-task (was: New Feature) > Support Shuffle Spec in D

[jira] [Updated] (SPARK-53401) Enable Direct Passthrough Partitioning in the DataFrame API

2025-08-27 Thread Shujing Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shujing Yang updated SPARK-53401: - Parent: SPARK-53400 Issue Type: Sub-task (was: New Feature) > Enable Direct Passthrough

[jira] [Updated] (SPARK-53402) Support Direct Passthrough Partitioning in Spark Connect

2025-08-27 Thread Shujing Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shujing Yang updated SPARK-53402: - Parent: SPARK-53400 Issue Type: Sub-task (was: New Feature) > Support Direct Passthroug

[jira] [Updated] (SPARK-53401) Enable Direct Passthrough Partitioning in the DataFrame API

2025-08-27 Thread Shujing Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shujing Yang updated SPARK-53401: - Epic Link: (was: SPARK-53400) > Enable Direct Passthrough Partitioning in the DataFrame API

[jira] [Updated] (SPARK-53402) Support Direct Passthrough Partitioning in Spark Connect

2025-08-27 Thread Shujing Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shujing Yang updated SPARK-53402: - Epic Link: (was: SPARK-53400) > Support Direct Passthrough Partitioning in Spark Connect > -

[jira] [Updated] (SPARK-53400) Support Direct Passthrough Partitioning in the DataFrame API

2025-08-27 Thread Shujing Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shujing Yang updated SPARK-53400: - Issue Type: New Feature (was: Epic) > Support Direct Passthrough Partitioning in the DataFrame

[jira] [Updated] (SPARK-53406) Support Shuffle Spec in Direct Partition ID Pass Through

2025-08-27 Thread Shujing Yang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shujing Yang updated SPARK-53406: - Epic Link: (was: SPARK-53400) > Support Shuffle Spec in Direct Partition ID Pass Through > -

[jira] [Updated] (SPARK-53401) Enable Direct Passthrough Partitioning in the DataFrame API

2025-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53401: --- Labels: pull-request-available (was: ) > Enable Direct Passthrough Partitioning in the Data

[jira] [Created] (SPARK-53406) Support Shuffle Spec in Direct Partition ID Pass Through

2025-08-27 Thread Shujing Yang (Jira)
Shujing Yang created SPARK-53406: Summary: Support Shuffle Spec in Direct Partition ID Pass Through Key: SPARK-53406 URL: https://issues.apache.org/jira/browse/SPARK-53406 Project: Spark Issu

[jira] [Updated] (SPARK-53405) Add metrics recording for latency of Spark app state transition

2025-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53405: --- Labels: pull-request-available (was: ) > Add metrics recording for latency of Spark app sta

[jira] [Created] (SPARK-53405) Add metrics recording for latency of Spark app state transition

2025-08-27 Thread Zhou JIANG (Jira)
Zhou JIANG created SPARK-53405: -- Summary: Add metrics recording for latency of Spark app state transition Key: SPARK-53405 URL: https://issues.apache.org/jira/browse/SPARK-53405 Project: Spark

[jira] [Created] (SPARK-53404) Make `Build` step to verify test compilation too

2025-08-27 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-53404: - Summary: Make `Build` step to verify test compilation too Key: SPARK-53404 URL: https://issues.apache.org/jira/browse/SPARK-53404 Project: Spark Issue Type

[jira] [Created] (SPARK-53403) Improve add/sub tests under ANSI

2025-08-27 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-53403: Summary: Improve add/sub tests under ANSI Key: SPARK-53403 URL: https://issues.apache.org/jira/browse/SPARK-53403 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-53402) Support Direct Passthrough Partitioning in Spark Connect

2025-08-27 Thread Shujing Yang (Jira)
Shujing Yang created SPARK-53402: Summary: Support Direct Passthrough Partitioning in Spark Connect Key: SPARK-53402 URL: https://issues.apache.org/jira/browse/SPARK-53402 Project: Spark Issu

[jira] [Created] (SPARK-53401) Enable Direct Passthrough Partitioning in the DataFrame API

2025-08-27 Thread Shujing Yang (Jira)
Shujing Yang created SPARK-53401: Summary: Enable Direct Passthrough Partitioning in the DataFrame API Key: SPARK-53401 URL: https://issues.apache.org/jira/browse/SPARK-53401 Project: Spark

[jira] [Created] (SPARK-53400) Support Direct Passthrough Partitioning in the DataFrame API

2025-08-27 Thread Shujing Yang (Jira)
Shujing Yang created SPARK-53400: Summary: Support Direct Passthrough Partitioning in the DataFrame API Key: SPARK-53400 URL: https://issues.apache.org/jira/browse/SPARK-53400 Project: Spark

[jira] [Updated] (SPARK-53348) Always persist ANSI value when creating a view or assume it when querying if not stored

2025-08-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-53348: -- Fix Version/s: 4.0.1 > Always persist ANSI value when creating a view or assume it when queryi

[jira] [Resolved] (SPARK-53390) Raise error when bools with None `astype` to ints under ANSI

2025-08-27 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-53390. -- Assignee: Xinrong Meng Resolution: Resolved Resolved by https://github.com/apache/spark/

[jira] [Updated] (SPARK-53389) Improvements for Pandas API on Spark under ANSI

2025-08-27 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-53389: - Description: Improve test coverage, and possible improvements exposed by the newly added tests.

[jira] [Updated] (SPARK-53390) Raise error when bools with None `astype` to ints under ANSI

2025-08-27 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-53390: - Summary: Raise error when bools with None `astype` to ints under ANSI (was: Improve tests of `a

[jira] [Updated] (SPARK-53389) Improvements for Pandas API on Spark under ANSI

2025-08-27 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-53389: - Summary: Improvements for Pandas API on Spark under ANSI (was: Improve ANSI test coverage for P

[jira] [Updated] (SPARK-53399) Improve CollapseProject

2025-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53399: --- Labels: pull-request-available (was: ) > Improve CollapseProject > ---

[jira] [Updated] (SPARK-53333) Enable StateDataSource with state checkpoint v2 (only readChangeFeed)

2025-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-5?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-5: --- Labels: pull-request-available (was: ) > Enable StateDataSource with state checkpoint v2 (o

[jira] [Created] (SPARK-53399) Improve CollapseProject

2025-08-27 Thread Peter Toth (Jira)
Peter Toth created SPARK-53399: -- Summary: Improve CollapseProject Key: SPARK-53399 URL: https://issues.apache.org/jira/browse/SPARK-53399 Project: Spark Issue Type: Bug Components: SQL

[jira] [Commented] (SPARK-53294) Enable StateDataSource with state checkpoint v2 (only batchId option)

2025-08-27 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18016598#comment-18016598 ] Anish Shrigondekar commented on SPARK-53294: PR merged here: https://github.

[jira] [Assigned] (SPARK-53294) Enable StateDataSource with state checkpoint v2 (only batchId option)

2025-08-27 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar reassigned SPARK-53294: -- Assignee: Dylan Wong > Enable StateDataSource with state checkpoint v2 (only

[jira] [Resolved] (SPARK-53294) Enable StateDataSource with state checkpoint v2 (only batchId option)

2025-08-27 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar resolved SPARK-53294. Fix Version/s: 4.1.0 Resolution: Fixed > Enable StateDataSource with state

[jira] [Closed] (SPARK-53294) Enable StateDataSource with state checkpoint v2 (only batchId option)

2025-08-27 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar closed SPARK-53294. -- > Enable StateDataSource with state checkpoint v2 (only batchId option) >

[jira] [Updated] (SPARK-53398) Ensure that RocksDBMemoryManager metrics reporting doesn't throw a NullPointerException

2025-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53398: --- Labels: pull-request-available (was: ) > Ensure that RocksDBMemoryManager metrics reporting

[jira] [Created] (SPARK-53398) Ensure that RocksDBMemoryManager metrics reporting doesn't throw a NullPointerException

2025-08-27 Thread Eric Marnadi (Jira)
Eric Marnadi created SPARK-53398: Summary: Ensure that RocksDBMemoryManager metrics reporting doesn't throw a NullPointerException Key: SPARK-53398 URL: https://issues.apache.org/jira/browse/SPARK-53398

[jira] [Resolved] (SPARK-52777) Enable shuffle cleanup mode configuration in Spark SQL

2025-08-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-52777. - Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51458 [https://gith

[jira] [Assigned] (SPARK-52777) Enable shuffle cleanup mode configuration in Spark SQL

2025-08-27 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-52777: --- Assignee: Karuppayya > Enable shuffle cleanup mode configuration in Spark SQL > ---

[jira] [Resolved] (SPARK-53392) Move SpecializedArray handling to connect-common

2025-08-27 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-53392. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 52137 [https://

[jira] [Updated] (SPARK-53397) Fix UDTF test with collations

2025-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53397: --- Labels: pull-request-available (was: ) > Fix UDTF test with collations > --

[jira] [Created] (SPARK-53397) Fix UDTF test with collations

2025-08-27 Thread Marko Ilic (Jira)
Marko Ilic created SPARK-53397: -- Summary: Fix UDTF test with collations Key: SPARK-53397 URL: https://issues.apache.org/jira/browse/SPARK-53397 Project: Spark Issue Type: Improvement C

[jira] [Assigned] (SPARK-53392) Move SpecializedArray handling to connect-common

2025-08-27 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-53392: - Assignee: Ruifeng Zheng > Move SpecializedArray handling to connect-common > --

[jira] [Resolved] (SPARK-53395) Add tests for combinations of different scalar UDFs

2025-08-27 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-53395. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 52141 [https://

[jira] [Resolved] (SPARK-53391) Remove unused PrimitiveKeyOpenHashMap

2025-08-27 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-53391. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 52136 [https://github.com

[jira] [Assigned] (SPARK-53391) Remove unused PrimitiveKeyOpenHashMap

2025-08-27 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-53391: Assignee: Kent Yao > Remove unused PrimitiveKeyOpenHashMap >

[jira] [Assigned] (SPARK-53395) Add tests for combinations of different scalar UDFs

2025-08-27 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-53395: - Assignee: Ruifeng Zheng > Add tests for combinations of different scalar UDFs > ---

[jira] [Resolved] (SPARK-53393) Disable memory profiler for Arrow Scalar Iterator UDFs

2025-08-27 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-53393. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 52138 [https://

[jira] [Assigned] (SPARK-53393) Disable memory profiler for Arrow Scalar Iterator UDFs

2025-08-27 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-53393: - Assignee: Ruifeng Zheng > Disable memory profiler for Arrow Scalar Iterator UDFs >

[jira] [Assigned] (SPARK-52858) Retry SBT compilation when OOM

2025-08-27 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-52858: - Assignee: Ruifeng Zheng > Retry SBT compilation when OOM >

[jira] [Resolved] (SPARK-52858) Retry SBT compilation when OOM

2025-08-27 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-52858. --- Resolution: Not A Problem > Retry SBT compilation when OOM > --

[jira] [Updated] (SPARK-53396) Improve error message when input of `GetStructField` is not `StructType`

2025-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53396: --- Labels: pull-request-available (was: ) > Improve error message when input of `GetStructFiel

[jira] [Created] (SPARK-53396) Improve error message when input of `GetStructField` is not `StructType`

2025-08-27 Thread Mihailo Timotic (Jira)
Mihailo Timotic created SPARK-53396: --- Summary: Improve error message when input of `GetStructField` is not `StructType` Key: SPARK-53396 URL: https://issues.apache.org/jira/browse/SPARK-53396 Projec

[jira] [Updated] (SPARK-53395) Add tests for combinations of different scalar UDFs

2025-08-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53395: --- Labels: pull-request-available (was: ) > Add tests for combinations of different scalar UDF

[jira] [Created] (SPARK-53395) Add tests for combinations of different scalar UDFs

2025-08-27 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-53395: - Summary: Add tests for combinations of different scalar UDFs Key: SPARK-53395 URL: https://issues.apache.org/jira/browse/SPARK-53395 Project: Spark Issue T