[jira] [Updated] (SPARK-53523) Named parameters respect `spark.sql.caseSensitive`

2025-09-11 Thread Cheng Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-53523: -- Summary: Named parameters respect `spark.sql.caseSensitive` (was: NamedParameter respects `spark.sql.

[jira] [Updated] (SPARK-53522) simplify PipelineTest

2025-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-53522: -- Parent: SPARK-51727 Issue Type: Sub-task (was: Test) > simplify PipelineTest > --

[jira] [Assigned] (SPARK-53540) Remove `pypy3` from `spark-rm` image

2025-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53540: - Assignee: Dongjoon Hyun > Remove `pypy3` from `spark-rm` image > --

[jira] [Created] (SPARK-53565) return wrong result for some aggregate functions when groupSets contains '{}'

2025-09-11 Thread weihua zhang (Jira)
weihua zhang created SPARK-53565: Summary: return wrong result for some aggregate functions when groupSets contains '{}' Key: SPARK-53565 URL: https://issues.apache.org/jira/browse/SPARK-53565 Project

[jira] [Updated] (SPARK-53550) Union output partitioning should compare canonicalized attributes

2025-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53550: --- Labels: pull-request-available (was: ) > Union output partitioning should compare canonical

[jira] [Created] (SPARK-53564) DAGScheduler exits due to RPC timeout to blockmanager in DAGScheduler loop

2025-09-11 Thread Tengfei Huang (Jira)
Tengfei Huang created SPARK-53564: - Summary: DAGScheduler exits due to RPC timeout to blockmanager in DAGScheduler loop Key: SPARK-53564 URL: https://issues.apache.org/jira/browse/SPARK-53564 Project:

[jira] [Created] (SPARK-53563) PS: SQLProcessor.execute() builds a string inefficiently by concatenation

2025-09-11 Thread Peter Nguyen (Jira)
Peter Nguyen created SPARK-53563: Summary: PS: SQLProcessor.execute() builds a string inefficiently by concatenation Key: SPARK-53563 URL: https://issues.apache.org/jira/browse/SPARK-53563 Project: Sp

[jira] [Updated] (SPARK-53563) PS: SQLProcessor.execute() builds a string inefficiently by concatenation

2025-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53563: --- Labels: pull-request-available (was: ) > PS: SQLProcessor.execute() builds a string ineffic

[jira] [Assigned] (SPARK-53491) SS | `inputRowsPerSecond` and `processedRowsPerSecond` in progress report show values in exponential notation for large numbers

2025-09-11 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-53491: Assignee: Jayant Sharma > SS | `inputRowsPerSecond` and `processedRowsPerSecond` in progr

[jira] [Resolved] (SPARK-53491) SS | `inputRowsPerSecond` and `processedRowsPerSecond` in progress report show values in exponential notation for large numbers

2025-09-11 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-53491. -- Fix Version/s: 4.1.0 (was: 4.0.0) Resolution: Fixed Issue resolv

[jira] [Updated] (SPARK-53557) Reduce automated vote email deadline from 4 days to 73 hours

2025-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53557: --- Labels: pull-request-available (was: ) > Reduce automated vote email deadline from 4 days t

[jira] [Updated] (SPARK-50888) Fix Flaky Test: `SparkConnectServiceSuite.SPARK-44776: LocalTableScanExe`

2025-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50888: -- Fix Version/s: 4.0.2 > Fix Flaky Test: `SparkConnectServiceSuite.SPARK-44776: LocalTableScanEx

[jira] [Updated] (SPARK-53527) Improve fallback of analyzeExistenceDefaultValue

2025-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53527: --- Labels: pull-request-available (was: ) > Improve fallback of analyzeExistenceDefaultValue >

[jira] [Resolved] (SPARK-53550) Union output partitioning should compare canonicalized attributes

2025-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-53550. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 52305 [https://

[jira] [Assigned] (SPARK-53522) simplify PipelineTest

2025-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53522: - Assignee: Wenchen Fan > simplify PipelineTest > - > >

[jira] [Updated] (SPARK-53413) Shuffle cleanup doesnot happen for commands

2025-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-53413: -- Affects Version/s: 4.1.0 (was: 4.0.0) > Shuffle cleanup doesnot hap

[jira] [Resolved] (SPARK-53498) [SDP] Add `pyspark.pipelines` to `classic/setup.py`

2025-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-53498. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 52254 [https://

[jira] [Updated] (SPARK-53413) Shuffle cleanup doesnot happen for commands

2025-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-53413: -- Parent: SPARK-51166 Issue Type: Sub-task (was: Bug) > Shuffle cleanup doesnot happen

[jira] [Assigned] (SPARK-53413) Shuffle cleanup doesnot happen for commands

2025-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53413: - Assignee: Karuppayya > Shuffle cleanup doesnot happen for commands > --

[jira] [Resolved] (SPARK-53413) Shuffle cleanup doesnot happen for commands

2025-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-53413. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 52157 [https://

[jira] [Assigned] (SPARK-53498) [SDP] Add `pyspark.pipelines` to `classic/setup.py`

2025-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53498: - Assignee: Anish Mahto > [SDP] Add `pyspark.pipelines` to `classic/setup.py` > -

[jira] [Updated] (SPARK-53538) Update with nondeterministic assigments can fail when whole-stage codegen is off

2025-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-53538: -- Parent: SPARK-51166 Issue Type: Sub-task (was: Bug) > Update with nondeterministic as

[jira] [Updated] (SPARK-53538) Update with nondeterministic assigments can fail when whole-stage codegen is off

2025-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-53538: -- Fix Version/s: 4.0.2 > Update with nondeterministic assigments can fail when whole-stage codeg

[jira] [Assigned] (SPARK-53557) Reduce automated vote email deadline from 4 days to 73 hours

2025-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53557: - Assignee: Dongjoon Hyun > Reduce automated vote email deadline from 4 days to 73 hours

[jira] [Resolved] (SPARK-53557) Reduce automated vote email deadline from 4 days to 73 hours

2025-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-53557. --- Fix Version/s: 4.1.0 3.5.7 4.0.2 Resolution: Fix

[jira] [Updated] (SPARK-53544) Support complex data type in python observed metrics

2025-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53544: --- Labels: pull-request-available (was: ) > Support complex data type in python observed metri

[jira] [Updated] (SPARK-53560) Crash looping when retrying uncommitted batch in Kafka source and AvailableNow trigger

2025-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53560: --- Labels: pull-request-available (was: ) > Crash looping when retrying uncommitted batch in K

[jira] [Assigned] (SPARK-53560) Crash looping when retrying uncommitted batch in Kafka source and AvailableNow trigger

2025-09-11 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar reassigned SPARK-53560: -- Assignee: Yuchen Liu > Crash looping when retrying uncommitted batch in Kafka

[jira] [Resolved] (SPARK-53560) Crash looping when retrying uncommitted batch in Kafka source and AvailableNow trigger

2025-09-11 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar resolved SPARK-53560. Fix Version/s: 4.1.0 Resolution: Fixed > Crash looping when retrying uncomm

[jira] [Commented] (SPARK-53560) Crash looping when retrying uncommitted batch in Kafka source and AvailableNow trigger

2025-09-11 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18019742#comment-18019742 ] Anish Shrigondekar commented on SPARK-53560: PR merged here: https://github.

[jira] [Commented] (SPARK-53560) Crash looping when retrying uncommitted batch in Kafka source and AvailableNow trigger

2025-09-11 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18019743#comment-18019743 ] Anish Shrigondekar commented on SPARK-53560: [~eason-yuchen-liu] - Pls creat

[jira] [Updated] (SPARK-53562) Limit Arrow batch sizes in applyInArrow and applyInPandas

2025-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53562: --- Labels: pull-request-available (was: ) > Limit Arrow batch sizes in applyInArrow and applyI

[jira] [Created] (SPARK-53562) Limit Arrow batch sizes in applyInArrow and applyInPandas

2025-09-11 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-53562: - Summary: Limit Arrow batch sizes in applyInArrow and applyInPandas Key: SPARK-53562 URL: https://issues.apache.org/jira/browse/SPARK-53562 Project: Spark I

[jira] [Resolved] (SPARK-53561) Catch Interruption Exception in TransformWithStateInPySparkStateServer during outputStream.flush at the thread close time to avoid the worker crash

2025-09-11 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar resolved SPARK-53561. Fix Version/s: 4.1.0 Resolution: Fixed > Catch Interruption Exception in Tr

[jira] [Closed] (SPARK-53561) Catch Interruption Exception in TransformWithStateInPySparkStateServer during outputStream.flush at the thread close time to avoid the worker crash

2025-09-11 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar closed SPARK-53561. -- > Catch Interruption Exception in TransformWithStateInPySparkStateServer during > outputS

[jira] [Assigned] (SPARK-53561) Catch Interruption Exception in TransformWithStateInPySparkStateServer during outputStream.flush at the thread close time to avoid the worker crash

2025-09-11 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar reassigned SPARK-53561: -- Assignee: Huanli Wang > Catch Interruption Exception in TransformWithStateInP

[jira] [Commented] (SPARK-53561) Catch Interruption Exception in TransformWithStateInPySparkStateServer during outputStream.flush at the thread close time to avoid the worker crash

2025-09-11 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18019719#comment-18019719 ] Anish Shrigondekar commented on SPARK-53561: PR has been merged here: https:

[jira] [Assigned] (SPARK-53549) Fix the list state memory leak issue due to an unclosed Arrow allocator

2025-09-11 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar reassigned SPARK-53549: -- Assignee: Huanli Wang > Fix the list state memory leak issue due to an unclos

[jira] [Closed] (SPARK-53549) Fix the list state memory leak issue due to an unclosed Arrow allocator

2025-09-11 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar closed SPARK-53549. -- > Fix the list state memory leak issue due to an unclosed Arrow allocator > --

[jira] [Resolved] (SPARK-53549) Fix the list state memory leak issue due to an unclosed Arrow allocator

2025-09-11 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar resolved SPARK-53549. Fix Version/s: 4.1.0 Resolution: Fixed > Fix the list state memory leak iss

[jira] [Commented] (SPARK-53549) Fix the list state memory leak issue due to an unclosed Arrow allocator

2025-09-11 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18019696#comment-18019696 ] Anish Shrigondekar commented on SPARK-53549: PR merged here: https://github

[jira] [Updated] (SPARK-53561) Catch Interruption Exception in TransformWithStateInPySparkStateServer during outputStream.flush at the thread close time to avoid the worker crash

2025-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53561: --- Labels: pull-request-available (was: ) > Catch Interruption Exception in TransformWithState

[jira] [Updated] (SPARK-53387) Support PARTITION BY clause with Arrow Python UDTFs

2025-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53387: --- Labels: pull-request-available (was: ) > Support PARTITION BY clause with Arrow Python UDTF

[jira] [Updated] (SPARK-53559) Refactor HLL sketch update to use collation key bytes instead of string

2025-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53559: --- Labels: pull-request-available (was: ) > Refactor HLL sketch update to use collation key by

[jira] [Updated] (SPARK-53559) Refactor HLL sketch update to use collation key bytes instead of string

2025-09-11 Thread Christopher Boumalhab (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Boumalhab updated SPARK-53559: -- Description: Update the {{sketch.update}} logic to use {{CollationFactory.get

[jira] [Created] (SPARK-53559) Refactor HLL sketch update to use collation key bytes instead of string

2025-09-11 Thread Christopher Boumalhab (Jira)
Christopher Boumalhab created SPARK-53559: - Summary: Refactor HLL sketch update to use collation key bytes instead of string Key: SPARK-53559 URL: https://issues.apache.org/jira/browse/SPARK-53559

[jira] [Created] (SPARK-53558) Show fully qualified table name including the catalog name in the exception message when the table is not found

2025-09-11 Thread Ganesha Shreedhara (Jira)
Ganesha Shreedhara created SPARK-53558: -- Summary: Show fully qualified table name including the catalog name in the exception message when the table is not found Key: SPARK-53558 URL: https://issues.apache.or

[jira] [Updated] (SPARK-53558) Show fully qualified table name including the catalog name in the exception message when the table is not found

2025-09-11 Thread Ganesha Shreedhara (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ganesha Shreedhara updated SPARK-53558: --- Component/s: SQL (was: Spark Core) > Show fully qualified table

[jira] [Updated] (SPARK-53558) Show fully qualified table name including the catalog name in the exception message when the table is not found

2025-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53558: --- Labels: pull-request-available (was: ) > Show fully qualified table name including the cata

[jira] [Commented] (SPARK-53548) Support `def try_make_timestamp_ntz(date, time): Column` in Python

2025-09-11 Thread Yicong Huang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18019677#comment-18019677 ] Yicong Huang commented on SPARK-53548: -- I will work on this. > Support `def try_ma

[jira] [Commented] (SPARK-53547) Add `def make_timestamp_ntz(date, time): Column` in Python

2025-09-11 Thread Yicong Huang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18019676#comment-18019676 ] Yicong Huang commented on SPARK-53547: -- I will work on this. > Add `def make_times

[jira] [Created] (SPARK-53557) Reduce automated vote email deadline from 4 days to 73 hours

2025-09-11 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-53557: - Summary: Reduce automated vote email deadline from 4 days to 73 hours Key: SPARK-53557 URL: https://issues.apache.org/jira/browse/SPARK-53557 Project: Spark

[jira] [Updated] (SPARK-53556) Avoid setting redundant struct data types in LiteralValueProtoConverter

2025-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53556: --- Labels: pull-request-available (was: ) > Avoid setting redundant struct data types in Liter

[jira] [Resolved] (SPARK-53525) Spark Connect ArrowBatch Result Chunking

2025-09-11 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-53525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-53525. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 52271 [

[jira] [Created] (SPARK-53556) Avoid setting redundant struct data types in LiteralValueProtoConverter

2025-09-11 Thread Yihong He (Jira)
Yihong He created SPARK-53556: - Summary: Avoid setting redundant struct data types in LiteralValueProtoConverter Key: SPARK-53556 URL: https://issues.apache.org/jira/browse/SPARK-53556 Project: Spark

[jira] [Assigned] (SPARK-53555) SparkML-connect can't load SparkML (legacy mode) saved model

2025-09-11 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu reassigned SPARK-53555: -- Assignee: Weichen Xu > SparkML-connect can't load SparkML (legacy mode) saved model > ---

[jira] [Created] (SPARK-53555) SparkML-connect can't load SparkML (legacy mode) saved model

2025-09-11 Thread Weichen Xu (Jira)
Weichen Xu created SPARK-53555: -- Summary: SparkML-connect can't load SparkML (legacy mode) saved model Key: SPARK-53555 URL: https://issues.apache.org/jira/browse/SPARK-53555 Project: Spark Iss

[jira] [Updated] (SPARK-52426) Support redirecting stdout/stderr to logging system

2025-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-52426: -- Parent: SPARK-51166 Issue Type: Sub-task (was: New Feature) > Support redirecting std

[jira] [Updated] (SPARK-53525) Spark Connect ArrowBatch Result Chunking

2025-09-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-53525: -- Parent: SPARK-51166 Issue Type: Sub-task (was: Improvement) > Spark Connect ArrowBatc

[jira] [Created] (SPARK-53554) Fix sum_distinct in spark-connect-rust

2025-09-11 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-53554: -- Summary: Fix sum_distinct in spark-connect-rust Key: SPARK-53554 URL: https://issues.apache.org/jira/browse/SPARK-53554 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-53399) Restore CollapseProject to merge Python UDFs

2025-09-11 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth resolved SPARK-53399. Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 52238 [https://github

[jira] [Updated] (SPARK-53399) Restore CollapseProject to merge Python UDFs

2025-09-11 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-53399: --- Summary: Restore CollapseProject to merge Python UDFs (was: Restore CollapseProject to merge unrela

[jira] [Updated] (SPARK-53536) Add a Golden File Test With Randomly Generated SQL Scripts

2025-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53536: --- Labels: pull-request-available (was: ) > Add a Golden File Test With Randomly Generated SQL

[jira] [Updated] (SPARK-53537) Add support for parsing CONTINUE HANDLER

2025-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-53537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53537: --- Labels: pull-request-available (was: ) > Add support for parsing CONTINUE HANDLER > ---

[jira] [Created] (SPARK-53551) Improve OffsetAndLimit by avoiding duplicate evaluation

2025-09-11 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-53551: -- Summary: Improve OffsetAndLimit by avoiding duplicate evaluation Key: SPARK-53551 URL: https://issues.apache.org/jira/browse/SPARK-53551 Project: Spark Issue Typ

[jira] [Updated] (SPARK-52601) TransformingEncoder should work with primitive types

2025-09-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52601: --- Labels: pull-request-available (was: ) > TransformingEncoder should work with primitive typ

[jira] [Created] (SPARK-53550) Union output partitioning should compare canonicalized attributes

2025-09-11 Thread L. C. Hsieh (Jira)
L. C. Hsieh created SPARK-53550: --- Summary: Union output partitioning should compare canonicalized attributes Key: SPARK-53550 URL: https://issues.apache.org/jira/browse/SPARK-53550 Project: Spark