[jira] [Assigned] (SPARK-53568) Fix several small bugs in Spark Connect Python client error handling logic

[ https://issues.apache.org/jira/browse/SPARK-53568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-53568: Assignee: Alex Khakhlyuk > Fix several small bugs in Spark Connect Python client er

[jira] [Resolved] (SPARK-53544) Support complex data type in python observed metrics

[ https://issues.apache.org/jira/browse/SPARK-53544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-53544. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 52321 [https

[jira] [Assigned] (SPARK-53182) Broken and missing links in PySpark DataFrames user guide

[ https://issues.apache.org/jira/browse/SPARK-53182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-53182: Assignee: Jonny Comes > Broken and missing links in PySpark DataFrames user gu

[jira] [Resolved] (SPARK-53182) Broken and missing links in PySpark DataFrames user guide

[ https://issues.apache.org/jira/browse/SPARK-53182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-53182. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 51851 [https

[jira] [Assigned] (SPARK-53544) Support complex data type in python observed metrics

[ https://issues.apache.org/jira/browse/SPARK-53544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-53544: Assignee: Yihong He > Support complex data type in python observed metr

[jira] [Assigned] (SPARK-53563) PS: SQLProcessor.execute() builds a string inefficiently by concatenation

[ https://issues.apache.org/jira/browse/SPARK-53563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-53563: Assignee: Peter Nguyen > PS: SQLProcessor.execute() builds a string inefficiently

[jira] [Created] (SPARK-53529) Spark Connect Client throw exception if remote is ipv6 arress

EdisonWang created SPARK-53529: -- Summary: Spark Connect Client throw exception if remote is ipv6 arress Key: SPARK-53529 URL: https://issues.apache.org/jira/browse/SPARK-53529 Project: Spark

[jira] [Commented] (SPARK-52381) JsonProtocol: tidy up deserialization

[ https://issues.apache.org/jira/browse/SPARK-52381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18020201#comment-18020201 ] PJ Fanning commented on SPARK-52381: https://github.com/apache/spark/pull/5

[jira] [Updated] (SPARK-52381) JsonProtocol: tidy up deserialization

[ https://issues.apache.org/jira/browse/SPARK-52381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PJ Fanning updated SPARK-52381: --- Fix Version/s: 4.0.1 > JsonProtocol: tidy up deserializat

[jira] [Created] (SPARK-53531) Better error message for HadoopRDD.getInputFormat

Cheng Pan created SPARK-53531: - Summary: Better error message for HadoopRDD.getInputFormat Key: SPARK-53531 URL: https://issues.apache.org/jira/browse/SPARK-53531 Project: Spark Issue Type

[jira] [Resolved] (SPARK-53569) Use `Iceberg` 1.10 for Iceberg integration test

[ https://issues.apache.org/jira/browse/SPARK-53569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-53569. --- Fix Version/s: connect-swift-0.4.0 Resolution: Fixed Issue resolved by pull request

[jira] [Updated] (SPARK-53559) Refactor HLL sketch update to use collation key bytes instead of string

[ https://issues.apache.org/jira/browse/SPARK-53559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Boumalhab updated SPARK-53559: -- Description: Update the {{sketch.update}} logic to use CollationKey bytes

[jira] [Updated] (SPARK-53559) Refactor HLL sketch update to use collation key bytes instead of string

[ https://issues.apache.org/jira/browse/SPARK-53559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Boumalhab updated SPARK-53559: -- Description: Update the {{sketch.update}} logic to use CollationKey bytes

[jira] [Commented] (SPARK-53547) Add `def make_timestamp_ntz(date, time): Column` in Python

[ https://issues.apache.org/jira/browse/SPARK-53547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18020119#comment-18020119 ] Yicong Huang commented on SPARK-53547: -- I believe this is a duplicate of h

[jira] [Created] (SPARK-53532) Upgrade Jetty to 9.4.58.v20250814

Mingliang Zhu created SPARK-53532: - Summary: Upgrade Jetty to 9.4.58.v20250814 Key: SPARK-53532 URL: https://issues.apache.org/jira/browse/SPARK-53532 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-53573) Expand usage of parameter markers by means of a pre-processing parser

[ https://issues.apache.org/jira/browse/SPARK-53573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Serge Rielau updated SPARK-53573: - Description: The current implementation of parameter markers (:Parm and ?) is limited to two

[jira] [Created] (SPARK-53573) Expand usage of parameter markers by means of a pre-processing parser

Serge Rielau created SPARK-53573: Summary: Expand usage of parameter markers by means of a pre-processing parser Key: SPARK-53573 URL: https://issues.apache.org/jira/browse/SPARK-53573 Project: Spark

[jira] [Resolved] (SPARK-52601) TransformingEncoder should work with primitive types

[ https://issues.apache.org/jira/browse/SPARK-52601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-52601. --- Fix Version/s: 4.1.0 Assignee: Emil Ejbyfeldt Resolution: Fixed

[jira] [Updated] (SPARK-53547) Add `def make_timestamp_ntz(date, time): Column` in Python

[ https://issues.apache.org/jira/browse/SPARK-53547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53547: --- Labels: pull-request-available (was: ) > Add `def make_timestamp_ntz(date, time): Col

[jira] [Updated] (SPARK-53572) Avoid throwing from `ExtractValue.isExtractable`

[ https://issues.apache.org/jira/browse/SPARK-53572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53572: --- Labels: pull-request-available (was: ) > Avoid throwing from `ExtractValue.isExtracta

[jira] [Created] (SPARK-53572) Avoid throwing from `ExtractValue.isExtractable`

Vladimir Golubev created SPARK-53572: Summary: Avoid throwing from `ExtractValue.isExtractable` Key: SPARK-53572 URL: https://issues.apache.org/jira/browse/SPARK-53572 Project: Spark

[jira] [Updated] (SPARK-53571) Add `integration-test-mac-spark4-iceberg` GitHub Action job

[ https://issues.apache.org/jira/browse/SPARK-53571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53571: --- Labels: pull-request-available (was: ) > Add `integration-test-mac-spark4-iceberg` Git

[jira] [Assigned] (SPARK-53537) Add support for parsing CONTINUE HANDLER

[ https://issues.apache.org/jira/browse/SPARK-53537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-53537: --- Assignee: Teodor > Add support for parsing CONTINUE HAND

[jira] [Resolved] (SPARK-53537) Add support for parsing CONTINUE HANDLER

[ https://issues.apache.org/jira/browse/SPARK-53537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-53537. - Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 52298 [https

[jira] [Resolved] (SPARK-53444) Rework EXECUTE IMMEDIATE

[ https://issues.apache.org/jira/browse/SPARK-53444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-53444. - Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 52173 [https

[jira] [Assigned] (SPARK-53444) Rework EXECUTE IMMEDIATE

[ https://issues.apache.org/jira/browse/SPARK-53444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-53444: --- Assignee: Serge Rielau > Rework EXECUTE IMMEDI

[jira] [Assigned] (SPARK-53545) Enforce style check at project level for operator

[ https://issues.apache.org/jira/browse/SPARK-53545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth reassigned SPARK-53545: -- Assignee: Zhou JIANG > Enforce style check at project level for opera

[jira] [Updated] (SPARK-53569) Use `Iceberg` 1.10 for Iceberg integration test

[ https://issues.apache.org/jira/browse/SPARK-53569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53569: --- Labels: pull-request-available (was: ) > Use `Iceberg` 1.10 for Iceberg integration t

[jira] [Updated] (SPARK-53552) Optimize substr SQL function

[ https://issues.apache.org/jira/browse/SPARK-53552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53552: --- Labels: pull-request-available (was: ) > Optimize substr SQL funct

[jira] [Updated] (SPARK-43579) Cache the converter between Arrow and pandas for reuse

[ https://issues.apache.org/jira/browse/SPARK-43579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43579: --- Labels: pull-request-available (was: ) > Cache the converter between Arrow and pandas

[jira] [Updated] (SPARK-53529) Spark Connect Client throw exception if remote is ipv6 arress

[ https://issues.apache.org/jira/browse/SPARK-53529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-53529: -- Target Version/s: (was: 4.0.1) > Spark Connect Client throw exception if remote is i

[jira] [Assigned] (SPARK-53571) Add `integration-test-mac-spark4-iceberg` GitHub Action job

[ https://issues.apache.org/jira/browse/SPARK-53571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53571: - Assignee: Dongjoon Hyun > Add `integration-test-mac-spark4-iceberg` GitHub Action

[jira] [Resolved] (SPARK-53571) Add `integration-test-mac-spark4-iceberg` GitHub Action job

[ https://issues.apache.org/jira/browse/SPARK-53571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-53571. --- Fix Version/s: connect-swift-0.4.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-53570) Update `integration-test-token` to use Spark `4.1.0-preview`

[ https://issues.apache.org/jira/browse/SPARK-53570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53570: - Assignee: Dongjoon Hyun > Update `integration-test-token` to use Spark `4.1.0-prev

[jira] [Resolved] (SPARK-53570) Update `integration-test-token` to use Spark `4.1.0-preview`

[ https://issues.apache.org/jira/browse/SPARK-53570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-53570. --- Fix Version/s: connect-swift-0.4.0 Resolution: Fixed Issue resolved by pull request

[jira] [Created] (SPARK-53571) Add `integration-test-mac-spark4-iceberg` GitHub Action job

Dongjoon Hyun created SPARK-53571: - Summary: Add `integration-test-mac-spark4-iceberg` GitHub Action job Key: SPARK-53571 URL: https://issues.apache.org/jira/browse/SPARK-53571 Project: Spark

[jira] [Updated] (SPARK-53570) Update `integration-test-token` to use Spark `4.1.0-preview`

[ https://issues.apache.org/jira/browse/SPARK-53570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53570: --- Labels: pull-request-available (was: ) > Update `integration-test-token` to use Sp

[jira] [Updated] (SPARK-51162) SPIP: Add the TIME data type

[ https://issues.apache.org/jira/browse/SPARK-51162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-51162: --- Labels: SPIP pull-request-available releasenotes (was: SPIP releasenotes) > SPIP: Add

[jira] [Resolved] (SPARK-53157) Decouple driver and executor heartbeat intervals

[ https://issues.apache.org/jira/browse/SPARK-53157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-53157. - Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request

[jira] [Resolved] (SPARK-53545) Enforce style check at project level for operator

[ https://issues.apache.org/jira/browse/SPARK-53545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth resolved SPARK-53545. Fix Version/s: kubernetes-operator-0.5.0 Resolution: Fixed Issue resolved by pull request

[jira] [Updated] (SPARK-53157) Decouple driver and executor heartbeat intervals

[ https://issues.apache.org/jira/browse/SPARK-53157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-53157: -- Parent: SPARK-51166 Issue Type: Sub-task (was: Improvement) > Decouple driver

[jira] [Assigned] (SPARK-53157) Decouple driver and executor heartbeat intervals

[ https://issues.apache.org/jira/browse/SPARK-53157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-53157: --- Assignee: Victor Sunderland > Decouple driver and executor heartb

[jira] [Resolved] (SPARK-53529) Spark Connect Client throw exception if remote is ipv6 arress

[ https://issues.apache.org/jira/browse/SPARK-53529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-53529. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 52280 [https

[jira] [Assigned] (SPARK-53529) Spark Connect Client throw exception if remote is ipv6 arress

[ https://issues.apache.org/jira/browse/SPARK-53529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53529: - Assignee: EdisonWang > Spark Connect Client throw exception if remote is ipv6 arr

[jira] [Resolved] (SPARK-53567) Upgrade Netty to 4.1.125.Final

[ https://issues.apache.org/jira/browse/SPARK-53567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen resolved SPARK-53567. - Resolution: Fixed > Upgrade Netty to 4.1.125.Fi

[jira] [Created] (SPARK-53568) Fix several small bugs in Spark Connect Python client error handling logic

Alex Khakhlyuk created SPARK-53568: -- Summary: Fix several small bugs in Spark Connect Python client error handling logic Key: SPARK-53568 URL: https://issues.apache.org/jira/browse/SPARK-53568

[jira] [Commented] (SPARK-53567) Upgrade Netty to 4.1.125.Final

[ https://issues.apache.org/jira/browse/SPARK-53567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18019845#comment-18019845 ] Jota Martos commented on SPARK-53567: - sorry for the noise, didn't see yo

[jira] [Created] (SPARK-53567) Upgrade Netty to 4.1.125.Final

Jota Martos created SPARK-53567: --- Summary: Upgrade Netty to 4.1.125.Final Key: SPARK-53567 URL: https://issues.apache.org/jira/browse/SPARK-53567 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-23817) Create file source V2 framework and migrate ORC read path

[ https://issues.apache.org/jira/browse/SPARK-23817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-23817: --- Labels: pull-request-available (was: ) > Create file source V2 framework and migrate

[jira] [Created] (SPARK-53566) Docker images support OpenJDK

melin created SPARK-53566: - Summary: Docker images support OpenJDK Key: SPARK-53566 URL: https://issues.apache.org/jira/browse/SPARK-53566 Project: Spark Issue Type: Improvement Components

[jira] [Created] (SPARK-53561) Catch `InterruptedIOException` and `ClosedByInterruptException` in TransformWithStateInPySparkStateServer to avoid the worker crash

://issues.apache.org/jira/browse/SPARK-53561 Project: Spark Issue Type: Bug Components: Structured Streaming Affects Versions: 4.0.1, 4.0.0, 4.1.0 Reporter: Huanli Wang When the `query.stop` is invoked, we first close the the state store running in the query

[jira] [Updated] (SPARK-53523) Named parameters respect `spark.sql.caseSensitive`

[ https://issues.apache.org/jira/browse/SPARK-53523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-53523: -- Summary: Named parameters respect `spark.sql.caseSensitive` (was: NamedParameter respects

[jira] [Updated] (SPARK-53522) simplify PipelineTest

[ https://issues.apache.org/jira/browse/SPARK-53522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-53522: -- Parent: SPARK-51727 Issue Type: Sub-task (was: Test) > simplify PipelineT

[jira] [Assigned] (SPARK-53540) Remove `pypy3` from `spark-rm` image

[ https://issues.apache.org/jira/browse/SPARK-53540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53540: - Assignee: Dongjoon Hyun > Remove `pypy3` from `spark-rm` im

[jira] [Created] (SPARK-53565) return wrong result for some aggregate functions when groupSets contains '{}'

weihua zhang created SPARK-53565: Summary: return wrong result for some aggregate functions when groupSets contains '{}' Key: SPARK-53565 URL: https://issues.apache.org/jira/browse/S

[jira] [Updated] (SPARK-53550) Union output partitioning should compare canonicalized attributes

[ https://issues.apache.org/jira/browse/SPARK-53550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53550: --- Labels: pull-request-available (was: ) > Union output partitioning should comp

[jira] [Created] (SPARK-53564) DAGScheduler exits due to RPC timeout to blockmanager in DAGScheduler loop

Tengfei Huang created SPARK-53564: - Summary: DAGScheduler exits due to RPC timeout to blockmanager in DAGScheduler loop Key: SPARK-53564 URL: https://issues.apache.org/jira/browse/SPARK-53564 Project

[jira] [Created] (SPARK-53563) PS: SQLProcessor.execute() builds a string inefficiently by concatenation

Peter Nguyen created SPARK-53563: Summary: PS: SQLProcessor.execute() builds a string inefficiently by concatenation Key: SPARK-53563 URL: https://issues.apache.org/jira/browse/SPARK-53563 Project

[jira] [Updated] (SPARK-53563) PS: SQLProcessor.execute() builds a string inefficiently by concatenation

[ https://issues.apache.org/jira/browse/SPARK-53563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53563: --- Labels: pull-request-available (was: ) > PS: SQLProcessor.execute() builds a str

[jira] [Assigned] (SPARK-53491) SS | `inputRowsPerSecond` and `processedRowsPerSecond` in progress report show values in exponential notation for large numbers

[ https://issues.apache.org/jira/browse/SPARK-53491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-53491: Assignee: Jayant Sharma > SS | `inputRowsPerSecond` and `processedRowsPerSecond`

[jira] [Resolved] (SPARK-53491) SS | `inputRowsPerSecond` and `processedRowsPerSecond` in progress report show values in exponential notation for large numbers

[ https://issues.apache.org/jira/browse/SPARK-53491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-53491. -- Fix Version/s: 4.1.0 (was: 4.0.0) Resolution: Fixed Issue

[jira] [Updated] (SPARK-53557) Reduce automated vote email deadline from 4 days to 73 hours

[ https://issues.apache.org/jira/browse/SPARK-53557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53557: --- Labels: pull-request-available (was: ) > Reduce automated vote email deadline from 4 d

[jira] [Updated] (SPARK-50888) Fix Flaky Test: `SparkConnectServiceSuite.SPARK-44776: LocalTableScanExe`

[ https://issues.apache.org/jira/browse/SPARK-50888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50888: -- Fix Version/s: 4.0.2 > Fix Flaky Test: `SparkConnectServiceSuite.SPARK-44

[jira] [Updated] (SPARK-53527) Improve fallback of analyzeExistenceDefaultValue

[ https://issues.apache.org/jira/browse/SPARK-53527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53527: --- Labels: pull-request-available (was: ) > Improve fallback of analyzeExistenceDefaultVa

[jira] [Resolved] (SPARK-53550) Union output partitioning should compare canonicalized attributes

[ https://issues.apache.org/jira/browse/SPARK-53550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-53550. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 52305 [https

[jira] [Assigned] (SPARK-53522) simplify PipelineTest

[ https://issues.apache.org/jira/browse/SPARK-53522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53522: - Assignee: Wenchen Fan > simplify PipelineT

[jira] [Updated] (SPARK-53413) Shuffle cleanup doesnot happen for commands

[ https://issues.apache.org/jira/browse/SPARK-53413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-53413: -- Affects Version/s: 4.1.0 (was: 4.0.0) > Shuffle cleanup does

[jira] [Resolved] (SPARK-53498) [SDP] Add `pyspark.pipelines` to `classic/setup.py`

[ https://issues.apache.org/jira/browse/SPARK-53498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-53498. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 52254 [https

[jira] [Updated] (SPARK-53413) Shuffle cleanup doesnot happen for commands

[ https://issues.apache.org/jira/browse/SPARK-53413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-53413: -- Parent: SPARK-51166 Issue Type: Sub-task (was: Bug) > Shuffle cleanup doesnot hap

[jira] [Assigned] (SPARK-53413) Shuffle cleanup doesnot happen for commands

[ https://issues.apache.org/jira/browse/SPARK-53413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53413: - Assignee: Karuppayya > Shuffle cleanup doesnot happen for comma

[jira] [Resolved] (SPARK-53413) Shuffle cleanup doesnot happen for commands

[ https://issues.apache.org/jira/browse/SPARK-53413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-53413. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 52157 [https

[jira] [Assigned] (SPARK-53498) [SDP] Add `pyspark.pipelines` to `classic/setup.py`

[ https://issues.apache.org/jira/browse/SPARK-53498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53498: - Assignee: Anish Mahto > [SDP] Add `pyspark.pipelines` to `classic/setup

[jira] [Updated] (SPARK-53538) Update with nondeterministic assigments can fail when whole-stage codegen is off

[ https://issues.apache.org/jira/browse/SPARK-53538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-53538: -- Parent: SPARK-51166 Issue Type: Sub-task (was: Bug) > Update with nondeterminis

[jira] [Updated] (SPARK-53538) Update with nondeterministic assigments can fail when whole-stage codegen is off

[ https://issues.apache.org/jira/browse/SPARK-53538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-53538: -- Fix Version/s: 4.0.2 > Update with nondeterministic assigments can fail when whole-st

[jira] [Assigned] (SPARK-53557) Reduce automated vote email deadline from 4 days to 73 hours

[ https://issues.apache.org/jira/browse/SPARK-53557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-53557: - Assignee: Dongjoon Hyun > Reduce automated vote email deadline from 4 days to 73 ho

[jira] [Resolved] (SPARK-53557) Reduce automated vote email deadline from 4 days to 73 hours

[ https://issues.apache.org/jira/browse/SPARK-53557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-53557. --- Fix Version/s: 4.1.0 3.5.7 4.0.2 Resolution

[jira] [Updated] (SPARK-53544) Support complex data type in python observed metrics

[ https://issues.apache.org/jira/browse/SPARK-53544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53544: --- Labels: pull-request-available (was: ) > Support complex data type in python obser

[jira] [Updated] (SPARK-53560) Crash looping when retrying uncommitted batch in Kafka source and AvailableNow trigger

[ https://issues.apache.org/jira/browse/SPARK-53560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53560: --- Labels: pull-request-available (was: ) > Crash looping when retrying uncommitted batch

[jira] [Assigned] (SPARK-53560) Crash looping when retrying uncommitted batch in Kafka source and AvailableNow trigger

[ https://issues.apache.org/jira/browse/SPARK-53560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar reassigned SPARK-53560: -- Assignee: Yuchen Liu > Crash looping when retrying uncommitted batch

[jira] [Resolved] (SPARK-53560) Crash looping when retrying uncommitted batch in Kafka source and AvailableNow trigger

[ https://issues.apache.org/jira/browse/SPARK-53560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar resolved SPARK-53560. Fix Version/s: 4.1.0 Resolution: Fixed > Crash looping when retry

[jira] [Commented] (SPARK-53560) Crash looping when retrying uncommitted batch in Kafka source and AvailableNow trigger

[ https://issues.apache.org/jira/browse/SPARK-53560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18019742#comment-18019742 ] Anish Shrigondekar commented on SPARK-53560: PR merged here: h

[jira] [Commented] (SPARK-53560) Crash looping when retrying uncommitted batch in Kafka source and AvailableNow trigger

[ https://issues.apache.org/jira/browse/SPARK-53560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18019743#comment-18019743 ] Anish Shrigondekar commented on SPARK-53560: [~eason-yuchen-liu] -

[jira] [Updated] (SPARK-53562) Limit Arrow batch sizes in applyInArrow and applyInPandas

[ https://issues.apache.org/jira/browse/SPARK-53562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53562: --- Labels: pull-request-available (was: ) > Limit Arrow batch sizes in applyInArrow

[jira] [Created] (SPARK-53562) Limit Arrow batch sizes in applyInArrow and applyInPandas

Ruifeng Zheng created SPARK-53562: - Summary: Limit Arrow batch sizes in applyInArrow and applyInPandas Key: SPARK-53562 URL: https://issues.apache.org/jira/browse/SPARK-53562 Project: Spark

[jira] [Resolved] (SPARK-53561) Catch Interruption Exception in TransformWithStateInPySparkStateServer during outputStream.flush at the thread close time to avoid the worker crash

[ https://issues.apache.org/jira/browse/SPARK-53561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar resolved SPARK-53561. Fix Version/s: 4.1.0 Resolution: Fixed > Catch Interruption Exception

[jira] [Closed] (SPARK-53561) Catch Interruption Exception in TransformWithStateInPySparkStateServer during outputStream.flush at the thread close time to avoid the worker crash

[ https://issues.apache.org/jira/browse/SPARK-53561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar closed SPARK-53561. -- > Catch Interruption Exception in TransformWithStateInPySparkStateServer dur

[jira] [Assigned] (SPARK-53561) Catch Interruption Exception in TransformWithStateInPySparkStateServer during outputStream.flush at the thread close time to avoid the worker crash

[ https://issues.apache.org/jira/browse/SPARK-53561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar reassigned SPARK-53561: -- Assignee: Huanli Wang > Catch Interruption Exception

[jira] [Commented] (SPARK-53561) Catch Interruption Exception in TransformWithStateInPySparkStateServer during outputStream.flush at the thread close time to avoid the worker crash

[ https://issues.apache.org/jira/browse/SPARK-53561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18019719#comment-18019719 ] Anish Shrigondekar commented on SPARK-53561: PR has been merged here: h

[jira] [Assigned] (SPARK-53549) Fix the list state memory leak issue due to an unclosed Arrow allocator

[ https://issues.apache.org/jira/browse/SPARK-53549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar reassigned SPARK-53549: -- Assignee: Huanli Wang > Fix the list state memory leak issue due to

[jira] [Closed] (SPARK-53549) Fix the list state memory leak issue due to an unclosed Arrow allocator

[ https://issues.apache.org/jira/browse/SPARK-53549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar closed SPARK-53549. -- > Fix the list state memory leak issue due to an unclosed Arrow alloca

[jira] [Resolved] (SPARK-53549) Fix the list state memory leak issue due to an unclosed Arrow allocator

[ https://issues.apache.org/jira/browse/SPARK-53549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar resolved SPARK-53549. Fix Version/s: 4.1.0 Resolution: Fixed > Fix the list state memory l

[jira] [Commented] (SPARK-53549) Fix the list state memory leak issue due to an unclosed Arrow allocator

[ https://issues.apache.org/jira/browse/SPARK-53549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18019696#comment-18019696 ] Anish Shrigondekar commented on SPARK-53549: PR merged here: h

[jira] [Updated] (SPARK-53561) Catch Interruption Exception in TransformWithStateInPySparkStateServer during outputStream.flush at the thread close time to avoid the worker crash

[ https://issues.apache.org/jira/browse/SPARK-53561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53561: --- Labels: pull-request-available (was: ) > Catch Interruption Exception

[jira] [Updated] (SPARK-53387) Support PARTITION BY clause with Arrow Python UDTFs

[ https://issues.apache.org/jira/browse/SPARK-53387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53387: --- Labels: pull-request-available (was: ) > Support PARTITION BY clause with Arrow Pyt

[jira] [Updated] (SPARK-53559) Refactor HLL sketch update to use collation key bytes instead of string

[ https://issues.apache.org/jira/browse/SPARK-53559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53559: --- Labels: pull-request-available (was: ) > Refactor HLL sketch update to use collation

[jira] [Updated] (SPARK-53559) Refactor HLL sketch update to use collation key bytes instead of string

[ https://issues.apache.org/jira/browse/SPARK-53559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christopher Boumalhab updated SPARK-53559: -- Description: Update the {{sketch.update}} logic to use

[jira] [Created] (SPARK-53559) Refactor HLL sketch update to use collation key bytes instead of string

Christopher Boumalhab created SPARK-53559: - Summary: Refactor HLL sketch update to use collation key bytes instead of string Key: SPARK-53559 URL: https://issues.apache.org/jira/browse/SPARK-53559

[jira] [Created] (SPARK-53558) Show fully qualified table name including the catalog name in the exception message when the table is not found

://issues.apache.org/jira/browse/SPARK-53558 Project: Spark Issue Type: Task Components: Spark Core Affects Versions: 4.0.1 Reporter: Ganesha Shreedhara Currently, we only display the database name and table name in the error message when the table is not found. We

[jira] [Updated] (SPARK-53558) Show fully qualified table name including the catalog name in the exception message when the table is not found

[ https://issues.apache.org/jira/browse/SPARK-53558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ganesha Shreedhara updated SPARK-53558: --- Component/s: SQL (was: Spark Core) > Show fully qualif

[jira] [Updated] (SPARK-53558) Show fully qualified table name including the catalog name in the exception message when the table is not found

[ https://issues.apache.org/jira/browse/SPARK-53558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-53558: --- Labels: pull-request-available (was: ) > Show fully qualified table name including

  1   2   3   4   5   6   7   8   9   10   >