[jira] [Created] (SPARK-46088) Add a self-contained example about creating dataframe from jdbc

BingKun Pan created SPARK-46088: --- Summary: Add a self-contained example about creating dataframe from jdbc Key: SPARK-46088 URL: https://issues.apache.org/jira/browse/SPARK-46088 Project: Spark

[jira] [Updated] (SPARK-46054) SPIP: Proposal to Adopt Google's Spark K8s Operator as Official Spark Operator

[ https://issues.apache.org/jira/browse/SPARK-46054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vara Bonthu updated SPARK-46054: Description: *Description:* This proposal aims to recommend the adoption of [Google's Spark K8s

[jira] [Resolved] (SPARK-46084) Refactor data type casting operation for Categorical type.

[ https://issues.apache.org/jira/browse/SPARK-46084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46084. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43993

[jira] [Resolved] (SPARK-46073) Remove the special resolution of UnresolvedNamespace for certain commands

[ https://issues.apache.org/jira/browse/SPARK-46073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46073. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43980

[jira] [Assigned] (SPARK-46083) Make SparkNoSuchElementException as a canonical error API

[ https://issues.apache.org/jira/browse/SPARK-46083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46083: - Assignee: Hyukjin Kwon > Make SparkNoSuchElementException as a canonical error API >

[jira] [Resolved] (SPARK-46083) Make SparkNoSuchElementException as a canonical error API

[ https://issues.apache.org/jira/browse/SPARK-46083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46083. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43992

[jira] [Updated] (SPARK-46087) Sync PySpark dependencies in docs and dev requirements

[ https://issues.apache.org/jira/browse/SPARK-46087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46087: --- Labels: pull-request-available (was: ) > Sync PySpark dependencies in docs and dev

[jira] [Created] (SPARK-46087) Sync PySpark dependencies in docs and dev requirements

Haejoon Lee created SPARK-46087: --- Summary: Sync PySpark dependencies in docs and dev requirements Key: SPARK-46087 URL: https://issues.apache.org/jira/browse/SPARK-46087 Project: Spark Issue

[jira] [Comment Edited] (SPARK-45311) Encoder fails on many "NoSuchElementException: None.get" since 3.4.x, search for an encoder for a generic type, and since 3.5.x isn't "an expression encoder"

[ https://issues.apache.org/jira/browse/SPARK-45311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789328#comment-17789328 ] Marc Le Bihan edited comment on SPARK-45311 at 11/24/23 5:39 AM: - A

[jira] [Comment Edited] (SPARK-45311) Encoder fails on many "NoSuchElementException: None.get" since 3.4.x, search for an encoder for a generic type, and since 3.5.x isn't "an expression encoder"

[ https://issues.apache.org/jira/browse/SPARK-45311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789328#comment-17789328 ] Marc Le Bihan edited comment on SPARK-45311 at 11/24/23 5:39 AM: - A

[jira] [Updated] (SPARK-45356) Adjust the Maven daily test configuration

[ https://issues.apache.org/jira/browse/SPARK-45356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45356: - Summary: Adjust the Maven daily test configuration (was: Optimize the Maven daily test configuration)

[jira] [Commented] (SPARK-45311) Encoder fails on many "NoSuchElementException: None.get" since 3.4.x, search for an encoder for a generic type, and since 3.5.x isn't "an expression encoder"

[ https://issues.apache.org/jira/browse/SPARK-45311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789328#comment-17789328 ] Marc Le Bihan commented on SPARK-45311: --- A breakpoint in :     at

[jira] [Updated] (SPARK-45311) Encoder fails on many "NoSuchElementException: None.get" since 3.4.x, search for an encoder for a generic type, and since 3.5.x isn't "an expression encoder"

[ https://issues.apache.org/jira/browse/SPARK-45311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc Le Bihan updated SPARK-45311: -- Attachment: sparkIssue_02.png > Encoder fails on many "NoSuchElementException: None.get"

[jira] [Updated] (SPARK-45311) Encoder fails on many "NoSuchElementException: None.get" since 3.4.x, search for an encoder for a generic type, and since 3.5.x isn't "an expression encoder"

[ https://issues.apache.org/jira/browse/SPARK-45311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marc Le Bihan updated SPARK-45311: -- Attachment: JavaTypeInference_116.png > Encoder fails on many "NoSuchElementException:

[jira] [Comment Edited] (SPARK-45311) Encoder fails on many "NoSuchElementException: None.get" since 3.4.x, search for an encoder for a generic type, and since 3.5.x isn't "an expression encoder"

[ https://issues.apache.org/jira/browse/SPARK-45311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789321#comment-17789321 ] Marc Le Bihan edited comment on SPARK-45311 at 11/24/23 5:15 AM: - I've

[jira] [Updated] (SPARK-46058) [CORE] Add separate flag for privateKeyPassword

[ https://issues.apache.org/jira/browse/SPARK-46058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46058: --- Labels: pull-request-available (was: ) > [CORE] Add separate flag for privateKeyPassword >

[jira] [Commented] (SPARK-45311) Encoder fails on many "NoSuchElementException: None.get" since 3.4.x, search for an encoder for a generic type, and since 3.5.x isn't "an expression encoder"

[ https://issues.apache.org/jira/browse/SPARK-45311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789321#comment-17789321 ] Marc Le Bihan commented on SPARK-45311: --- I've updated the  

[jira] [Created] (SPARK-46086) Fix potential non-atomic operation issues in ReloadingX509TrustManager

Yang Jie created SPARK-46086: Summary: Fix potential non-atomic operation issues in ReloadingX509TrustManager Key: SPARK-46086 URL: https://issues.apache.org/jira/browse/SPARK-46086 Project: Spark

[jira] [Updated] (SPARK-46086) Fix potential non-atomic operation issues in ReloadingX509TrustManager

[ https://issues.apache.org/jira/browse/SPARK-46086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46086: --- Labels: pull-request-available (was: ) > Fix potential non-atomic operation issues in

[jira] [Updated] (SPARK-45568) WholeStageCodegenSparkSubmitSuite flakiness

[ https://issues.apache.org/jira/browse/SPARK-45568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-45568: --- Component/s: Tests > WholeStageCodegenSparkSubmitSuite flakiness >

[jira] [Updated] (SPARK-45751) The default value of ‘spark.executor.logs.rolling.maxRetainedFiles' on the official website is incorrect

[ https://issues.apache.org/jira/browse/SPARK-45751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-45751: --- Component/s: Documentation > The default value of ‘spark.executor.logs.rolling.maxRetainedFiles' on

[jira] [Updated] (SPARK-45791) Rename `SparkConnectSessionHodlerSuite.scala` to `SparkConnectSessionHolderSuite.scala`

[ https://issues.apache.org/jira/browse/SPARK-45791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-45791: --- Component/s: Tests > Rename `SparkConnectSessionHodlerSuite.scala` to >

[jira] [Updated] (SPARK-46016) Fix pandas API support list properly

[ https://issues.apache.org/jira/browse/SPARK-46016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46016: --- Labels: pull-request-available (was: ) > Fix pandas API support list properly >

[jira] [Updated] (SPARK-46016) Fix pandas API support list properly

[ https://issues.apache.org/jira/browse/SPARK-46016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-46016: Summary: Fix pandas API support list properly (was: Correct Supported pandas API list) > Fix

[jira] [Updated] (SPARK-46016) Correct Supported pandas API list

[ https://issues.apache.org/jira/browse/SPARK-46016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-46016: Summary: Correct Supported pandas API list (was: Fix the script for Supported pandas API to work

[jira] [Assigned] (SPARK-46082) Fix protobuf string representation for Pandas Functions API with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-46082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46082: Assignee: Hyukjin Kwon > Fix protobuf string representation for Pandas Functions API

[jira] [Resolved] (SPARK-46082) Fix protobuf string representation for Pandas Functions API with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-46082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46082. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43991

[jira] [Resolved] (SPARK-46067) Upgrade commons-compress to 1.25.0

[ https://issues.apache.org/jira/browse/SPARK-46067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-46067. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43974

[jira] [Assigned] (SPARK-46067) Upgrade commons-compress to 1.25.0

[ https://issues.apache.org/jira/browse/SPARK-46067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-46067: Assignee: Yang Jie > Upgrade commons-compress to 1.25.0 > -- > >

[jira] [Updated] (SPARK-46085) Dataset.groupingSets in Scala Spark Connect client

[ https://issues.apache.org/jira/browse/SPARK-46085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46085: --- Labels: pull-request-available (was: ) > Dataset.groupingSets in Scala Spark Connect

[jira] [Created] (SPARK-46085) Dataset.groupingSets in Scala Spark Connect client

Hyukjin Kwon created SPARK-46085: Summary: Dataset.groupingSets in Scala Spark Connect client Key: SPARK-46085 URL: https://issues.apache.org/jira/browse/SPARK-46085 Project: Spark Issue

[jira] [Updated] (SPARK-46084) Refactor data type casting operation for Categorical type.

[ https://issues.apache.org/jira/browse/SPARK-46084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46084: --- Labels: pull-request-available (was: ) > Refactor data type casting operation for

[jira] [Resolved] (SPARK-46080) Upgrade Cloudpickle to 3.0.0

[ https://issues.apache.org/jira/browse/SPARK-46080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46080. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43989

[jira] [Updated] (SPARK-46083) Make SparkNoSuchElementException as a canonical error API

[ https://issues.apache.org/jira/browse/SPARK-46083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46083: --- Labels: pull-request-available (was: ) > Make SparkNoSuchElementException as a canonical

[jira] [Assigned] (SPARK-46080) Upgrade Cloudpickle to 3.0.0

[ https://issues.apache.org/jira/browse/SPARK-46080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46080: - Assignee: Hyukjin Kwon > Upgrade Cloudpickle to 3.0.0 > >

[jira] [Updated] (SPARK-46084) Refactor data type casting operation for Categorical type.

[ https://issues.apache.org/jira/browse/SPARK-46084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-46084: Description: Using official API for better performance and readability. (was: Using official API

[jira] [Created] (SPARK-46083) Make SparkNoSuchElementException as a canonical error API

Hyukjin Kwon created SPARK-46083: Summary: Make SparkNoSuchElementException as a canonical error API Key: SPARK-46083 URL: https://issues.apache.org/jira/browse/SPARK-46083 Project: Spark

[jira] [Created] (SPARK-46084) Refactor data type casting operation for Categorical type.

Haejoon Lee created SPARK-46084: --- Summary: Refactor data type casting operation for Categorical type. Key: SPARK-46084 URL: https://issues.apache.org/jira/browse/SPARK-46084 Project: Spark

[jira] [Updated] (SPARK-46082) Fix protobuf string representation for Pandas Functions API with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-46082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46082: --- Labels: pull-request-available (was: ) > Fix protobuf string representation for Pandas

[jira] [Created] (SPARK-46082) Fix protobuf string representation for Pandas Functions API with Spark Connect

Hyukjin Kwon created SPARK-46082: Summary: Fix protobuf string representation for Pandas Functions API with Spark Connect Key: SPARK-46082 URL: https://issues.apache.org/jira/browse/SPARK-46082

[jira] [Updated] (SPARK-42891) Implement CoGrouped Map API

[ https://issues.apache.org/jira/browse/SPARK-42891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-42891: --- Labels: pull-request-available (was: ) > Implement CoGrouped Map API >

[jira] [Updated] (SPARK-46080) Upgrade Cloudpickle to 3.0.0

[ https://issues.apache.org/jira/browse/SPARK-46080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46080: -- Parent: SPARK-45981 Issue Type: Sub-task (was: Improvement) > Upgrade Cloudpickle to

[jira] [Assigned] (SPARK-46081) Set DEDICATED_JVM_SBT_TESTS in `build_java21.yml`

[ https://issues.apache.org/jira/browse/SPARK-46081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46081: - Assignee: Dongjoon Hyun > Set DEDICATED_JVM_SBT_TESTS in `build_java21.yml` >

[jira] [Resolved] (SPARK-46081) Set DEDICATED_JVM_SBT_TESTS in `build_java21.yml`

[ https://issues.apache.org/jira/browse/SPARK-46081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46081. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43990

[jira] [Assigned] (SPARK-46079) Install `torch` nightly only at Python 3.12 in Infra docker image

[ https://issues.apache.org/jira/browse/SPARK-46079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46079: - Assignee: Dongjoon Hyun > Install `torch` nightly only at Python 3.12 in Infra docker

[jira] [Resolved] (SPARK-46079) Install `torch` nightly only at Python 3.12 in Infra docker image

[ https://issues.apache.org/jira/browse/SPARK-46079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46079. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43988

[jira] [Updated] (SPARK-46081) Set DEDICATED_JVM_SBT_TESTS in `build_java21.yml`

[ https://issues.apache.org/jira/browse/SPARK-46081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46081: --- Labels: pull-request-available (was: ) > Set DEDICATED_JVM_SBT_TESTS in `build_java21.yml`

[jira] [Updated] (SPARK-46080) Upgrade Cloudpickle to 3.0.0

[ https://issues.apache.org/jira/browse/SPARK-46080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46080: --- Labels: pull-request-available (was: ) > Upgrade Cloudpickle to 3.0.0 >

[jira] [Created] (SPARK-46081) Set DEDICATED_JVM_SBT_TESTS in `build_java21.yml`

Dongjoon Hyun created SPARK-46081: - Summary: Set DEDICATED_JVM_SBT_TESTS in `build_java21.yml` Key: SPARK-46081 URL: https://issues.apache.org/jira/browse/SPARK-46081 Project: Spark Issue

[jira] [Created] (SPARK-46080) Upgrade Cloudpickle to 3.0.0

Hyukjin Kwon created SPARK-46080: Summary: Upgrade Cloudpickle to 3.0.0 Key: SPARK-46080 URL: https://issues.apache.org/jira/browse/SPARK-46080 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-42905) pyspark.ml.stat.Correlation - Spearman Correlation method giving incorrect and inconsistent results for the same DataFrame if it has huge amount of Ties.

[ https://issues.apache.org/jira/browse/SPARK-42905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-42905: --- Labels: correctness pull-request-available (was: correctness) >

[jira] [Updated] (SPARK-46079) Install `torch` nightly only at Python 3.12 in Infra docker image

[ https://issues.apache.org/jira/browse/SPARK-46079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46079: --- Labels: pull-request-available (was: ) > Install `torch` nightly only at Python 3.12 in

[jira] [Created] (SPARK-46079) Install `torch` nightly only at Python 3.12 in Infra docker image

Dongjoon Hyun created SPARK-46079: - Summary: Install `torch` nightly only at Python 3.12 in Infra docker image Key: SPARK-46079 URL: https://issues.apache.org/jira/browse/SPARK-46079 Project: Spark

[jira] [Resolved] (SPARK-46076) Remove `unittest` deprecated alias usage for Python 3.12

[ https://issues.apache.org/jira/browse/SPARK-46076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46076. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43986

[jira] [Updated] (SPARK-46076) Remove `unittest` deprecated alias usage for Python 3.12

[ https://issues.apache.org/jira/browse/SPARK-46076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46076: -- Summary: Remove `unittest` deprecated alias usage for Python 3.12 (was: Remove `unittest`

[jira] [Assigned] (SPARK-46076) Remove `unittest` alias usage for Python 3.12

[ https://issues.apache.org/jira/browse/SPARK-46076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46076: - Assignee: Dongjoon Hyun > Remove `unittest` alias usage for Python 3.12 >

[jira] [Created] (SPARK-46078) Upgrade `pytorch` for Python 3.12

Dongjoon Hyun created SPARK-46078: - Summary: Upgrade `pytorch` for Python 3.12 Key: SPARK-46078 URL: https://issues.apache.org/jira/browse/SPARK-46078 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-46076) Remove `unittest` alias usage for Python 3.12

[ https://issues.apache.org/jira/browse/SPARK-46076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46076: -- Parent: SPARK-45981 Issue Type: Sub-task (was: Test) > Remove `unittest` alias usage

[jira] [Created] (SPARK-46077) Error in postgresql when pushing down filter by timestamp_ntz field

Marina Krasilnikova created SPARK-46077: --- Summary: Error in postgresql when pushing down filter by timestamp_ntz field Key: SPARK-46077 URL: https://issues.apache.org/jira/browse/SPARK-46077

[jira] [Updated] (SPARK-46076) Remove `unittest` alias usage for Python 3.12

[ https://issues.apache.org/jira/browse/SPARK-46076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46076: --- Labels: pull-request-available (was: ) > Remove `unittest` alias usage for Python 3.12 >

[jira] [Created] (SPARK-46076) Remove `unittest` alias usage for Python 3.12

Dongjoon Hyun created SPARK-46076: - Summary: Remove `unittest` alias usage for Python 3.12 Key: SPARK-46076 URL: https://issues.apache.org/jira/browse/SPARK-46076 Project: Spark Issue Type:

[jira] [Updated] (SPARK-46075) Refactor SparkConnectSessionManager to not use guava cache

[ https://issues.apache.org/jira/browse/SPARK-46075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46075: --- Labels: pull-request-available (was: ) > Refactor SparkConnectSessionManager to not use

[jira] [Commented] (SPARK-37660) Spark-3.2.0 Fetch Hbase Data not working

[ https://issues.apache.org/jira/browse/SPARK-37660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789235#comment-17789235 ] Istvan Toth commented on SPARK-37660: - I have encountered this. There are several issues: - Hbase

[jira] [Updated] (SPARK-46011) Spark Connect session heartbeat / keepalive

[ https://issues.apache.org/jira/browse/SPARK-46011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliusz Sompolski updated SPARK-46011: -- Epic Link: SPARK-43754 > Spark Connect session heartbeat / keepalive >

[jira] [Resolved] (SPARK-46011) Spark Connect session heartbeat / keepalive

[ https://issues.apache.org/jira/browse/SPARK-46011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliusz Sompolski resolved SPARK-46011. --- Resolution: Won't Fix Decided to not add this at this point. > Spark Connect

[jira] [Created] (SPARK-46075) Refactor SparkConnectSessionManager to not use guava cache

Juliusz Sompolski created SPARK-46075: - Summary: Refactor SparkConnectSessionManager to not use guava cache Key: SPARK-46075 URL: https://issues.apache.org/jira/browse/SPARK-46075 Project: Spark

[jira] [Updated] (SPARK-46074) [CONNECT][SCALA] Insufficient details in error when a UDF fails

[ https://issues.apache.org/jira/browse/SPARK-46074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46074: --- Labels: pull-request-available (was: ) > [CONNECT][SCALA] Insufficient details in error

[jira] [Created] (SPARK-46074) [CONNECT][SCALA] Insufficient details in error when a UDF fails

Niranjan Jayakar created SPARK-46074: Summary: [CONNECT][SCALA] Insufficient details in error when a UDF fails Key: SPARK-46074 URL: https://issues.apache.org/jira/browse/SPARK-46074 Project:

[jira] [Assigned] (SPARK-46070) Precompile regex patterns in SparkDateTimeUtils.getZoneId

[ https://issues.apache.org/jira/browse/SPARK-46070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-46070: Assignee: Tanel Kiis > Precompile regex patterns in SparkDateTimeUtils.getZoneId >

[jira] [Resolved] (SPARK-46070) Precompile regex patterns in SparkDateTimeUtils.getZoneId

[ https://issues.apache.org/jira/browse/SPARK-46070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-46070. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43976

[jira] [Updated] (SPARK-46069) Support unwrap timestamp type to date type

[ https://issues.apache.org/jira/browse/SPARK-46069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46069: --- Labels: pull-request-available (was: ) > Support unwrap timestamp type to date type >

[jira] [Assigned] (SPARK-43980) Add support for EXCEPT in select clause, similar to what databricks provides

[ https://issues.apache.org/jira/browse/SPARK-43980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-43980: --- Assignee: Yash Kothari > Add support for EXCEPT in select clause, similar to what

[jira] [Resolved] (SPARK-43980) Add support for EXCEPT in select clause, similar to what databricks provides

[ https://issues.apache.org/jira/browse/SPARK-43980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-43980. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43843

[jira] [Assigned] (SPARK-46062) CTE reference node does not inherit the flag `isStreaming` from CTE definition node

[ https://issues.apache.org/jira/browse/SPARK-46062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-46062: Assignee: Jungtaek Lim > CTE reference node does not inherit the flag `isStreaming` from

[jira] [Updated] (SPARK-46054) SPIP: Proposal to Adopt Google's Spark K8s Operator as Official Spark Operator

[ https://issues.apache.org/jira/browse/SPARK-46054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vara Bonthu updated SPARK-46054: Description: *Description:* This proposal aims to recommend the adoption of [Google's Spark K8s

[jira] [Resolved] (SPARK-46062) CTE reference node does not inherit the flag `isStreaming` from CTE definition node

[ https://issues.apache.org/jira/browse/SPARK-46062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-46062. -- Fix Version/s: 3.5.1 4.0.0 3.4.2 Resolution:

[jira] [Updated] (SPARK-46054) SPIP: Proposal to Adopt Google's Spark K8s Operator as Official Spark Operator

[ https://issues.apache.org/jira/browse/SPARK-46054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vara Bonthu updated SPARK-46054: Description: *Description:* This proposal aims to recommend the adoption of [Google's Spark K8s

[jira] [Created] (SPARK-46073) Remove the special resolution of UnresolvedNamespace for certain commands

Wenchen Fan created SPARK-46073: --- Summary: Remove the special resolution of UnresolvedNamespace for certain commands Key: SPARK-46073 URL: https://issues.apache.org/jira/browse/SPARK-46073 Project:

[jira] [Updated] (SPARK-46071) TreeNode.toJSON may result in OOM when there are multiple levels of nesting of expressions.

[ https://issues.apache.org/jira/browse/SPARK-46071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46071: --- Labels: pull-request-available (was: ) > TreeNode.toJSON may result in OOM when there are

[jira] [Resolved] (SPARK-46021) Support canceling future jobs belonging to a certain job group on `cancelJobGroup` call

[ https://issues.apache.org/jira/browse/SPARK-46021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46021. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43926

[jira] [Assigned] (SPARK-46021) Support canceling future jobs belonging to a certain job group on `cancelJobGroup` call

[ https://issues.apache.org/jira/browse/SPARK-46021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46021: --- Assignee: Xinyi Yu > Support canceling future jobs belonging to a certain job group on >

[jira] [Updated] (SPARK-30385) WebUI occasionally throw IOException on stop()

[ https://issues.apache.org/jira/browse/SPARK-30385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-30385: --- Labels: pull-request-available (was: ) > WebUI occasionally throw IOException on stop() >

[jira] [Updated] (SPARK-46054) SPIP: Proposal to Adopt Google's Spark K8s Operator as Official Spark Operator

[ https://issues.apache.org/jira/browse/SPARK-46054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-46054: - Fix Version/s: (was: 4.0.0) > SPIP: Proposal to Adopt Google's Spark K8s Operator as

[jira] [Resolved] (SPARK-46065) Refactor `(DataFrame|Series).factorize()` to use `create_map`.

[ https://issues.apache.org/jira/browse/SPARK-46065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46065. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43970

[jira] [Assigned] (SPARK-46065) Refactor `(DataFrame|Series).factorize()` to use `create_map`.

[ https://issues.apache.org/jira/browse/SPARK-46065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46065: Assignee: Haejoon Lee > Refactor `(DataFrame|Series).factorize()` to use `create_map`. >

[jira] [Updated] (SPARK-46072) Missing .jars when applying code to spark-connect

[ https://issues.apache.org/jira/browse/SPARK-46072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Kravchuk updated SPARK-46072: Summary: Missing .jars when applying code to spark-connect (was: Missing .jars when

[jira] [Created] (SPARK-46072) Missing .jars when trying to apply code to spark-connect

Dmitry Kravchuk created SPARK-46072: --- Summary: Missing .jars when trying to apply code to spark-connect Key: SPARK-46072 URL: https://issues.apache.org/jira/browse/SPARK-46072 Project: Spark

[jira] [Resolved] (SPARK-46064) EliminateEventTimeWatermark does not consider the fact that isStreaming flag can change for current child during resolution

[ https://issues.apache.org/jira/browse/SPARK-46064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-46064. -- Fix Version/s: 3.5.1 4.0.0 3.4.2 Resolution:

[jira] [Commented] (SPARK-45311) Encoder fails on many "NoSuchElementException: None.get" since 3.4.x, search for an encoder for a generic type, and since 3.5.x isn't "an expression encoder"

[ https://issues.apache.org/jira/browse/SPARK-45311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789058#comment-17789058 ] Giambattista Bloisi commented on SPARK-45311: - Could you disable failsafe trimStackTrace (as

[jira] [Updated] (SPARK-46070) Precompile regex patterns in SparkDateTimeUtils.getZoneId

[ https://issues.apache.org/jira/browse/SPARK-46070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46070: --- Labels: pull-request-available (was: ) > Precompile regex patterns in

[jira] [Created] (SPARK-46071) TreeNode.toJSON may result in OOM when there are multiple levels of nesting of expressions.

JacobZheng created SPARK-46071: -- Summary: TreeNode.toJSON may result in OOM when there are multiple levels of nesting of expressions. Key: SPARK-46071 URL: https://issues.apache.org/jira/browse/SPARK-46071

[jira] [Created] (SPARK-46070) Precompile regex patterns in SparkDateTimeUtils.getZoneId

Tanel Kiis created SPARK-46070: -- Summary: Precompile regex patterns in SparkDateTimeUtils.getZoneId Key: SPARK-46070 URL: https://issues.apache.org/jira/browse/SPARK-46070 Project: Spark Issue

[jira] [Created] (SPARK-46069) Support unwrap timestamp type to date type

Wan Kun created SPARK-46069: --- Summary: Support unwrap timestamp type to date type Key: SPARK-46069 URL: https://issues.apache.org/jira/browse/SPARK-46069 Project: Spark Issue Type: Improvement