[jira] [Commented] (SPARK-43986) Add error classes for HyperLogLog functions

[ https://issues.apache.org/jira/browse/SPARK-43986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735507#comment-17735507 ] Hudson commented on SPARK-43986: User 'dtenedor' has created a pull request for this issue:

[jira] [Updated] (SPARK-44108) Cannot parse Type from german "umlaut"

[ https://issues.apache.org/jira/browse/SPARK-44108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Machado updated SPARK-44108: -- Priority: Major (was: Minor) > Cannot parse Type from german "umlaut" >

[jira] [Commented] (SPARK-43816) Spark Corrupts Data In-Transit for High Volume (> 20 TB/hr) of Data

[ https://issues.apache.org/jira/browse/SPARK-43816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735421#comment-17735421 ] Sai Allu commented on SPARK-43816: -- Thank you for solving this Yuming, I really appreciate your help!

[jira] [Created] (SPARK-44116) Utilize Hadoop vectorized APIs

Dongjoon Hyun created SPARK-44116: - Summary: Utilize Hadoop vectorized APIs Key: SPARK-44116 URL: https://issues.apache.org/jira/browse/SPARK-44116 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-44115) Upgrade to Apache ORC 2.0

Dongjoon Hyun created SPARK-44115: - Summary: Upgrade to Apache ORC 2.0 Key: SPARK-44115 URL: https://issues.apache.org/jira/browse/SPARK-44115 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-44120) Support Python 3.12

Dongjoon Hyun created SPARK-44120: - Summary: Support Python 3.12 Key: SPARK-44120 URL: https://issues.apache.org/jira/browse/SPARK-44120 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-44124) Upgrade AWS SDK to v2

Dongjoon Hyun created SPARK-44124: - Summary: Upgrade AWS SDK to v2 Key: SPARK-44124 URL: https://issues.apache.org/jira/browse/SPARK-44124 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-44125) Support Java 21 in SparkR

Dongjoon Hyun created SPARK-44125: - Summary: Support Java 21 in SparkR Key: SPARK-44125 URL: https://issues.apache.org/jira/browse/SPARK-44125 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-44127) Reenable test_sparkSQL_arrow.R test in Java 21

Dongjoon Hyun created SPARK-44127: - Summary: Reenable test_sparkSQL_arrow.R test in Java 21 Key: SPARK-44127 URL: https://issues.apache.org/jira/browse/SPARK-44127 Project: Spark Issue Type:

[jira] [Updated] (SPARK-44127) Reenable test_sparkSQL_arrow.R in Java 21

[ https://issues.apache.org/jira/browse/SPARK-44127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44127: -- Summary: Reenable test_sparkSQL_arrow.R in Java 21 (was: Reenable test_sparkSQL_arrow.R test

[jira] [Commented] (SPARK-44114) Upgrade built-in Hive to 3.x

[ https://issues.apache.org/jira/browse/SPARK-44114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735440#comment-17735440 ] Dongjoon Hyun commented on SPARK-44114: --- Hi, [~yumwang]. What do you think about this although it

[jira] [Commented] (SPARK-43351) Support Golang in Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-43351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735448#comment-17735448 ] Dongjoon Hyun commented on SPARK-43351: --- If this is not aligned to Apache Spark 3.5.0, you can

[jira] [Updated] (SPARK-44098) Introduce python breaking change detection

[ https://issues.apache.org/jira/browse/SPARK-44098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-44098: -- Affects Version/s: 4.0.0 (was: 3.5.0) > Introduce python breaking

[jira] [Created] (SPARK-44111) Prepare Apache Spark 4.0.0

Dongjoon Hyun created SPARK-44111: - Summary: Prepare Apache Spark 4.0.0 Key: SPARK-44111 URL: https://issues.apache.org/jira/browse/SPARK-44111 Project: Spark Issue Type: Umbrella

[jira] [Updated] (SPARK-44108) Cannot parse Type from german "umlaut"

[ https://issues.apache.org/jira/browse/SPARK-44108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Machado updated SPARK-44108: -- Description: Hello all,    I have a client that has a column named : bfzgtäeil Spark

[jira] [Updated] (SPARK-44108) Cannot parse Type from german "umlaut"

[ https://issues.apache.org/jira/browse/SPARK-44108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Machado updated SPARK-44108: -- Description: Hello all,    I have a client that has a column named : bfzgtäeil Spark

[jira] [Created] (SPARK-44114) Upgrade built-in Hive to 3.x

Dongjoon Hyun created SPARK-44114: - Summary: Upgrade built-in Hive to 3.x Key: SPARK-44114 URL: https://issues.apache.org/jira/browse/SPARK-44114 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-44123) Clear persistent objects before SQL analyzer golden file tests

Daniel created SPARK-44123: -- Summary: Clear persistent objects before SQL analyzer golden file tests Key: SPARK-44123 URL: https://issues.apache.org/jira/browse/SPARK-44123 Project: Spark Issue

[jira] [Assigned] (SPARK-44012) KafkaDataConsumer to log some stats

[ https://issues.apache.org/jira/browse/SPARK-44012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-44012: Assignee: Siying Dong > KafkaDataConsumer to log some stats >

[jira] [Resolved] (SPARK-44012) KafkaDataConsumer to log some stats

[ https://issues.apache.org/jira/browse/SPARK-44012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-44012. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41525

[jira] [Updated] (SPARK-44126) Migration shuffle to decommissioned executor should not count as block failure

[ https://issues.apache.org/jira/browse/SPARK-44126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongwei Zhu updated SPARK-44126: - Description: When shuffle migration to decommissioned executor, the below exception is thrown:

[jira] [Resolved] (SPARK-44122) Make connect module pass except arrow-related ones in Java 21

[ https://issues.apache.org/jira/browse/SPARK-44122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44122. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 41679

[jira] [Assigned] (SPARK-43952) Cancel Spark jobs not only by a single "jobgroup", but allow multiple "job tags"

[ https://issues.apache.org/jira/browse/SPARK-43952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43952: Assignee: Juliusz Sompolski > Cancel Spark jobs not only by a single "jobgroup", but

[jira] [Updated] (SPARK-43952) Cancel Spark jobs not only by a single "jobgroup", but allow multiple "job tags"

[ https://issues.apache.org/jira/browse/SPARK-43952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43952: -- Fix Version/s: 3.5.0 (was: 4.0.0) > Cancel Spark jobs not only by a

[jira] [Resolved] (SPARK-43952) Cancel Spark jobs not only by a single "jobgroup", but allow multiple "job tags"

[ https://issues.apache.org/jira/browse/SPARK-43952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43952. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 41440

[jira] [Updated] (SPARK-44122) Make connect module pass except arrow-related ones in Java 21

[ https://issues.apache.org/jira/browse/SPARK-44122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44122: -- Fix Version/s: 3.5.0 (was: 4.0.0) > Make connect module pass except

[jira] [Assigned] (SPARK-44122) Make connect module pass except arrow-related ones in Java 21

[ https://issues.apache.org/jira/browse/SPARK-44122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44122: - Assignee: Dongjoon Hyun > Make connect module pass except arrow-related ones in Java

[jira] [Created] (SPARK-44113) Make Scala 2.13+ as default Scala version

Dongjoon Hyun created SPARK-44113: - Summary: Make Scala 2.13+ as default Scala version Key: SPARK-44113 URL: https://issues.apache.org/jira/browse/SPARK-44113 Project: Spark Issue Type:

[jira] [Updated] (SPARK-44114) Upgrade built-in Hive to 3+

[ https://issues.apache.org/jira/browse/SPARK-44114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44114: -- Summary: Upgrade built-in Hive to 3+ (was: Upgrade built-in Hive to 3.x) > Upgrade built-in

[jira] [Created] (SPARK-44119) Drop K8s v1.25 and lower version support

Dongjoon Hyun created SPARK-44119: - Summary: Drop K8s v1.25 and lower version support Key: SPARK-44119 URL: https://issues.apache.org/jira/browse/SPARK-44119 Project: Spark Issue Type:

[jira] [Commented] (SPARK-44101) Support pandas 2

[ https://issues.apache.org/jira/browse/SPARK-44101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735454#comment-17735454 ] Dongjoon Hyun commented on SPARK-44101: --- Thank you so much! > Support pandas 2 >

[jira] [Created] (SPARK-44122) Make connect module pass except arrow-related ones in Java 21

Dongjoon Hyun created SPARK-44122: - Summary: Make connect module pass except arrow-related ones in Java 21 Key: SPARK-44122 URL: https://issues.apache.org/jira/browse/SPARK-44122 Project: Spark

[jira] [Updated] (SPARK-44098) Introduce python breaking change detection

[ https://issues.apache.org/jira/browse/SPARK-44098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-44098: -- Description: We have breaking change detections for Binary Compatibility and Protobufs, but

[jira] [Created] (SPARK-44117) Spark Connect GA

Dongjoon Hyun created SPARK-44117: - Summary: Spark Connect GA Key: SPARK-44117 URL: https://issues.apache.org/jira/browse/SPARK-44117 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-44111) Prepare Apache Spark 4.0.0

[ https://issues.apache.org/jira/browse/SPARK-44111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44111: -- Description: For now, this issue aims to collect ideas for planning Apache Spark 4.0.0. We

[jira] [Updated] (SPARK-44111) Prepare Apache Spark 4.0.0

[ https://issues.apache.org/jira/browse/SPARK-44111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44111: -- Description: For now, this issue aims to collect ideas for planning Apache Spark 4.0.0. >

[jira] [Updated] (SPARK-44115) Upgrade to Apache ORC 2.0

[ https://issues.apache.org/jira/browse/SPARK-44115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44115: -- Description: Apache ORC community has the following release cycles which are synchronized

[jira] [Updated] (SPARK-44121) Reenable Arrow-based connect tests in Java 21

[ https://issues.apache.org/jira/browse/SPARK-44121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44121: -- Summary: Reenable Arrow-based connect tests in Java 21 (was: Renable Arrow-based connect

[jira] [Created] (SPARK-44126) Migration shuffle to decommissioned executor should not count as block failure

Zhongwei Zhu created SPARK-44126: Summary: Migration shuffle to decommissioned executor should not count as block failure Key: SPARK-44126 URL: https://issues.apache.org/jira/browse/SPARK-44126

[jira] [Commented] (SPARK-42941) Add support for streaming listener in Python

[ https://issues.apache.org/jira/browse/SPARK-42941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735505#comment-17735505 ] Hudson commented on SPARK-42941: User 'WweiL' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42941) Add support for streaming listener in Python

[ https://issues.apache.org/jira/browse/SPARK-42941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42941: Assignee: Wei Liu > Add support for streaming listener in Python >

[jira] [Resolved] (SPARK-42941) Add support for streaming listener in Python

[ https://issues.apache.org/jira/browse/SPARK-42941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42941. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41540

[jira] [Updated] (SPARK-44103) Make pending container jobs cancelable

[ https://issues.apache.org/jira/browse/SPARK-44103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44103: -- Fix Version/s: 3.5.0 (was: 4.0.0) > Make pending container jobs

[jira] [Assigned] (SPARK-44103) Make pending container jobs cancelable

[ https://issues.apache.org/jira/browse/SPARK-44103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-44103: - Assignee: Ruifeng Zheng > Make pending container jobs cancelable >

[jira] [Resolved] (SPARK-44103) Make pending container jobs cancelable

[ https://issues.apache.org/jira/browse/SPARK-44103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-44103. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 41668

[jira] [Created] (SPARK-44110) Forked unit tests don't receive proxy system properties

Steve Vaughan created SPARK-44110: - Summary: Forked unit tests don't receive proxy system properties Key: SPARK-44110 URL: https://issues.apache.org/jira/browse/SPARK-44110 Project: Spark

[jira] [Updated] (SPARK-43351) Support Golang in Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-43351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43351: -- Fix Version/s: 3.5.0 > Support Golang in Spark Connect > --- > >

[jira] [Commented] (SPARK-43351) Support Golang in Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-43351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735446#comment-17735446 ] Dongjoon Hyun commented on SPARK-43351: --- I updated the Fixed Version to 3.5.0. Please update it if

[jira] [Updated] (SPARK-44119) Drop K8s v1.25 and lower version support

[ https://issues.apache.org/jira/browse/SPARK-44119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44119: -- Description: EKS K8s v1.25 will reach the End-Of-Support on May 2024. > Drop K8s v1.25 and

[jira] [Created] (SPARK-44128) Upgrade netty from 4.1.92 to 4.1.94

BingKun Pan created SPARK-44128: --- Summary: Upgrade netty from 4.1.92 to 4.1.94 Key: SPARK-44128 URL: https://issues.apache.org/jira/browse/SPARK-44128 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-44128) Upgrade netty from 4.1.92 to 4.1.94

[ https://issues.apache.org/jira/browse/SPARK-44128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735500#comment-17735500 ] BingKun Pan commented on SPARK-44128: - I work on it. > Upgrade netty from 4.1.92 to 4.1.94 >

[jira] [Created] (SPARK-44129) Use "3.5.0" for `master` branch until creating `branch-3.5`

Dongjoon Hyun created SPARK-44129: - Summary: Use "3.5.0" for `master` branch until creating `branch-3.5` Key: SPARK-44129 URL: https://issues.apache.org/jira/browse/SPARK-44129 Project: Spark

[jira] [Resolved] (SPARK-44129) Use "3.5.0" for `master` branch until creating `branch-3.5`

[ https://issues.apache.org/jira/browse/SPARK-44129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44129. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 41682

[jira] [Assigned] (SPARK-44129) Use "3.5.0" for `master` branch until creating `branch-3.5`

[ https://issues.apache.org/jira/browse/SPARK-44129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44129: Assignee: Dongjoon Hyun > Use "3.5.0" for `master` branch until creating `branch-3.5` >

[jira] [Updated] (SPARK-44129) Use "3.5.0" for `master` branch until creating `branch-3.5`

[ https://issues.apache.org/jira/browse/SPARK-44129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-44129: - Fix Version/s: 3.5.0 (was: 4.0.0) > Use "3.5.0" for `master` branch

[jira] [Created] (SPARK-44118) Support K8s scheduling gates

Dongjoon Hyun created SPARK-44118: - Summary: Support K8s scheduling gates Key: SPARK-44118 URL: https://issues.apache.org/jira/browse/SPARK-44118 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-44112) Drop Java 8 Support

Dongjoon Hyun created SPARK-44112: - Summary: Drop Java 8 Support Key: SPARK-44112 URL: https://issues.apache.org/jira/browse/SPARK-44112 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-44121) Renable Arrow-based connect tests in Java 21

Dongjoon Hyun created SPARK-44121: - Summary: Renable Arrow-based connect tests in Java 21 Key: SPARK-44121 URL: https://issues.apache.org/jira/browse/SPARK-44121 Project: Spark Issue Type:

[jira] [Created] (SPARK-44130) Revert SPARK-44129 after branch-3.5

Dongjoon Hyun created SPARK-44130: - Summary: Revert SPARK-44129 after branch-3.5 Key: SPARK-44130 URL: https://issues.apache.org/jira/browse/SPARK-44130 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-44125) Support Java 21 in SparkR

[ https://issues.apache.org/jira/browse/SPARK-44125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44125: Assignee: Dongjoon Hyun > Support Java 21 in SparkR > - > >

[jira] [Commented] (SPARK-38230) InsertIntoHadoopFsRelationCommand unnecessarily fetches details of partitions in most cases

[ https://issues.apache.org/jira/browse/SPARK-38230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735519#comment-17735519 ] jeanlyn commented on SPARK-38230: - We found Hive metastore crash frequently after upgrade Spark from

[jira] [Assigned] (SPARK-44129) Use "3.5.0" for `master` branch until creating `branch-3.5`

[ https://issues.apache.org/jira/browse/SPARK-44129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44129: - Assignee: Hyukjin Kwon > Use "3.5.0" for `master` branch until creating `branch-3.5` >

[jira] [Resolved] (SPARK-44129) Use "3.5.0" for `master` branch until creating `branch-3.5`

[ https://issues.apache.org/jira/browse/SPARK-44129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44129. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41684

[jira] [Commented] (SPARK-44125) Support Java 21 in SparkR

[ https://issues.apache.org/jira/browse/SPARK-44125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735526#comment-17735526 ] Paul King commented on SPARK-44125: --- User 'dongjoon-hyun' has created a pull request for this issue:

[jira] [Commented] (SPARK-44125) Support Java 21 in SparkR

[ https://issues.apache.org/jira/browse/SPARK-44125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735527#comment-17735527 ] Paul King commented on SPARK-44125: --- User 'dongjoon-hyun' has created a pull request for this issue:

[jira] [Created] (SPARK-44131) Add call_function and deprecate call_udf for Scala API

jiaan.geng created SPARK-44131: -- Summary: Add call_function and deprecate call_udf for Scala API Key: SPARK-44131 URL: https://issues.apache.org/jira/browse/SPARK-44131 Project: Spark Issue

[jira] [Reopened] (SPARK-44129) Use "3.5.0" for `master` branch until creating `branch-3.5`

[ https://issues.apache.org/jira/browse/SPARK-44129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-44129: -- Assignee: (was: Dongjoon Hyun) Reverted in

[jira] [Updated] (SPARK-44129) Use "3.5.0" for `master` branch until creating `branch-3.5`

[ https://issues.apache.org/jira/browse/SPARK-44129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-44129: - Fix Version/s: (was: 3.5.0) > Use "3.5.0" for `master` branch until creating `branch-3.5` >

[jira] [Updated] (SPARK-44107) Hide unsupported Column methods from auto-completion

[ https://issues.apache.org/jira/browse/SPARK-44107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-44107: -- Fix Version/s: 3.5.0 > Hide unsupported Column methods from auto-completion >

[jira] [Assigned] (SPARK-44107) Hide unsupported Column methods from auto-completion

[ https://issues.apache.org/jira/browse/SPARK-44107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-44107: - Assignee: Ruifeng Zheng > Hide unsupported Column methods from auto-completion >

[jira] (SPARK-43929) Add date time functions to Scala and Python - part 1

[ https://issues.apache.org/jira/browse/SPARK-43929 ] jiaan.geng deleted comment on SPARK-43929: was (Author: beliefer): I will take over this one. > Add date time functions to Scala and Python - part 1 >

[jira] [Resolved] (SPARK-44125) Support Java 21 in SparkR

[ https://issues.apache.org/jira/browse/SPARK-44125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44125. --- Fix Version/s: 3.5.0 Resolution: Fixed > Support Java 21 in SparkR >

[jira] [Updated] (SPARK-44102) Unable to create spark session when master URL set to k8s in spark 3.3.0

[ https://issues.apache.org/jira/browse/SPARK-44102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shamim updated SPARK-44102: --- Description: Getting below error while creating Spark Session using JAVA    Error in executing job: 

[jira] [Updated] (SPARK-44104) Enabled protobuf module mima check for Spark 3.5.0

[ https://issues.apache.org/jira/browse/SPARK-44104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-44104: - Summary: Enabled protobuf module mima check for Spark 3.5.0 (was: Enable protobuf module mima check

[jira] [Created] (SPARK-44104) Enable protobuf module mima check for Spark 3.5.0

Yang Jie created SPARK-44104: Summary: Enable protobuf module mima check for Spark 3.5.0 Key: SPARK-44104 URL: https://issues.apache.org/jira/browse/SPARK-44104 Project: Spark Issue Type:

[jira] [Updated] (SPARK-44105) `LastNonNull` should be lazily resolved

[ https://issues.apache.org/jira/browse/SPARK-44105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-44105: -- Component/s: SQL > `LastNonNull` should be lazily resolved >

[jira] [Created] (SPARK-44106) Add `__repr__` for `GroupedData`

Ruifeng Zheng created SPARK-44106: - Summary: Add `__repr__` for `GroupedData` Key: SPARK-44106 URL: https://issues.apache.org/jira/browse/SPARK-44106 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-44100) Move namespace from `pyspark.mlv2` to `pyspark.ml.connect`

[ https://issues.apache.org/jira/browse/SPARK-44100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu resolved SPARK-44100. Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41666

[jira] [Created] (SPARK-44103) Make pending container jobs cancelable

Ruifeng Zheng created SPARK-44103: - Summary: Make pending container jobs cancelable Key: SPARK-44103 URL: https://issues.apache.org/jira/browse/SPARK-44103 Project: Spark Issue Type: Test

[jira] [Created] (SPARK-44105) `LastNonNull` should be lazily resolved

Ruifeng Zheng created SPARK-44105: - Summary: `LastNonNull` should be lazily resolved Key: SPARK-44105 URL: https://issues.apache.org/jira/browse/SPARK-44105 Project: Spark Issue Type:

[jira] [Commented] (SPARK-44105) `LastNonNull` should be lazily resolved

[ https://issues.apache.org/jira/browse/SPARK-44105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735196#comment-17735196 ] ASF GitHub Bot commented on SPARK-44105: User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-44079) Json reader crashes when a different schema is present

[ https://issues.apache.org/jira/browse/SPARK-44079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735199#comment-17735199 ] ASF GitHub Bot commented on SPARK-44079: User 'Hisoka-X' has created a pull request for this

[jira] [Assigned] (SPARK-43939) Add try_* functions to Scala and Python

[ https://issues.apache.org/jira/browse/SPARK-43939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43939: - Assignee: BingKun Pan > Add try_* functions to Scala and Python >

[jira] [Resolved] (SPARK-43939) Add try_* functions to Scala and Python

[ https://issues.apache.org/jira/browse/SPARK-43939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43939. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41653

[jira] [Resolved] (SPARK-43876) Enable fast hashmap for distinct queries

[ https://issues.apache.org/jira/browse/SPARK-43876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43876. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41374

[jira] [Assigned] (SPARK-43876) Enable fast hashmap for distinct queries

[ https://issues.apache.org/jira/browse/SPARK-43876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43876: - Assignee: Wan Kun > Enable fast hashmap for distinct queries >

[jira] [Resolved] (SPARK-44105) `LastNonNull` should be lazily resolved

[ https://issues.apache.org/jira/browse/SPARK-44105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44105. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41672

[jira] [Assigned] (SPARK-44105) `LastNonNull` should be lazily resolved

[ https://issues.apache.org/jira/browse/SPARK-44105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44105: - Assignee: Ruifeng Zheng > `LastNonNull` should be lazily resolved >

[jira] [Created] (SPARK-44108) Cannot parse Type from german "umlaut"

Jorge Machado created SPARK-44108: - Summary: Cannot parse Type from german "umlaut" Key: SPARK-44108 URL: https://issues.apache.org/jira/browse/SPARK-44108 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-44022) Enforce max bytecode version on Maven dependencies

[ https://issues.apache.org/jira/browse/SPARK-44022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44022. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41546

[jira] [Assigned] (SPARK-44022) Enforce max bytecode version on Maven dependencies

[ https://issues.apache.org/jira/browse/SPARK-44022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44022: - Assignee: Bowen Liang > Enforce max bytecode version on Maven dependencies >

[jira] [Updated] (SPARK-44022) Enforce max bytecode version on Maven dependencies

[ https://issues.apache.org/jira/browse/SPARK-44022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44022: -- Affects Version/s: 3.5.0 (was: 3.4.0) > Enforce max bytecode

[jira] [Commented] (SPARK-43891) Support SHOW VIEWS IN . when not is not the current selected catalog

[ https://issues.apache.org/jira/browse/SPARK-43891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17735348#comment-17735348 ] Rui Wang commented on SPARK-43891: -- Sorry I am bit busy recently and will follow up on this ticket. >

[jira] [Created] (SPARK-44107) Hide unsupported Column methods from auto-completion

Ruifeng Zheng created SPARK-44107: - Summary: Hide unsupported Column methods from auto-completion Key: SPARK-44107 URL: https://issues.apache.org/jira/browse/SPARK-44107 Project: Spark Issue

[jira] [Resolved] (SPARK-44104) Enabled protobuf module mima check for Spark 3.5.0

[ https://issues.apache.org/jira/browse/SPARK-44104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44104. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41671

[jira] [Assigned] (SPARK-44104) Enabled protobuf module mima check for Spark 3.5.0

[ https://issues.apache.org/jira/browse/SPARK-44104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44104: - Assignee: Yang Jie > Enabled protobuf module mima check for Spark 3.5.0 >

[jira] [Created] (SPARK-44109) Remove duplicate preferred locations of each RDD partition

Wan Kun created SPARK-44109: --- Summary: Remove duplicate preferred locations of each RDD partition Key: SPARK-44109 URL: https://issues.apache.org/jira/browse/SPARK-44109 Project: Spark Issue Type:

[jira] [Updated] (SPARK-44108) Cannot parse Type from german "umlaut"

[ https://issues.apache.org/jira/browse/SPARK-44108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jorge Machado updated SPARK-44108: -- Priority: Minor (was: Critical) > Cannot parse Type from german "umlaut" >