[jira] [Resolved] (SPARK-49246) TableCatalog#loadTable should indicate if it's for writing

[ https://issues.apache.org/jira/browse/SPARK-49246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-49246. - Fix Version/s: 4.0.0 3.5.3 Resolution: Fixed Issue resolved by pull re

[jira] [Updated] (SPARK-49332) Add K8s service for Workers to SparkClusterResourceSpec

[ https://issues.apache.org/jira/browse/SPARK-49332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49332: --- Labels: pull-request-available (was: ) > Add K8s service for Workers to SparkClusterResourc

[jira] [Updated] (SPARK-49332) Add K8s service for Workers to SparkClusterResourceSpec

[ https://issues.apache.org/jira/browse/SPARK-49332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49332: -- Summary: Add K8s service for Workers to SparkClusterResourceSpec (was: Add K8s service for Wo

[jira] [Created] (SPARK-49332) Add K8s service for Workers

Dongjoon Hyun created SPARK-49332: - Summary: Add K8s service for Workers Key: SPARK-49332 URL: https://issues.apache.org/jira/browse/SPARK-49332 Project: Spark Issue Type: Bug Compo

[jira] [Updated] (SPARK-49331) Use Minikube Github Action

[ https://issues.apache.org/jira/browse/SPARK-49331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49331: --- Labels: pull-request-available (was: ) > Use Minikube Github Action > -

[jira] [Updated] (SPARK-49331) Use `setup-minikube` GitHub Action

[ https://issues.apache.org/jira/browse/SPARK-49331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49331: -- Summary: Use `setup-minikube` GitHub Action (was: Use Minikube Github Action) > Use `setup-m

[jira] [Resolved] (SPARK-49331) Use Minikube Github Action

[ https://issues.apache.org/jira/browse/SPARK-49331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49331. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull req

[jira] [Created] (SPARK-49331) Use Minikube Github Action

Qi Tan created SPARK-49331: -- Summary: Use Minikube Github Action Key: SPARK-49331 URL: https://issues.apache.org/jira/browse/SPARK-49331 Project: Spark Issue Type: Sub-task Components: Kub

[jira] [Assigned] (SPARK-49330) Revise `InstanceConfig` to `ExecutorInstanceConfig` class

[ https://issues.apache.org/jira/browse/SPARK-49330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49330: - Assignee: Dongjoon Hyun > Revise `InstanceConfig` to `ExecutorInstanceConfig` class > -

[jira] [Updated] (SPARK-49330) Revise `InstanceConfig` to `ExecutorInstanceConfig` class

[ https://issues.apache.org/jira/browse/SPARK-49330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49330: --- Labels: pull-request-available (was: ) > Revise `InstanceConfig` to `ExecutorInstanceConfig

[jira] [Created] (SPARK-49330) Revise `InstanceConfig` to `ExecutorInstanceConfig` class

Dongjoon Hyun created SPARK-49330: - Summary: Revise `InstanceConfig` to `ExecutorInstanceConfig` class Key: SPARK-49330 URL: https://issues.apache.org/jira/browse/SPARK-49330 Project: Spark I

[jira] [Assigned] (SPARK-49214) Add e2e test in operator workflow

[ https://issues.apache.org/jira/browse/SPARK-49214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49214: - Assignee: Qi Tan > Add e2e test in operator workflow >

[jira] [Assigned] (SPARK-49329) Support user provided spec for SparkCluster

[ https://issues.apache.org/jira/browse/SPARK-49329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49329: - Assignee: Zhou JIANG > Support user provided spec for SparkCluster > --

[jira] [Assigned] (SPARK-49328) Propagate Spark configurations to SparkCluster

[ https://issues.apache.org/jira/browse/SPARK-49328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49328: - Assignee: Dongjoon Hyun > Propagate Spark configurations to SparkCluster >

[jira] [Resolved] (SPARK-49328) Propagate Spark configurations to SparkCluster

[ https://issues.apache.org/jira/browse/SPARK-49328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49328. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull req

[jira] [Assigned] (SPARK-49300) Fix Hadoop delegation token leak when tokenRenewalInterval is not set.

[ https://issues.apache.org/jira/browse/SPARK-49300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-49300: Assignee: Shuyan Zhang > Fix Hadoop delegation token leak when tokenRenewalInterval is not set. >

[jira] [Resolved] (SPARK-49300) Fix Hadoop delegation token leak when tokenRenewalInterval is not set.

[ https://issues.apache.org/jira/browse/SPARK-49300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-49300. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47800 [https://github.com

[jira] [Updated] (SPARK-49329) Support user provided spec for SparkCluster

[ https://issues.apache.org/jira/browse/SPARK-49329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49329: --- Labels: pull-request-available (was: ) > Support user provided spec for SparkCluster >

[jira] [Created] (SPARK-49329) Support user provided spec for SparkCluster

Zhou JIANG created SPARK-49329: -- Summary: Support user provided spec for SparkCluster Key: SPARK-49329 URL: https://issues.apache.org/jira/browse/SPARK-49329 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-49328) Propagate Spark configurations to SparkCluster

[ https://issues.apache.org/jira/browse/SPARK-49328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49328: --- Labels: pull-request-available (was: ) > Propagate Spark configurations to SparkCluster > -

[jira] [Created] (SPARK-49328) Propagate Spark configurations to SparkCluster

Dongjoon Hyun created SPARK-49328: - Summary: Propagate Spark configurations to SparkCluster Key: SPARK-49328 URL: https://issues.apache.org/jira/browse/SPARK-49328 Project: Spark Issue Type:

[jira] [Updated] (SPARK-49327) Upgrade commons-compress to 1.27.1

[ https://issues.apache.org/jira/browse/SPARK-49327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49327: --- Labels: pull-request-available (was: ) > Upgrade commons-compress to 1.27.1 > -

[jira] [Created] (SPARK-49327) Upgrade commons-compress to 1.27.1

BingKun Pan created SPARK-49327: --- Summary: Upgrade commons-compress to 1.27.1 Key: SPARK-49327 URL: https://issues.apache.org/jira/browse/SPARK-49327 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-49026) Create conversions Column API to Connect protos

[ https://issues.apache.org/jira/browse/SPARK-49026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49026. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47812 [https://gi

[jira] [Resolved] (SPARK-49274) Support java serialization in AgnosticEncoders

[ https://issues.apache.org/jira/browse/SPARK-49274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49274. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47806 [https://gi

[jira] [Resolved] (SPARK-49269) Improve performance and memory footprint of all VALUES clauses

[ https://issues.apache.org/jira/browse/SPARK-49269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-49269. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47791 [https://gi

[jira] [Assigned] (SPARK-49269) Improve performance and memory footprint of all VALUES clauses

[ https://issues.apache.org/jira/browse/SPARK-49269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-49269: Assignee: Costas Zarifis > Improve performance and memory footprint of all VALUES clauses

[jira] [Created] (SPARK-49326) Classify Error class for Foreach sink user function error

Jing Zhan created SPARK-49326: - Summary: Classify Error class for Foreach sink user function error Key: SPARK-49326 URL: https://issues.apache.org/jira/browse/SPARK-49326 Project: Spark Issue Ty

[jira] [Updated] (SPARK-49218) Caching observed dataframes blocks metric retrieval when the dataframe is empty

[ https://issues.apache.org/jira/browse/SPARK-49218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Payson updated SPARK-49218: --- Component/s: Spark Core > Caching observed dataframes blocks metric retrieval when the dataframe is

[jira] [Commented] (SPARK-49044) Improve error message in ValidateExternalType

[ https://issues.apache.org/jira/browse/SPARK-49044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17875296#comment-17875296 ] Mich Talebzadeh commented on SPARK-49044: - My be improving description to add mo

[jira] [Resolved] (SPARK-49325) Add `SparkCluster` to `spark-operator` module

[ https://issues.apache.org/jira/browse/SPARK-49325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49325. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull req

[jira] [Assigned] (SPARK-49325) Add `SparkCluster` to `spark-operator` module

[ https://issues.apache.org/jira/browse/SPARK-49325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49325: - Assignee: Dongjoon Hyun > Add `SparkCluster` to `spark-operator` module > -

[jira] [Updated] (SPARK-49318) Preempt low priority LCA internal error until the end of check analysis

[ https://issues.apache.org/jira/browse/SPARK-49318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49318: --- Labels: pull-request-available (was: ) > Preempt low priority LCA internal error until the

[jira] [Updated] (SPARK-49323) Move MockObserver from Spark Connect Server's test folder to the Server's main folder

[ https://issues.apache.org/jira/browse/SPARK-49323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49323: --- Labels: pull-request-available (was: ) > Move MockObserver from Spark Connect Server's tes

[jira] [Updated] (SPARK-49325) Add `SparkCluster` to `spark-operator` module

[ https://issues.apache.org/jira/browse/SPARK-49325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49325: --- Labels: pull-request-available (was: ) > Add `SparkCluster` to `spark-operator` module > --

[jira] [Created] (SPARK-49325) Add `SparkCluster` to `spark-operator` module

Dongjoon Hyun created SPARK-49325: - Summary: Add `SparkCluster` to `spark-operator` module Key: SPARK-49325 URL: https://issues.apache.org/jira/browse/SPARK-49325 Project: Spark Issue Type: S

[jira] [Resolved] (SPARK-49322) Add `SparkCluster` to `spark-submission-worker` module

[ https://issues.apache.org/jira/browse/SPARK-49322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49322. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull req

[jira] [Updated] (SPARK-49306) Create SQL function aliases for 'zeroifnull' and 'nullifzero'

[ https://issues.apache.org/jira/browse/SPARK-49306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49306: --- Labels: pull-request-available (was: ) > Create SQL function aliases for 'zeroifnull' and '

[jira] [Updated] (SPARK-49214) Add e2e test in operator workflow

[ https://issues.apache.org/jira/browse/SPARK-49214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49214: --- Labels: pull-request-available (was: ) > Add e2e test in operator workflow > --

[jira] [Created] (SPARK-49324) E2E Test Components

Qi Tan created SPARK-49324: -- Summary: E2E Test Components Key: SPARK-49324 URL: https://issues.apache.org/jira/browse/SPARK-49324 Project: Spark Issue Type: Sub-task Components: Kubernetes

[jira] [Created] (SPARK-49323) Move MockObserver from Spark Connect Server's test folder to the Server's main folder

Thang Long Vu created SPARK-49323: - Summary: Move MockObserver from Spark Connect Server's test folder to the Server's main folder Key: SPARK-49323 URL: https://issues.apache.org/jira/browse/SPARK-49323

[jira] [Assigned] (SPARK-49322) Add `SparkCluster` to `spark-submission-worker` module

[ https://issues.apache.org/jira/browse/SPARK-49322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49322: - Assignee: Dongjoon Hyun > Add `SparkCluster` to `spark-submission-worker` module >

[jira] [Updated] (SPARK-49322) Add `SparkCluster` to `spark-submission-worker` module

[ https://issues.apache.org/jira/browse/SPARK-49322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49322: --- Labels: pull-request-available (was: ) > Add `SparkCluster` to `spark-submission-worker` mo

[jira] [Created] (SPARK-49322) Add `SparkCluster` to `spark-submission-worker` module

Dongjoon Hyun created SPARK-49322: - Summary: Add `SparkCluster` to `spark-submission-worker` module Key: SPARK-49322 URL: https://issues.apache.org/jira/browse/SPARK-49322 Project: Spark Issu

[jira] [Resolved] (SPARK-49319) Add `SparkCluster` to `spark-operator-api` module and examples

[ https://issues.apache.org/jira/browse/SPARK-49319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49319. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull req

[jira] [Updated] (SPARK-45502) Upgrade Kafka to 3.6.1

[ https://issues.apache.org/jira/browse/SPARK-45502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45502: -- Parent Issue: SPARK-44111 (was: SPARK-47046) > Upgrade Kafka to 3.6.1 > -

[jira] [Updated] (SPARK-46892) Upgrade dropwizard metrics 4.2.25

[ https://issues.apache.org/jira/browse/SPARK-46892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46892: -- Parent: SPARK-47046 Issue Type: Sub-task (was: Improvement) > Upgrade dropwizard metr

[jira] [Updated] (SPARK-49292) Upgrade dropwizard metrics to 4.2.27

[ https://issues.apache.org/jira/browse/SPARK-49292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49292: -- Parent: SPARK-47046 Issue Type: Sub-task (was: Improvement) > Upgrade dropwizard metr

[jira] [Resolved] (SPARK-49292) Upgrade dropwizard metrics to 4.2.27

[ https://issues.apache.org/jira/browse/SPARK-49292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49292. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47799 [https://

[jira] [Updated] (SPARK-49321) Additional JDBC dialects not registered in executor

[ https://issues.apache.org/jira/browse/SPARK-49321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49321: --- Labels: pull-request-available (was: ) > Additional JDBC dialects not registered in executo

[jira] [Created] (SPARK-49321) Additional JDBC dialects not registered in executor

Johnny Walker created SPARK-49321: - Summary: Additional JDBC dialects not registered in executor Key: SPARK-49321 URL: https://issues.apache.org/jira/browse/SPARK-49321 Project: Spark Issue T

[jira] [Created] (SPARK-49320) Add tag-related APs to Spark Core

Pengfei Xu created SPARK-49320: -- Summary: Add tag-related APs to Spark Core Key: SPARK-49320 URL: https://issues.apache.org/jira/browse/SPARK-49320 Project: Spark Issue Type: Bug Compo

[jira] [Updated] (SPARK-49017) Insert statement fails when multiple parameters are being used

[ https://issues.apache.org/jira/browse/SPARK-49017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-49017: - Affects Version/s: 3.5.3 > Insert statement fails when multiple parameters are being used >

[jira] [Assigned] (SPARK-49043) Fix group by on map containing collated strings

[ https://issues.apache.org/jira/browse/SPARK-49043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-49043: Assignee: Marko Ilic > Fix group by on map containing collated strings > ---

[jira] [Resolved] (SPARK-49043) Fix group by on map containing collated strings

[ https://issues.apache.org/jira/browse/SPARK-49043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-49043. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47521 [https://github.com

[jira] [Updated] (SPARK-49043) Fix group by on map containing collated strings

[ https://issues.apache.org/jira/browse/SPARK-49043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marko Ilic updated SPARK-49043: --- Affects Version/s: 4.0.0 (was: 3.4.3) > Fix group by on map containing co

[jira] [Assigned] (SPARK-49312) Improve error message for assertSchemaEqual

[ https://issues.apache.org/jira/browse/SPARK-49312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee reassigned SPARK-49312: --- Assignee: Haejoon Lee > Improve error message for assertSchemaEqual > -

[jira] [Resolved] (SPARK-49312) Improve error message for assertSchemaEqual

[ https://issues.apache.org/jira/browse/SPARK-49312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-49312. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47811 [https://gith

[jira] [Resolved] (SPARK-49267) Improve collation support testing - unit tests for StringTranslate

[ https://issues.apache.org/jira/browse/SPARK-49267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-49267. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47729 [https://github.com

[jira] [Assigned] (SPARK-49267) Improve collation support testing - unit tests for StringTranslate

[ https://issues.apache.org/jira/browse/SPARK-49267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-49267: Assignee: Uroš Bojanić > Improve collation support testing - unit tests for StringTranslate > ---

[jira] [Updated] (SPARK-49267) Improve collation support testing - unit tests for StringTranslate

[ https://issues.apache.org/jira/browse/SPARK-49267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49267: --- Labels: pull-request-available (was: ) > Improve collation support testing - unit tests for

[jira] [Assigned] (SPARK-49319) Add `SparkCluster` to `spark-operator-api` module and examples

[ https://issues.apache.org/jira/browse/SPARK-49319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49319: - Assignee: Dongjoon Hyun > Add `SparkCluster` to `spark-operator-api` module and example