[jira] [Updated] (SPARK-37023) Avoid fetching merge status when shuffleMergeEnabled is false for a shuffleDependency during retry

[ https://issues.apache.org/jira/browse/SPARK-37023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37023: -- Affects Version/s: (was: 3.2.0) 3.3.0 > Avoid fetching merge

[jira] [Updated] (SPARK-37023) Avoid fetching merge status when shuffleMergeEnabled is false for a shuffleDependency during retry

[ https://issues.apache.org/jira/browse/SPARK-37023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37023: -- Affects Version/s: (was: 3.3.0) 3.2.0 > Avoid fetching merge

[jira] [Commented] (SPARK-37206) Upgrade Avro to 1.11.0

[ https://issues.apache.org/jira/browse/SPARK-37206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438472#comment-17438472 ] Apache Spark commented on SPARK-37206: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37206) Upgrade Avro to 1.11.0

[ https://issues.apache.org/jira/browse/SPARK-37206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37206: Assignee: Kousuke Saruta (was: Apache Spark) > Upgrade Avro to 1.11.0 >

[jira] [Commented] (SPARK-37206) Upgrade Avro to 1.11.0

[ https://issues.apache.org/jira/browse/SPARK-37206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438473#comment-17438473 ] Apache Spark commented on SPARK-37206: -- User 'sarutak' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37206) Upgrade Avro to 1.11.0

[ https://issues.apache.org/jira/browse/SPARK-37206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37206: Assignee: Apache Spark (was: Kousuke Saruta) > Upgrade Avro to 1.11.0 >

[jira] [Created] (SPARK-37206) Upgrade Avro to 1.11.0

Kousuke Saruta created SPARK-37206: -- Summary: Upgrade Avro to 1.11.0 Key: SPARK-37206 URL: https://issues.apache.org/jira/browse/SPARK-37206 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-37108) Expose make_date expression in R

[ https://issues.apache.org/jira/browse/SPARK-37108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta resolved SPARK-37108. Fix Version/s: 3.3.0 Assignee: Leona Yoda Resolution: Fixed Issue

[jira] [Resolved] (SPARK-37054) Porting "pandas API on Spark: Internals" to PySpark docs.

[ https://issues.apache.org/jira/browse/SPARK-37054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-37054. - Resolution: Won't Do Won't do for now. > Porting "pandas API on Spark: Internals" to PySpark

[jira] [Commented] (SPARK-36989) Migrate type hint data tests

[ https://issues.apache.org/jira/browse/SPARK-36989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438432#comment-17438432 ] Apache Spark commented on SPARK-36989: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-37108) Expose make_date expression in R

[ https://issues.apache.org/jira/browse/SPARK-37108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438389#comment-17438389 ] Apache Spark commented on SPARK-37108: -- User 'yoda-mon' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37108) Expose make_date expression in R

[ https://issues.apache.org/jira/browse/SPARK-37108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37108: Assignee: Apache Spark > Expose make_date expression in R >

[jira] [Commented] (SPARK-37108) Expose make_date expression in R

[ https://issues.apache.org/jira/browse/SPARK-37108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438386#comment-17438386 ] Apache Spark commented on SPARK-37108: -- User 'yoda-mon' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37108) Expose make_date expression in R

[ https://issues.apache.org/jira/browse/SPARK-37108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37108: Assignee: (was: Apache Spark) > Expose make_date expression in R >

[jira] [Updated] (SPARK-37205) Support mapreduce.job.send-token-conf when starting containers in YARN

[ https://issues.apache.org/jira/browse/SPARK-37205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-37205: - Description: {{mapreduce.job.send-token-conf}} is a useful feature in Hadoop (see

[jira] [Created] (SPARK-37205) Support mapreduce.job.send-token-conf when starting containers in YARN

Chao Sun created SPARK-37205: Summary: Support mapreduce.job.send-token-conf when starting containers in YARN Key: SPARK-37205 URL: https://issues.apache.org/jira/browse/SPARK-37205 Project: Spark

[jira] [Resolved] (SPARK-36566) Add Spark appname as a label to the executor pods

[ https://issues.apache.org/jira/browse/SPARK-36566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-36566. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34460

[jira] [Assigned] (SPARK-36566) Add Spark appname as a label to the executor pods

[ https://issues.apache.org/jira/browse/SPARK-36566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36566: - Assignee: Yikun Jiang (was: Apache Spark) > Add Spark appname as a label to the

[jira] [Assigned] (SPARK-36566) Add Spark appname as a label to the executor pods

[ https://issues.apache.org/jira/browse/SPARK-36566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-36566: - Assignee: Apache Spark > Add Spark appname as a label to the executor pods >

[jira] [Updated] (SPARK-37060) Report driver status does not handle response from backup masters

[ https://issues.apache.org/jira/browse/SPARK-37060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohamadreza Rostami updated SPARK-37060: Priority: Critical (was: Major) > Report driver status does not handle response

[jira] [Updated] (SPARK-37202) Temp view didn't collect temp function that registered with catalog API

[ https://issues.apache.org/jira/browse/SPARK-37202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-37202: --- Issue Type: Bug (was: Task) > Temp view didn't collect temp function that registered with catalog

[jira] [Commented] (SPARK-37149) Improve error messages for arithmetic overflow under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-37149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438194#comment-17438194 ] Apache Spark commented on SPARK-37149: -- User 'allisonwang-db' has created a pull request for this

[jira] [Commented] (SPARK-37149) Improve error messages for arithmetic overflow under ANSI mode

[ https://issues.apache.org/jira/browse/SPARK-37149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438192#comment-17438192 ] Apache Spark commented on SPARK-37149: -- User 'allisonwang-db' has created a pull request for this

[jira] [Commented] (SPARK-26365) spark-submit for k8s cluster doesn't propagate exit code

[ https://issues.apache.org/jira/browse/SPARK-26365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438174#comment-17438174 ] Naresh commented on SPARK-26365: Yes. Its not fixed in 3.x yet. I am using spark 3.2 and still see the

[jira] [Updated] (SPARK-37201) Spark SQL reads unnecessary nested fields (filter after explode)

[ https://issues.apache.org/jira/browse/SPARK-37201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Kotlov updated SPARK-37201: -- Description: In this example, reading unnecessary nested fields still happens. Data

[jira] [Commented] (SPARK-37077) Annotations for pyspark.sql.context.SQLContext.createDataFrame are broken

[ https://issues.apache.org/jira/browse/SPARK-37077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438147#comment-17438147 ] Apache Spark commented on SPARK-37077: -- User 'zero323' has created a pull request for this issue:

[jira] [Updated] (SPARK-37201) Spark SQL reads unnecessary nested fields (filter after explode)

[ https://issues.apache.org/jira/browse/SPARK-37201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Kotlov updated SPARK-37201: -- Description: In this example, reading unnecessary nested fields still happens. Data

[jira] [Commented] (SPARK-37077) Annotations for pyspark.sql.context.SQLContext.createDataFrame are broken

[ https://issues.apache.org/jira/browse/SPARK-37077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438149#comment-17438149 ] Apache Spark commented on SPARK-37077: -- User 'zero323' has created a pull request for this issue:

[jira] [Commented] (SPARK-37077) Annotations for pyspark.sql.context.SQLContext.createDataFrame are broken

[ https://issues.apache.org/jira/browse/SPARK-37077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438146#comment-17438146 ] Apache Spark commented on SPARK-37077: -- User 'zero323' has created a pull request for this issue:

[jira] [Commented] (SPARK-36894) RDD.toDF should be synchronized with dispatched variants of SparkSession.createDataFrame

[ https://issues.apache.org/jira/browse/SPARK-36894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438142#comment-17438142 ] Apache Spark commented on SPARK-36894: -- User 'zero323' has created a pull request for this issue:

[jira] [Commented] (SPARK-36894) RDD.toDF should be synchronized with dispatched variants of SparkSession.createDataFrame

[ https://issues.apache.org/jira/browse/SPARK-36894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438141#comment-17438141 ] Apache Spark commented on SPARK-36894: -- User 'zero323' has created a pull request for this issue:

[jira] [Commented] (SPARK-37204) Update Apache Parent POM version to 24 in the pom.xml

[ https://issues.apache.org/jira/browse/SPARK-37204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438131#comment-17438131 ] Janardhan Pulivarthi commented on SPARK-37204: -- Hi, I am new here. Can I work on this? >

[jira] [Created] (SPARK-37204) Update Apache Parent POM version to 24 in the pom.xml

Janardhan Pulivarthi created SPARK-37204: Summary: Update Apache Parent POM version to 24 in the pom.xml Key: SPARK-37204 URL: https://issues.apache.org/jira/browse/SPARK-37204 Project: Spark

[jira] [Updated] (SPARK-37204) Update Apache Parent POM version to 24 in the pom.xml

[ https://issues.apache.org/jira/browse/SPARK-37204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janardhan Pulivarthi updated SPARK-37204: - Priority: Minor (was: Major) > Update Apache Parent POM version to 24 in the

[jira] [Updated] (SPARK-37201) Spark SQL reads unnecessary nested fields (filter after explode)

[ https://issues.apache.org/jira/browse/SPARK-37201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Kotlov updated SPARK-37201: -- Description: In this example, reading unnecessary nested fields still happens. Data

[jira] [Commented] (SPARK-37077) Annotations for pyspark.sql.context.SQLContext.createDataFrame are broken

[ https://issues.apache.org/jira/browse/SPARK-37077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438089#comment-17438089 ] Apache Spark commented on SPARK-37077: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-37077) Annotations for pyspark.sql.context.SQLContext.createDataFrame are broken

[ https://issues.apache.org/jira/browse/SPARK-37077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438087#comment-17438087 ] Apache Spark commented on SPARK-37077: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-36894) RDD.toDF should be synchronized with dispatched variants of SparkSession.createDataFrame

[ https://issues.apache.org/jira/browse/SPARK-36894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438085#comment-17438085 ] Apache Spark commented on SPARK-36894: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-37077) Annotations for pyspark.sql.context.SQLContext.createDataFrame are broken

[ https://issues.apache.org/jira/browse/SPARK-37077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438086#comment-17438086 ] Apache Spark commented on SPARK-37077: -- User 'sarutak' has created a pull request for this issue:

[jira] [Commented] (SPARK-36894) RDD.toDF should be synchronized with dispatched variants of SparkSession.createDataFrame

[ https://issues.apache.org/jira/browse/SPARK-36894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438084#comment-17438084 ] Apache Spark commented on SPARK-36894: -- User 'sarutak' has created a pull request for this issue:

[jira] [Resolved] (SPARK-37030) Maven build failed in windows!

[ https://issues.apache.org/jira/browse/SPARK-37030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shockang resolved SPARK-37030. -- Resolution: Done > Maven build failed in windows! > -- > >

[jira] [Commented] (SPARK-37030) Maven build failed in windows!

[ https://issues.apache.org/jira/browse/SPARK-37030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438017#comment-17438017 ] Shockang commented on SPARK-37030: -- [~hyukjin.kwon] Thank you for your suggestion. This problem has

[jira] [Resolved] (SPARK-37077) Annotations for pyspark.sql.context.SQLContext.createDataFrame are broken

[ https://issues.apache.org/jira/browse/SPARK-37077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-37077. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Resolved] (SPARK-36894) RDD.toDF should be synchronized with dispatched variants of SparkSession.createDataFrame

[ https://issues.apache.org/jira/browse/SPARK-36894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-36894. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-37077) Annotations for pyspark.sql.context.SQLContext.createDataFrame are broken

[ https://issues.apache.org/jira/browse/SPARK-37077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-37077: -- Assignee: Maciej Szymkiewicz > Annotations for

[jira] [Assigned] (SPARK-36894) RDD.toDF should be synchronized with dispatched variants of SparkSession.createDataFrame

[ https://issues.apache.org/jira/browse/SPARK-36894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-36894: -- Assignee: Maciej Szymkiewicz > RDD.toDF should be synchronized with

[jira] [Commented] (SPARK-37195) Unify v1 and v2 SHOW TBLPROPERTIES tests

[ https://issues.apache.org/jira/browse/SPARK-37195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437871#comment-17437871 ] Apache Spark commented on SPARK-37195: -- User 'Peng-Lei' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37195) Unify v1 and v2 SHOW TBLPROPERTIES tests

[ https://issues.apache.org/jira/browse/SPARK-37195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37195: Assignee: Apache Spark > Unify v1 and v2 SHOW TBLPROPERTIES tests >

[jira] [Assigned] (SPARK-37195) Unify v1 and v2 SHOW TBLPROPERTIES tests

[ https://issues.apache.org/jira/browse/SPARK-37195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37195: Assignee: (was: Apache Spark) > Unify v1 and v2 SHOW TBLPROPERTIES tests >

[jira] [Commented] (SPARK-37195) Unify v1 and v2 SHOW TBLPROPERTIES tests

[ https://issues.apache.org/jira/browse/SPARK-37195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437869#comment-17437869 ] Apache Spark commented on SPARK-37195: -- User 'Peng-Lei' has created a pull request for this issue:

[jira] [Commented] (SPARK-26365) spark-submit for k8s cluster doesn't propagate exit code

[ https://issues.apache.org/jira/browse/SPARK-26365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437830#comment-17437830 ] Oscar Bonilla commented on SPARK-26365: --- I've changed the priority to Major, to see if someone can

[jira] [Updated] (SPARK-26365) spark-submit for k8s cluster doesn't propagate exit code

[ https://issues.apache.org/jira/browse/SPARK-26365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oscar Bonilla updated SPARK-26365: -- Affects Version/s: 3.0.0 3.1.0 > spark-submit for k8s cluster doesn't

[jira] [Updated] (SPARK-26365) spark-submit for k8s cluster doesn't propagate exit code

[ https://issues.apache.org/jira/browse/SPARK-26365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oscar Bonilla updated SPARK-26365: -- Priority: Major (was: Minor) > spark-submit for k8s cluster doesn't propagate exit code >

[jira] [Commented] (SPARK-26365) spark-submit for k8s cluster doesn't propagate exit code

[ https://issues.apache.org/jira/browse/SPARK-26365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437825#comment-17437825 ] Vivien Brissat commented on SPARK-26365: Hi [~oscar.bonilla], this is not since i made tests in

[jira] [Commented] (SPARK-37203) Fix NotSerializableException when observe with percentile_approx

[ https://issues.apache.org/jira/browse/SPARK-37203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437805#comment-17437805 ] Apache Spark commented on SPARK-37203: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37203) Fix NotSerializableException when observe with percentile_approx

[ https://issues.apache.org/jira/browse/SPARK-37203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37203: Assignee: (was: Apache Spark) > Fix NotSerializableException when observe with

[jira] [Commented] (SPARK-37203) Fix NotSerializableException when observe with percentile_approx

[ https://issues.apache.org/jira/browse/SPARK-37203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437803#comment-17437803 ] Apache Spark commented on SPARK-37203: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37203) Fix NotSerializableException when observe with percentile_approx

[ https://issues.apache.org/jira/browse/SPARK-37203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37203: Assignee: Apache Spark > Fix NotSerializableException when observe with

[jira] [Commented] (SPARK-37203) Fix NotSerializableException when observe with percentile_approx

[ https://issues.apache.org/jira/browse/SPARK-37203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437780#comment-17437780 ] jiaan.geng commented on SPARK-37203: I'm working on. > Fix NotSerializableException when observe

[jira] [Created] (SPARK-37203) Fix NotSerializableException when observe with percentile_approx

jiaan.geng created SPARK-37203: -- Summary: Fix NotSerializableException when observe with percentile_approx Key: SPARK-37203 URL: https://issues.apache.org/jira/browse/SPARK-37203 Project: Spark

[jira] [Resolved] (SPARK-37200) Drop index support

[ https://issues.apache.org/jira/browse/SPARK-37200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao resolved SPARK-37200. Fix Version/s: 3.3.0 Assignee: Huaxin Gao Resolution: Fixed > Drop index support

[jira] [Updated] (SPARK-37179) ANSI mode: Add a config to allow casting between Datetime and Numeric

[ https://issues.apache.org/jira/browse/SPARK-37179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-37179: --- Description: Add a config `spark.sql.ansi.allowCastBetweenDatetimeAndNumeric`to allow

[jira] [Updated] (SPARK-37179) ANSI mode: Add a config to allow casting between Datetime and Numeric

[ https://issues.apache.org/jira/browse/SPARK-37179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-37179: --- Summary: ANSI mode: Add a config to allow casting between Datetime and Numeric (was: ANSI

[jira] [Resolved] (SPARK-37179) ANSI mode: Allow casting between Timestamp and Numeric

[ https://issues.apache.org/jira/browse/SPARK-37179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37179. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34459