[jira] [Resolved] (SPARK-43271) Match behavior with DataFrame.reindex with specifying `index`.

[ https://issues.apache.org/jira/browse/SPARK-43271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-43271. - Resolution: Fixed > Match behavior with DataFrame.reindex with specifying `index`. >

[jira] [Resolved] (SPARK-45168) Increate Pandas minimum version to 1.4.4

[ https://issues.apache.org/jira/browse/SPARK-45168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-45168. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42930

[jira] [Assigned] (SPARK-45168) Increate Pandas minimum version to 1.4.4

[ https://issues.apache.org/jira/browse/SPARK-45168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-45168: - Assignee: Ruifeng Zheng > Increate Pandas minimum version to 1.4.4 >

[jira] [Updated] (SPARK-45178) Fallback to use single batch executor for Trigger.AvailableNow with unsupported sources rather than using wrapper

[ https://issues.apache.org/jira/browse/SPARK-45178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45178: --- Labels: pull-request-available (was: ) > Fallback to use single batch executor for

[jira] [Updated] (SPARK-43254) Assign a name to the error class _LEGACY_ERROR_TEMP_2018

[ https://issues.apache.org/jira/browse/SPARK-43254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43254: --- Labels: pull-request-available starter (was: starter) > Assign a name to the error class

[jira] [Commented] (SPARK-45178) Fallback to use single batch executor for Trigger.AvailableNow with unsupported sources rather than using wrapper

[ https://issues.apache.org/jira/browse/SPARK-45178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765451#comment-17765451 ] Jungtaek Lim commented on SPARK-45178: -- PR will be available sooner. > Fallback to use single

[jira] [Updated] (SPARK-44788) XML: Add pyspark.sql.functions

[ https://issues.apache.org/jira/browse/SPARK-44788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44788: --- Labels: pull-request-available (was: ) > XML: Add pyspark.sql.functions >

[jira] [Created] (SPARK-45178) Fallback to use single batch executor for Trigger.AvailableNow with unsupported sources rather than using wrapper

Jungtaek Lim created SPARK-45178: Summary: Fallback to use single batch executor for Trigger.AvailableNow with unsupported sources rather than using wrapper Key: SPARK-45178 URL:

[jira] [Updated] (SPARK-45177) Remove `col_space` parameter from `DataFrame.to_latex`

[ https://issues.apache.org/jira/browse/SPARK-45177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45177: --- Labels: pull-request-available (was: ) > Remove `col_space` parameter from

[jira] [Created] (SPARK-45177) Remove `col_space` parameter from `DataFrame.to_latex`

Haejoon Lee created SPARK-45177: --- Summary: Remove `col_space` parameter from `DataFrame.to_latex` Key: SPARK-45177 URL: https://issues.apache.org/jira/browse/SPARK-45177 Project: Spark Issue

[jira] [Updated] (SPARK-45143) Make PySpark compatible with PyArrow 13.0.0

[ https://issues.apache.org/jira/browse/SPARK-45143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45143: -- Parent: SPARK-43831 Issue Type: Sub-task (was: Improvement) > Make PySpark

[jira] [Updated] (SPARK-44434) Add more tests for Scala foreachBatch and streaming listeners

[ https://issues.apache.org/jira/browse/SPARK-44434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-44434: - Fix Version/s: (was: 3.5.0) > Add more tests for Scala foreachBatch and streaming listeners

[jira] [Resolved] (SPARK-45143) Make PySpark compatible with PyArrow 13.0.0

[ https://issues.apache.org/jira/browse/SPARK-45143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45143. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42920

[jira] [Assigned] (SPARK-45143) Make PySpark compatible with PyArrow 13.0.0

[ https://issues.apache.org/jira/browse/SPARK-45143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45143: - Assignee: Ruifeng Zheng > Make PySpark compatible with PyArrow 13.0.0 >

[jira] [Updated] (SPARK-44699) Add logging for complete write events to file in EventLogFileWriter.closeWriter

[ https://issues.apache.org/jira/browse/SPARK-44699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-44699: - Fix Version/s: (was: 3.5.0) > Add logging for complete write events to file in >

[jira] [Updated] (SPARK-42252) Deprecate spark.shuffle.unsafe.file.output.buffer and add a new config

[ https://issues.apache.org/jira/browse/SPARK-42252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42252: - Target Version/s: (was: 3.5.0) > Deprecate spark.shuffle.unsafe.file.output.buffer and add a

[jira] [Updated] (SPARK-44307) Bloom filter is not added for left outer join if the left side table is smaller than broadcast threshold.

[ https://issues.apache.org/jira/browse/SPARK-44307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-44307: - Fix Version/s: (was: 3.5.0) > Bloom filter is not added for left outer join if the left

[jira] [Updated] (SPARK-38945) simply KEYTAB and PRINCIPAL in KerberosConfDriverFeatureStep

[ https://issues.apache.org/jira/browse/SPARK-38945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38945: - Fix Version/s: (was: 3.5.0) > simply KEYTAB and PRINCIPAL in KerberosConfDriverFeatureStep

[jira] [Updated] (SPARK-43155) DataSourceV2 is hard to be implemented without following V1

[ https://issues.apache.org/jira/browse/SPARK-43155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-43155: - Target Version/s: (was: 3.5.0) > DataSourceV2 is hard to be implemented without following V1

[jira] [Updated] (SPARK-41259) Spark-sql cli query results should correspond to schema

[ https://issues.apache.org/jira/browse/SPARK-41259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-41259: - Fix Version/s: (was: 3.5.0) > Spark-sql cli query results should correspond to schema >

[jira] [Updated] (SPARK-42252) Deprecate spark.shuffle.unsafe.file.output.buffer and add a new config

[ https://issues.apache.org/jira/browse/SPARK-42252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42252: - Fix Version/s: (was: 3.5.0) > Deprecate spark.shuffle.unsafe.file.output.buffer and add a

[jira] [Updated] (SPARK-37935) Migrate onto error classes

[ https://issues.apache.org/jira/browse/SPARK-37935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37935: - Fix Version/s: (was: 3.5.0) > Migrate onto error classes > -- > >

[jira] [Updated] (SPARK-39136) JDBCTable support properties

[ https://issues.apache.org/jira/browse/SPARK-39136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39136: - Fix Version/s: (was: 3.5.0) > JDBCTable support properties > >

[jira] [Updated] (SPARK-39892) Use ArrowType.Decimal(precision, scale, bitWidth) instead of ArrowType.Decimal(precision, scale)

[ https://issues.apache.org/jira/browse/SPARK-39892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39892: - Fix Version/s: (was: 3.5.0) > Use ArrowType.Decimal(precision, scale, bitWidth) instead of

[jira] [Updated] (SPARK-43155) DataSourceV2 is hard to be implemented without following V1

[ https://issues.apache.org/jira/browse/SPARK-43155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-43155: - Fix Version/s: (was: 3.5.0) > DataSourceV2 is hard to be implemented without following V1 >

[jira] [Updated] (SPARK-39814) Use AmazonKinesisClientBuilder.withCredentials instead of new AmazonKinesisClient(credentials)

[ https://issues.apache.org/jira/browse/SPARK-39814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39814: - Fix Version/s: (was: 3.5.0) > Use AmazonKinesisClientBuilder.withCredentials instead of new

[jira] [Updated] (SPARK-44307) Bloom filter is not added for left outer join if the left side table is smaller than broadcast threshold.

[ https://issues.apache.org/jira/browse/SPARK-44307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-44307: - Target Version/s: (was: 3.4.1) > Bloom filter is not added for left outer join if the left

[jira] [Updated] (SPARK-43318) spark reader csv and json support wholetext parameters

[ https://issues.apache.org/jira/browse/SPARK-43318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-43318: - Fix Version/s: (was: 3.5.0) > spark reader csv and json support wholetext parameters >

[jira] [Resolved] (SPARK-45172) Upgrade commons-compress.version from 1.23.0 to 1.24.0

[ https://issues.apache.org/jira/browse/SPARK-45172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45172. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42934

[jira] [Assigned] (SPARK-45172) Upgrade commons-compress.version from 1.23.0 to 1.24.0

[ https://issues.apache.org/jira/browse/SPARK-45172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45172: - Assignee: Hyukjin Kwon > Upgrade commons-compress.version from 1.23.0 to 1.24.0 >

[jira] [Assigned] (SPARK-45171) GenerateExec fails to initialize non-deterministic expressions before use

[ https://issues.apache.org/jira/browse/SPARK-45171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45171: Assignee: Bruce Robbins > GenerateExec fails to initialize non-deterministic expressions

[jira] [Resolved] (SPARK-45171) GenerateExec fails to initialize non-deterministic expressions before use

[ https://issues.apache.org/jira/browse/SPARK-45171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45171. -- Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-45174) Support spark.deploy.maxDrivers

[ https://issues.apache.org/jira/browse/SPARK-45174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45174. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42936

[jira] [Assigned] (SPARK-45174) Support spark.deploy.maxDrivers

[ https://issues.apache.org/jira/browse/SPARK-45174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45174: - Assignee: Dongjoon Hyun > Support spark.deploy.maxDrivers >

[jira] [Resolved] (SPARK-45165) Remove `inplace` parameter from `Categorical` APIs

[ https://issues.apache.org/jira/browse/SPARK-45165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45165. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42927

[jira] [Assigned] (SPARK-45165) Remove `inplace` parameter from `Categorical` APIs

[ https://issues.apache.org/jira/browse/SPARK-45165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45165: - Assignee: Haejoon Lee > Remove `inplace` parameter from `Categorical` APIs >

[jira] [Updated] (SPARK-45176) AggregatingAccumulator with TypedImperativeAggregate throwing ClassCastException

[ https://issues.apache.org/jira/browse/SPARK-45176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huw updated SPARK-45176: Description: Probably related to SPARK-39044. But potentially also this comment in Executor.scala. {quote}//

[jira] [Created] (SPARK-45176) AggregatingAccumulator with TypedImperativeAggregate throwing ClassCastException

Huw created SPARK-45176: --- Summary: AggregatingAccumulator with TypedImperativeAggregate throwing ClassCastException Key: SPARK-45176 URL: https://issues.apache.org/jira/browse/SPARK-45176 Project: Spark

[jira] [Created] (SPARK-45175) download krb5.conf from remote storage in spark-sumbit on k8s

Qian Sun created SPARK-45175: Summary: download krb5.conf from remote storage in spark-sumbit on k8s Key: SPARK-45175 URL: https://issues.apache.org/jira/browse/SPARK-45175 Project: Spark Issue

[jira] [Created] (SPARK-45174) Support spark.deploy.maxDrivers

Dongjoon Hyun created SPARK-45174: - Summary: Support spark.deploy.maxDrivers Key: SPARK-45174 URL: https://issues.apache.org/jira/browse/SPARK-45174 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-45174) Support spark.deploy.maxDrivers

[ https://issues.apache.org/jira/browse/SPARK-45174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45174: --- Labels: pull-request-available (was: ) > Support spark.deploy.maxDrivers >

[jira] [Updated] (SPARK-45173) Remove some unnecessary sourceMapping files in UI

[ https://issues.apache.org/jira/browse/SPARK-45173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45173: --- Labels: pull-request-available (was: ) > Remove some unnecessary sourceMapping files in UI

[jira] [Created] (SPARK-45173) Remove some unnecessary sourceMapping files in UI

Kent Yao created SPARK-45173: Summary: Remove some unnecessary sourceMapping files in UI Key: SPARK-45173 URL: https://issues.apache.org/jira/browse/SPARK-45173 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-45159) Handle named arguments only when necessary

[ https://issues.apache.org/jira/browse/SPARK-45159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45159. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42915

[jira] [Assigned] (SPARK-45159) Handle named arguments only when necessary

[ https://issues.apache.org/jira/browse/SPARK-45159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45159: Assignee: Takuya Ueshin > Handle named arguments only when necessary >

[jira] [Commented] (SPARK-44752) XML: Update Spark Docs

[ https://issues.apache.org/jira/browse/SPARK-44752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765419#comment-17765419 ] tangjiafu commented on SPARK-44752: --- I have used Spark XML in my project before, and I think I can do

[jira] [Resolved] (SPARK-45084) ProgressReport should include an accurate effective shuffle partition number

[ https://issues.apache.org/jira/browse/SPARK-45084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-45084. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42822

[jira] [Assigned] (SPARK-45084) ProgressReport should include an accurate effective shuffle partition number

[ https://issues.apache.org/jira/browse/SPARK-45084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-45084: Assignee: Siying Dong > ProgressReport should include an accurate effective shuffle

[jira] [Resolved] (SPARK-43406) enable spark sql to drop multiple partitions in one call

[ https://issues.apache.org/jira/browse/SPARK-43406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-43406. - Resolution: Duplicate > enable spark sql to drop multiple partitions in one call >

[jira] [Updated] (SPARK-43406) enable spark sql to drop multiple partitions in one call

[ https://issues.apache.org/jira/browse/SPARK-43406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43406: Target Version/s: (was: 4.0.0) > enable spark sql to drop multiple partitions in one call >

[jira] [Updated] (SPARK-43406) enable spark sql to drop multiple partitions in one call

[ https://issues.apache.org/jira/browse/SPARK-43406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43406: Fix Version/s: (was: 3.5.0) > enable spark sql to drop multiple partitions in one call >

[jira] [Updated] (SPARK-43406) enable spark sql to drop multiple partitions in one call

[ https://issues.apache.org/jira/browse/SPARK-43406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-43406: Target Version/s: 4.0.0 > enable spark sql to drop multiple partitions in one call >

[jira] (SPARK-37487) CollectMetrics is executed twice if it is followed by a sort

[ https://issues.apache.org/jira/browse/SPARK-37487 ] Huw deleted comment on SPARK-37487: - was (Author: JIRAUSER288917): I think I've seen crashes because of this in production. I can't reproduce locally, but I believe that Imperative aggregates are

[jira] [Updated] (SPARK-42466) spark.kubernetes.file.upload.path not deleting files under HDFS after job completes

[ https://issues.apache.org/jira/browse/SPARK-42466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-42466: --- Labels: pull-request-available (was: ) > spark.kubernetes.file.upload.path not deleting

[jira] [Updated] (SPARK-16484) Incremental Cardinality estimation operations with Hyperloglog

[ https://issues.apache.org/jira/browse/SPARK-16484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-16484: --- Labels: bulk-closed pull-request-available (was: bulk-closed) > Incremental Cardinality

[jira] [Updated] (SPARK-45172) Upgrade commons-compress.version from 1.23.0 to 1.24.0

[ https://issues.apache.org/jira/browse/SPARK-45172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45172: --- Labels: pull-request-available (was: ) > Upgrade commons-compress.version from 1.23.0 to

[jira] [Updated] (SPARK-45172) Upgrade commons-compress.version from 1.23.0 to 1.24.0

[ https://issues.apache.org/jira/browse/SPARK-45172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-45172: - Summary: Upgrade commons-compress.version from 1.23.0 to 1.24.0 (was: Upgrade

[jira] [Created] (SPARK-45172) Upgrade commons-compress.version from 1.23.0 to .124.0

Hyukjin Kwon created SPARK-45172: Summary: Upgrade commons-compress.version from 1.23.0 to .124.0 Key: SPARK-45172 URL: https://issues.apache.org/jira/browse/SPARK-45172 Project: Spark Issue

[jira] [Updated] (SPARK-45172) Upgrade commons-compress.version from 1.23.0 to .124.0

[ https://issues.apache.org/jira/browse/SPARK-45172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-45172: - Issue Type: Improvement (was: Bug) > Upgrade commons-compress.version from 1.23.0 to .124.0 >

[jira] [Updated] (SPARK-45171) GenerateExec fails to initialize non-deterministic expressions before use

[ https://issues.apache.org/jira/browse/SPARK-45171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45171: --- Labels: pull-request-available (was: ) > GenerateExec fails to initialize

[jira] [Assigned] (SPARK-45161) Bump `previousSparkVersion` to 3.5.0

[ https://issues.apache.org/jira/browse/SPARK-45161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45161: - Assignee: Yang Jie > Bump `previousSparkVersion` to 3.5.0 >

[jira] [Resolved] (SPARK-45161) Bump `previousSparkVersion` to 3.5.0

[ https://issues.apache.org/jira/browse/SPARK-45161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45161. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42921

[jira] [Updated] (SPARK-45137) Unsupported map and array constructors by `sql()` in connect clients

[ https://issues.apache.org/jira/browse/SPARK-45137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45137: --- Labels: pull-request-available (was: ) > Unsupported map and array constructors by `sql()`

[jira] [Resolved] (SPARK-45118) Refactor converters for complex types to short cut when the element types don't need converters

[ https://issues.apache.org/jira/browse/SPARK-45118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-45118. --- Fix Version/s: 4.0.0 Assignee: Takuya Ueshin Resolution: Fixed Issue

[jira] [Created] (SPARK-45171) GenerateExec fails to initialize non-deterministic expressions before use

Bruce Robbins created SPARK-45171: - Summary: GenerateExec fails to initialize non-deterministic expressions before use Key: SPARK-45171 URL: https://issues.apache.org/jira/browse/SPARK-45171 Project:

[jira] [Updated] (SPARK-43966) Support non-deterministic Python UDTFs

[ https://issues.apache.org/jira/browse/SPARK-43966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43966: --- Labels: pull-request-available (was: ) > Support non-deterministic Python UDTFs >

[jira] [Created] (SPARK-45170) Scala-specific improvements in Dataset[T] API

Danila Goloshchapov created SPARK-45170: --- Summary: Scala-specific improvements in Dataset[T] API Key: SPARK-45170 URL: https://issues.apache.org/jira/browse/SPARK-45170 Project: Spark

[jira] [Updated] (SPARK-44141) Remove need to preinstall the buf compiler

[ https://issues.apache.org/jira/browse/SPARK-44141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44141: --- Labels: pull-request-available (was: ) > Remove need to preinstall the buf compiler >

[jira] [Resolved] (SPARK-45169) Add official image Dockerfile for Apache Spark 3.5.0

[ https://issues.apache.org/jira/browse/SPARK-45169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yikun Jiang resolved SPARK-45169. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 55

[jira] [Updated] (SPARK-45169) Add official image Dockerfile for Apache Spark 3.5.0

[ https://issues.apache.org/jira/browse/SPARK-45169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45169: --- Labels: pull-request-available (was: ) > Add official image Dockerfile for Apache Spark

[jira] [Created] (SPARK-45169) Add official image Dockerfile for Apache Spark 3.5.0

Yikun Jiang created SPARK-45169: --- Summary: Add official image Dockerfile for Apache Spark 3.5.0 Key: SPARK-45169 URL: https://issues.apache.org/jira/browse/SPARK-45169 Project: Spark Issue

[jira] [Updated] (SPARK-45168) Increate Pandas minimum version to 1.4.4

[ https://issues.apache.org/jira/browse/SPARK-45168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45168: --- Labels: pull-request-available (was: ) > Increate Pandas minimum version to 1.4.4 >

[jira] [Created] (SPARK-45168) Increate Pandas minimum version to 1.4.4

Ruifeng Zheng created SPARK-45168: - Summary: Increate Pandas minimum version to 1.4.4 Key: SPARK-45168 URL: https://issues.apache.org/jira/browse/SPARK-45168 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45167) Python Spark Connect client does not call `releaseAll`

[ https://issues.apache.org/jira/browse/SPARK-45167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45167: --- Labels: pull-request-available (was: ) > Python Spark Connect client does not call

[jira] [Updated] (SPARK-45167) Python Spark Connect client does not call `releaseAll`

[ https://issues.apache.org/jira/browse/SPARK-45167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juliusz Sompolski updated SPARK-45167: -- Epic Link: SPARK-43754 (was: SPARK-39375) > Python Spark Connect client does not

[jira] [Updated] (SPARK-45167) Python Spark Connect client does not call `releaseAll`

[ https://issues.apache.org/jira/browse/SPARK-45167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Martin Grund updated SPARK-45167: - Issue Type: Bug (was: Improvement) > Python Spark Connect client does not call `releaseAll` >

[jira] [Created] (SPARK-45167) Python Spark Connect client does not call `releaseAll`

Martin Grund created SPARK-45167: Summary: Python Spark Connect client does not call `releaseAll` Key: SPARK-45167 URL: https://issues.apache.org/jira/browse/SPARK-45167 Project: Spark Issue

[jira] [Updated] (SPARK-45166) Clean up unused code paths for pyarrow<4

[ https://issues.apache.org/jira/browse/SPARK-45166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45166: --- Labels: pull-request-available (was: ) > Clean up unused code paths for pyarrow<4 >

[jira] [Created] (SPARK-45166) Clean up unused code paths for pyarrow<4

Ruifeng Zheng created SPARK-45166: - Summary: Clean up unused code paths for pyarrow<4 Key: SPARK-45166 URL: https://issues.apache.org/jira/browse/SPARK-45166 Project: Spark Issue Type:

[jira] [Commented] (SPARK-31177) DataFrameReader.csv incorrectly reads gzip encoded CSV from S3 when it has non-".gz" extension

[ https://issues.apache.org/jira/browse/SPARK-31177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17765139#comment-17765139 ] Avi minsky commented on SPARK-31177: [~markwaddle] , [~maropu] how was this resolved?  >

[jira] [Assigned] (SPARK-45119) Refine docstring of `inline`

[ https://issues.apache.org/jira/browse/SPARK-45119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-45119: - Assignee: Allison Wang > Refine docstring of `inline` > >

[jira] [Resolved] (SPARK-45119) Refine docstring of `inline`

[ https://issues.apache.org/jira/browse/SPARK-45119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-45119. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42875

[jira] [Updated] (SPARK-45154) Pyspark DecisionTreeClassifier: results and tree structure in spark3 very different from that of the spark2 version on the same data and with the same hyperparameters.

[ https://issues.apache.org/jira/browse/SPARK-45154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oumar Nour updated SPARK-45154: --- Priority: Critical (was: Major) > Pyspark DecisionTreeClassifier: results and tree structure in

[jira] [Updated] (SPARK-45163) Merge TABLE_OPERATION & _LEGACY_ERROR_TEMP_1113 into UNSUPPORTED_TABLE_OPERATION and refactor some logic

[ https://issues.apache.org/jira/browse/SPARK-45163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45163: --- Labels: pull-request-available (was: ) > Merge TABLE_OPERATION & _LEGACY_ERROR_TEMP_1113

[jira] [Updated] (SPARK-45163) Merge TABLE_OPERATION & _LEGACY_ERROR_TEMP_1113 into UNSUPPORTED_TABLE_OPERATION and refactor some logic

[ https://issues.apache.org/jira/browse/SPARK-45163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-45163: Summary: Merge TABLE_OPERATION & _LEGACY_ERROR_TEMP_1113 into UNSUPPORTED_TABLE_OPERATION and

[jira] [Assigned] (SPARK-45088) Make `getitem` work with duplicated columns

[ https://issues.apache.org/jira/browse/SPARK-45088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-45088: - Assignee: Ruifeng Zheng > Make `getitem` work with duplicated columns >

[jira] [Resolved] (SPARK-45088) Make `getitem` work with duplicated columns

[ https://issues.apache.org/jira/browse/SPARK-45088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-45088. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42828

[jira] [Updated] (SPARK-45165) Remove `inplace` parameter from `Categorical` APIs

[ https://issues.apache.org/jira/browse/SPARK-45165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45165: --- Labels: pull-request-available (was: ) > Remove `inplace` parameter from `Categorical`

[jira] [Updated] (SPARK-45165) Remove `inplace` parameter from `Categorical` APIs

[ https://issues.apache.org/jira/browse/SPARK-45165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-45165: Summary: Remove `inplace` parameter from `Categorical` APIs (was: Remove `inplace` parameter

[jira] [Created] (SPARK-45165) Remove `inplace` parameter from `CategoricalIndex` APIs

Haejoon Lee created SPARK-45165: --- Summary: Remove `inplace` parameter from `CategoricalIndex` APIs Key: SPARK-45165 URL: https://issues.apache.org/jira/browse/SPARK-45165 Project: Spark Issue

[jira] [Updated] (SPARK-45164) Remove deprecated Index APIs

[ https://issues.apache.org/jira/browse/SPARK-45164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45164: --- Labels: pull-request-available (was: ) > Remove deprecated Index APIs >

[jira] [Resolved] (SPARK-45156) Wrap inputName by backticks in the NON_FOLDABLE_INPUT error class

[ https://issues.apache.org/jira/browse/SPARK-45156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-45156. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42905

[jira] [Updated] (SPARK-45156) Wrap inputName by backticks in the NON_FOLDABLE_INPUT error class

[ https://issues.apache.org/jira/browse/SPARK-45156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45156: --- Labels: pull-request-available (was: ) > Wrap inputName by backticks in the

[jira] [Updated] (SPARK-45164) Remove deprecated Index APIs

[ https://issues.apache.org/jira/browse/SPARK-45164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-45164: Description: We should remove the deprecated Index APIs to match the behavior with Pandas 2.0.0

[jira] [Created] (SPARK-45164) Remove deprecated Index APIs

Haejoon Lee created SPARK-45164: --- Summary: Remove deprecated Index APIs Key: SPARK-45164 URL: https://issues.apache.org/jira/browse/SPARK-45164 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-45163) Merge UNSUPPORTED_FEATURE.TABLE_OPERATION into UNSUPPORTED_TABLE_OPERATION and refactor some logic

BingKun Pan created SPARK-45163: --- Summary: Merge UNSUPPORTED_FEATURE.TABLE_OPERATION into UNSUPPORTED_TABLE_OPERATION and refactor some logic Key: SPARK-45163 URL: https://issues.apache.org/jira/browse/SPARK-45163

[jira] [Resolved] (SPARK-43335) Migrate remaining Connect/SparkSQL errors into error classes

[ https://issues.apache.org/jira/browse/SPARK-43335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-43335. - Resolution: Fixed > Migrate remaining Connect/SparkSQL errors into error classes >

[jira] [Resolved] (SPARK-43303) Migrate NotImplementedError into PySparkNotImplementedError

[ https://issues.apache.org/jira/browse/SPARK-43303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-43303. - Resolution: Fixed > Migrate NotImplementedError into PySparkNotImplementedError >

[jira] [Resolved] (SPARK-42986) Migrate more PySpark errors onto error classes

[ https://issues.apache.org/jira/browse/SPARK-42986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-42986. - Resolution: Resolved > Migrate more PySpark errors onto error classes >

[jira] [Resolved] (SPARK-43020) Refactoring similar error classes such as `NOT_XXX`.

[ https://issues.apache.org/jira/browse/SPARK-43020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-43020. - Resolution: Won't Fix > Refactoring similar error classes such as `NOT_XXX`. >

  1   2   >