[jira] [Resolved] (SPARK-45943) DataSourceV2Relation.computeStats throws IllegalStateException in test mode

2023-11-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45943. -- Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Created] (SPARK-46181) Split scheduled Python build

2023-11-29 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46181: Summary: Split scheduled Python build Key: SPARK-46181 URL: https://issues.apache.org/jira/browse/SPARK-46181 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-46180) Reduce the number of shuffle partitions in pandas-on-Spark test cases

2023-11-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46180. -- Resolution: Invalid > Reduce the number of shuffle partitions in pandas-on-Spark test cases >

[jira] [Created] (SPARK-46180) Reduce the number of shuffle partitions in pandas-on-Spark test cases

2023-11-29 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46180: Summary: Reduce the number of shuffle partitions in pandas-on-Spark test cases Key: SPARK-46180 URL: https://issues.apache.org/jira/browse/SPARK-46180 Project: Spark

[jira] [Updated] (SPARK-46178) Move MockDF under PlanOnlyTestFixture

2023-11-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46178: - Issue Type: Test (was: Improvement) > Move MockDF under PlanOnlyTestFixture >

[jira] [Created] (SPARK-46178) Move MockDF under PlanOnlyTestFixture

2023-11-29 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46178: Summary: Move MockDF under PlanOnlyTestFixture Key: SPARK-46178 URL: https://issues.apache.org/jira/browse/SPARK-46178 Project: Spark Issue Type:

[jira] [Created] (SPARK-46177) Skip 'CrossValidatorTests.test_crossvalidator_with_fold_col' with Python 3.12

2023-11-29 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46177: Summary: Skip 'CrossValidatorTests.test_crossvalidator_with_fold_col' with Python 3.12 Key: SPARK-46177 URL: https://issues.apache.org/jira/browse/SPARK-46177

[jira] [Assigned] (SPARK-46149) Skip `TorchDistributorLocalUnitTests.test_end_to_end_run_locally` with Python 3.12

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46149: Assignee: Hyukjin Kwon > Skip

[jira] [Resolved] (SPARK-46149) Skip `TorchDistributorLocalUnitTests.test_end_to_end_run_locally` with Python 3.12

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46149. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44065

[jira] [Resolved] (SPARK-46148) Fix pyspark.pandas.mlflow.load_model test (Python 3.12)

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46148. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44064

[jira] [Assigned] (SPARK-46148) Fix pyspark.pandas.mlflow.load_model test (Python 3.12)

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46148: Assignee: Hyukjin Kwon > Fix pyspark.pandas.mlflow.load_model test (Python 3.12) >

[jira] [Updated] (SPARK-46149) Skip `TorchDistributorLocalUnitTests.test_end_to_end_run_locally` with Python 3.12

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46149: - Description: {code} ==

[jira] [Updated] (SPARK-46149) Skip `TorchDistributorLocalUnitTests.test_end_to_end_run_locally` with Python 3.12

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46149: - Summary: Skip `TorchDistributorLocalUnitTests.test_end_to_end_run_locally` with Python 3.12

[jira] [Updated] (SPARK-46149) Skip `TorchDistributorLocalUnitTests.test_end_to_end_run_locally`

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46149: - Priority: Minor (was: Major) > Skip

[jira] [Created] (SPARK-46149) Skip `TorchDistributorLocalUnitTests.test_end_to_end_run_locally`

2023-11-28 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46149: Summary: Skip `TorchDistributorLocalUnitTests.test_end_to_end_run_locally` Key: SPARK-46149 URL: https://issues.apache.org/jira/browse/SPARK-46149 Project: Spark

[jira] [Updated] (SPARK-46148) Fix pyspark.pandas.mlflow.load_model test (Python 3.12)

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46148: - Description: {code} ** File

[jira] [Created] (SPARK-46148) Fix pyspark.pandas.mlflow.load_model test (Python 3.12)

2023-11-28 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46148: Summary: Fix pyspark.pandas.mlflow.load_model test (Python 3.12) Key: SPARK-46148 URL: https://issues.apache.org/jira/browse/SPARK-46148 Project: Spark

[jira] [Updated] (SPARK-46147) Fix the doctest in pyspark.pandas.series.Series.to_dict (Python 3.12)

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46147: - Fix Version/s: (was: 4.0.0) > Fix the doctest in pyspark.pandas.series.Series.to_dict

[jira] [Updated] (SPARK-46147) Fix the doctest in pyspark.pandas.series.Series.to_dict (Python 3.12)

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46147: - Labels: (was: pull-request-available) > Fix the doctest in

[jira] [Updated] (SPARK-46147) Fix the doctest in pyspark.pandas.series.Series.to_dict (Python 3.12)

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46147: - Description: {code} File "/__w/spark/spark/python/pyspark/pandas/series.py", line 1633, in

[jira] [Created] (SPARK-46147) Fix the doctest in pyspark.pandas.series.Series.to_dict (Python 3.12)

2023-11-28 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46147: Summary: Fix the doctest in pyspark.pandas.series.Series.to_dict (Python 3.12) Key: SPARK-46147 URL: https://issues.apache.org/jira/browse/SPARK-46147 Project: Spark

[jira] [Deleted] (SPARK-46136) Do not combine adjacent Python UDFs if pythonExec is different

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon deleted SPARK-46136: - > Do not combine adjacent Python UDFs if pythonExec is different >

[jira] [Created] (SPARK-46136) Do not combine adjacent Python UDFs if pythonExec is different

2023-11-28 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46136: Summary: Do not combine adjacent Python UDFs if pythonExec is different Key: SPARK-46136 URL: https://issues.apache.org/jira/browse/SPARK-46136 Project: Spark

[jira] [Resolved] (SPARK-32407) Remove upperbound of Sphinx version

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32407. -- Fix Version/s: 4.0.0 Assignee: Ruifeng Zheng Resolution: Fixed

[jira] [Assigned] (SPARK-46127) Flaky `pyspark.tests.test_worker.WorkerSegfaultNonDaemonTest.test_python_segfault` with Python 3.12

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46127: Assignee: Hyukjin Kwon > Flaky >

[jira] [Resolved] (SPARK-46131) Install torchvision for Python 3.12 build

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46131. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44045

[jira] [Resolved] (SPARK-46127) Flaky `pyspark.tests.test_worker.WorkerSegfaultNonDaemonTest.test_python_segfault` with Python 3.12

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46127. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44044

[jira] [Assigned] (SPARK-46131) Install torchvision for Python 3.12 build

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46131: Assignee: Hyukjin Kwon > Install torchvision for Python 3.12 build >

[jira] [Updated] (SPARK-46131) Install torchvision for Python 3.12 build

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46131: - Description: {code} ==

[jira] [Created] (SPARK-46131) Install torchvision for Python 3.12 build

2023-11-27 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46131: Summary: Install torchvision for Python 3.12 build Key: SPARK-46131 URL: https://issues.apache.org/jira/browse/SPARK-46131 Project: Spark Issue Type:

[jira] [Updated] (SPARK-46130) Reeanble `pyspark.tests.test_worker.WorkerSegfaultNonDaemonTest.test_python_segfault` with Python 3.12

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46130: - Parent: SPARK-45981 Issue Type: Sub-task (was: Test) > Reeanble >

[jira] [Created] (SPARK-46130) Reeanble `pyspark.tests.test_worker.WorkerSegfaultNonDaemonTest.test_python_segfault` with Python 3.12

2023-11-27 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46130: Summary: Reeanble `pyspark.tests.test_worker.WorkerSegfaultNonDaemonTest.test_python_segfault` with Python 3.12 Key: SPARK-46130 URL:

[jira] [Assigned] (SPARK-46126) Fix the doctest in pyspark.pandas.frame.DataFrame.to_dict (Python 3.12)

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46126: Assignee: Hyukjin Kwon > Fix the doctest in pyspark.pandas.frame.DataFrame.to_dict

[jira] [Resolved] (SPARK-46126) Fix the doctest in pyspark.pandas.frame.DataFrame.to_dict (Python 3.12)

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46126. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44042

[jira] [Resolved] (SPARK-46111) Add copyright to the PySpark official documentation.

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46111. -- Fix Version/s: 4.0.0 Assignee: Haejoon Lee Resolution: Fixed Fixed in

[jira] [Created] (SPARK-46127) Flaky `pyspark.tests.test_worker.WorkerSegfaultNonDaemonTest.test_python_segfault` with Python 3.12

2023-11-27 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46127: Summary: Flaky `pyspark.tests.test_worker.WorkerSegfaultNonDaemonTest.test_python_segfault` with Python 3.12 Key: SPARK-46127 URL:

[jira] [Created] (SPARK-46126) Fix the doctest in pyspark.pandas.frame.DataFrame.to_dict (Python 3.12)

2023-11-27 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46126: Summary: Fix the doctest in pyspark.pandas.frame.DataFrame.to_dict (Python 3.12) Key: SPARK-46126 URL: https://issues.apache.org/jira/browse/SPARK-46126 Project:

[jira] [Resolved] (SPARK-46121) Refine docstring of `concat/array_position/element_at/try_element_at`

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46121. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44039

[jira] [Assigned] (SPARK-46121) Refine docstring of `concat/array_position/element_at/try_element_at`

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46121: Assignee: Yang Jie > Refine docstring of

[jira] [Resolved] (SPARK-46115) Restrict charsets in encode()

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46115. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44020

[jira] [Assigned] (SPARK-45957) SQL on streaming Temp view fails

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45957: Assignee: Jean-Francois Desjeans Gauthier > SQL on streaming Temp view fails >

[jira] [Resolved] (SPARK-45957) SQL on streaming Temp view fails

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45957. -- Resolution: Fixed Issue resolved by pull request 43851

[jira] [Assigned] (SPARK-46110) Use error classes in catalog, conf, connect, observation, pandas modules

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46110: Assignee: Hyukjin Kwon > Use error classes in catalog, conf, connect, observation,

[jira] [Resolved] (SPARK-46110) Use error classes in catalog, conf, connect, observation, pandas modules

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46110. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44024

[jira] [Assigned] (SPARK-46120) Remove helper function DataFrame.withPlan

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46120: Assignee: Ruifeng Zheng > Remove helper function DataFrame.withPlan >

[jira] [Resolved] (SPARK-46120) Remove helper function DataFrame.withPlan

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46120. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44037

[jira] [Assigned] (SPARK-46123) Using brighter color for document title for better visibility

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46123: Assignee: Haejoon Lee > Using brighter color for document title for better visibility >

[jira] [Resolved] (SPARK-46123) Using brighter color for document title for better visibility

2023-11-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46123. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44040

[jira] [Assigned] (SPARK-46114) Define IndexError for PySpark error framework

2023-11-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46114: Assignee: Hyukjin Kwon > Define IndexError for PySpark error framework >

[jira] [Resolved] (SPARK-46114) Define IndexError for PySpark error framework

2023-11-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46114. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44028

[jira] [Created] (SPARK-46114) Define IndexError for PySpark error framework

2023-11-26 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46114: Summary: Define IndexError for PySpark error framework Key: SPARK-46114 URL: https://issues.apache.org/jira/browse/SPARK-46114 Project: Spark Issue Type:

[jira] [Created] (SPARK-46110) Use error classes in catalog, conf, connect, observation, pandas modules

2023-11-26 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46110: Summary: Use error classes in catalog, conf, connect, observation, pandas modules Key: SPARK-46110 URL: https://issues.apache.org/jira/browse/SPARK-46110 Project:

[jira] [Deleted] (SPARK-46109) Migrate to error classes in PySpark

2023-11-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon deleted SPARK-46109: - > Migrate to error classes in PySpark > --- > >

[jira] [Updated] (SPARK-46109) Migrate to error classes in PySpark

2023-11-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46109: - > Migrate to error classes in PySpark > --- > >

[jira] [Created] (SPARK-46109) Migrate to error classes in PySpark

2023-11-26 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46109: Summary: Migrate to error classes in PySpark Key: SPARK-46109 URL: https://issues.apache.org/jira/browse/SPARK-46109 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-32933) Use keyword-only syntax for keyword_only methods

2023-11-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789874#comment-17789874 ] Hyukjin Kwon commented on SPARK-32933: -- Here the PR and JIRA:

[jira] [Created] (SPARK-46107) Deprecate pyspark.keyword_only API

2023-11-26 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46107: Summary: Deprecate pyspark.keyword_only API Key: SPARK-46107 URL: https://issues.apache.org/jira/browse/SPARK-46107 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-46074) [CONNECT][SCALA] Insufficient details in error when a UDF fails

2023-11-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46074. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43983

[jira] [Assigned] (SPARK-46074) [CONNECT][SCALA] Insufficient details in error when a UDF fails

2023-11-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46074: Assignee: Niranjan Jayakar > [CONNECT][SCALA] Insufficient details in error when a UDF

[jira] [Resolved] (SPARK-45922) Multiple policies follow-up (Python)

2023-11-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45922. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43800

[jira] [Resolved] (SPARK-46016) Fix pandas API support list properly

2023-11-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46016. -- Fix Version/s: 3.4.2 4.0.0 3.5.1 Assignee:

[jira] [Assigned] (SPARK-46082) Fix protobuf string representation for Pandas Functions API with Spark Connect

2023-11-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46082: Assignee: Hyukjin Kwon > Fix protobuf string representation for Pandas Functions API

[jira] [Resolved] (SPARK-46082) Fix protobuf string representation for Pandas Functions API with Spark Connect

2023-11-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46082. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43991

[jira] [Created] (SPARK-46085) Dataset.groupingSets in Scala Spark Connect client

2023-11-23 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46085: Summary: Dataset.groupingSets in Scala Spark Connect client Key: SPARK-46085 URL: https://issues.apache.org/jira/browse/SPARK-46085 Project: Spark Issue

[jira] [Created] (SPARK-46083) Make SparkNoSuchElementException as a canonical error API

2023-11-23 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46083: Summary: Make SparkNoSuchElementException as a canonical error API Key: SPARK-46083 URL: https://issues.apache.org/jira/browse/SPARK-46083 Project: Spark

[jira] [Created] (SPARK-46082) Fix protobuf string representation for Pandas Functions API with Spark Connect

2023-11-23 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46082: Summary: Fix protobuf string representation for Pandas Functions API with Spark Connect Key: SPARK-46082 URL: https://issues.apache.org/jira/browse/SPARK-46082

[jira] [Created] (SPARK-46080) Upgrade Cloudpickle to 3.0.0

2023-11-23 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46080: Summary: Upgrade Cloudpickle to 3.0.0 Key: SPARK-46080 URL: https://issues.apache.org/jira/browse/SPARK-46080 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-46076) Remove `unittest` deprecated alias usage for Python 3.12

2023-11-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46076. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43986

[jira] [Resolved] (SPARK-46065) Refactor `(DataFrame|Series).factorize()` to use `create_map`.

2023-11-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46065. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43970

[jira] [Assigned] (SPARK-46065) Refactor `(DataFrame|Series).factorize()` to use `create_map`.

2023-11-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46065: Assignee: Haejoon Lee > Refactor `(DataFrame|Series).factorize()` to use `create_map`. >

[jira] [Deleted] (SPARK-46049) Support groupingSets operation in PySpark (Spark Connect)

2023-11-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon deleted SPARK-46049: - > Support groupingSets operation in PySpark (Spark Connect) >

[jira] [Updated] (SPARK-46063) Improve error messages related to argument types in cute, rollup, groupby, and pivot

2023-11-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46063: - Description: {code} >>> spark.range(1).cube(cols=1.2) Traceback (most recent call last): File

[jira] [Updated] (SPARK-46063) Improve error messages related to argument types in cute, rollup, groupby, and pivot

2023-11-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46063: - Summary: Improve error messages related to argument types in cute, rollup, groupby, and pivot

[jira] [Created] (SPARK-46063) Improve error messages related to argument types in cute, rollup, and pivot

2023-11-22 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46063: Summary: Improve error messages related to argument types in cute, rollup, and pivot Key: SPARK-46063 URL: https://issues.apache.org/jira/browse/SPARK-46063 Project:

[jira] [Assigned] (SPARK-46061) Add the test party for reattach test case

2023-11-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46061: Assignee: Hyukjin Kwon > Add the test party for reattach test case >

[jira] [Resolved] (SPARK-46061) Add the test party for reattach test case

2023-11-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46061. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43965

[jira] [Resolved] (SPARK-45600) Make Python data source registration session level

2023-11-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45600. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43742

[jira] [Assigned] (SPARK-45600) Make Python data source registration session level

2023-11-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45600: Assignee: Allison Wang > Make Python data source registration session level >

[jira] [Created] (SPARK-46061) Add the test party for reattach test case

2023-11-22 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46061: Summary: Add the test party for reattach test case Key: SPARK-46061 URL: https://issues.apache.org/jira/browse/SPARK-46061 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-46061) Add the test party for reattach test case

2023-11-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46061: - Issue Type: Test (was: New Feature) > Add the test party for reattach test case >

[jira] [Resolved] (SPARK-46048) Support groupingSets operation in PySpark

2023-11-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46048. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43951

[jira] [Assigned] (SPARK-46048) Support groupingSets operation in PySpark

2023-11-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46048: Assignee: Hyukjin Kwon > Support groupingSets operation in PySpark >

[jira] [Updated] (SPARK-46048) Support groupingSets operation in PySpark

2023-11-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46048: - Issue Type: New Feature (was: Bug) > Support groupingSets operation in PySpark >

[jira] [Created] (SPARK-46048) Support groupingSets operation in PySpark

2023-11-21 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46048: Summary: Support groupingSets operation in PySpark Key: SPARK-46048 URL: https://issues.apache.org/jira/browse/SPARK-46048 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-46049) Support groupingSets operation in PySpark (Spark Connect)

2023-11-21 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46049: Summary: Support groupingSets operation in PySpark (Spark Connect) Key: SPARK-46049 URL: https://issues.apache.org/jira/browse/SPARK-46049 Project: Spark

[jira] [Assigned] (SPARK-46017) PySpark doc build doesn't work properly on Mac

2023-11-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46017: Assignee: Haejoon Lee > PySpark doc build doesn't work properly on Mac >

[jira] [Resolved] (SPARK-46022) Remove deprecated functions APIs from documents

2023-11-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46022. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43932

[jira] [Assigned] (SPARK-46022) Remove deprecated functions APIs from documents

2023-11-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46022: Assignee: Haejoon Lee > Remove deprecated functions APIs from documents >

[jira] [Assigned] (SPARK-46013) Improve basic datasource examples

2023-11-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46013: Assignee: Allison Wang > Improve basic datasource examples >

[jira] [Resolved] (SPARK-46017) PySpark doc build doesn't work properly on Mac

2023-11-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46017. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43932

[jira] [Resolved] (SPARK-46013) Improve basic datasource examples

2023-11-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46013. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43917

[jira] [Updated] (SPARK-46042) Reenable a `releaseSession` test case in SparkConnectServiceE2ESuite

2023-11-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46042: - Description: https://github.com/apache/spark/pull/43942#issuecomment-1821896165 > Reenable a

[jira] [Commented] (SPARK-46032) connect: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.spark.rdd.MapPartitionsRDD.f

2023-11-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788430#comment-17788430 ] Hyukjin Kwon commented on SPARK-46032: -- Are executors using the same versions too? The error is

[jira] [Commented] (SPARK-46032) connect: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.spark.rdd.MapPartitionsRDD.f

2023-11-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788381#comment-17788381 ] Hyukjin Kwon commented on SPARK-46032: -- and can you run without Spark Connect? Seems like just

[jira] [Commented] (SPARK-46032) connect: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.spark.rdd.MapPartitionsRDD.f

2023-11-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788380#comment-17788380 ] Hyukjin Kwon commented on SPARK-46032: -- What's your Scala version [~wbo4958]? > connect: cannot

[jira] [Comment Edited] (SPARK-46032) connect: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.spark.rdd.MapPartitionsRDD.f

2023-11-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788380#comment-17788380 ] Hyukjin Kwon edited comment on SPARK-46032 at 11/21/23 11:34 AM: - What's

[jira] [Updated] (SPARK-46032) connect: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.spark.rdd.MapPartitionsRDD.f

2023-11-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46032: - Priority: Major (was: Blocker) > connect: cannot assign instance of

[jira] [Assigned] (SPARK-46023) Annotate parameters at docstrings in pyspark.sql module

2023-11-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46023: Assignee: Hyukjin Kwon > Annotate parameters at docstrings in pyspark.sql module >

[jira] [Resolved] (SPARK-46023) Annotate parameters at docstrings in pyspark.sql module

2023-11-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46023. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43925

[jira] [Assigned] (SPARK-46026) Refine docstring of UDTF

2023-11-21 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46026: Assignee: Hyukjin Kwon > Refine docstring of UDTF > > >

<    5   6   7   8   9   10   11   12   13   14   >