[jira] [Assigned] (SPARK-42507) Simplify ORC schema merging conflict error check

2023-02-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reassigned SPARK-42507: Assignee: Dongjoon Hyun > Simplify ORC schema merging conflict error check >

[jira] [Comment Edited] (SPARK-41661) Support for User-defined Functions in Python

2023-02-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689732#comment-17689732 ] Xinrong Meng edited comment on SPARK-41661 at 2/16/23 12:23 PM: Hi

[jira] [Commented] (SPARK-41661) Support for User-defined Functions in Python

2023-02-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689732#comment-17689732 ] Xinrong Meng commented on SPARK-41661: -- Hi [~grundprinzip-db] the support for pickled Python UDF

[jira] [Updated] (SPARK-42393) Support for Pandas/Arrow Functions API

2023-02-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42393: - Summary: Support for Pandas/Arrow Functions API (was: Support for Pandas Functions API) >

[jira] [Resolved] (SPARK-41661) Support for User-defined Functions in Python

2023-02-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-41661. -- Resolution: Done > Support for User-defined Functions in Python >

[jira] [Assigned] (SPARK-42263) Implement `spark.catalog.registerFunction`

2023-02-13 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reassigned SPARK-42263: Assignee: Xinrong Meng > Implement `spark.catalog.registerFunction` >

[jira] [Resolved] (SPARK-42263) Implement `spark.catalog.registerFunction`

2023-02-13 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-42263. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39984

[jira] [Created] (SPARK-42428) Standardize __repr__ of CommonInlineUserDefinedFunction

2023-02-13 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42428: Summary: Standardize __repr__ of CommonInlineUserDefinedFunction Key: SPARK-42428 URL: https://issues.apache.org/jira/browse/SPARK-42428 Project: Spark

[jira] [Updated] (SPARK-42263) Implement `spark.catalog.registerFunction`

2023-02-12 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42263: - Parent: SPARK-41661 Issue Type: Sub-task (was: Improvement) > Implement

[jira] [Updated] (SPARK-42247) Standardize `returnType` property of UserDefinedFunction

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42247: - Affects Version/s: 3.4.0 (was: 3.5.0) > Standardize `returnType`

[jira] [Updated] (SPARK-42247) Standardize `returnType` property of UserDefinedFunction

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42247: - Parent: SPARK-42393 Issue Type: Sub-task (was: Improvement) > Standardize `returnType`

[jira] [Updated] (SPARK-42393) Support for Pandas Functions API

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42393: - Affects Version/s: 3.4.0 > Support for Pandas Functions API >

[jira] [Updated] (SPARK-42247) Standardize `returnType` property of UserDefinedFunction

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42247: - Affects Version/s: 3.5.0 (was: 3.4.0) > Standardize `returnType`

[jira] [Updated] (SPARK-42340) Implement GroupedData.applyInPandas

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42340: - Epic Link: (was: SPARK-39375) > Implement GroupedData.applyInPandas >

[jira] [Updated] (SPARK-42340) Implement GroupedData.applyInPandas

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42340: - Parent: SPARK-42393 Issue Type: Sub-task (was: New Feature) > Implement

[jira] [Created] (SPARK-42393) Support for Pandas Functions API

2023-02-09 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42393: Summary: Support for Pandas Functions API Key: SPARK-42393 URL: https://issues.apache.org/jira/browse/SPARK-42393 Project: Spark Issue Type: Umbrella

[jira] [Resolved] (SPARK-42264) Test Parity: pyspark.sql.tests.test_udf and pyspark.sql.tests.pandas.test_pandas_udf

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-42264. -- Resolution: Fixed > Test Parity: pyspark.sql.tests.test_udf and >

[jira] [Assigned] (SPARK-42264) Test Parity: pyspark.sql.tests.test_udf and pyspark.sql.tests.pandas.test_pandas_udf

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reassigned SPARK-42264: Assignee: Xinrong Meng > Test Parity: pyspark.sql.tests.test_udf and >

[jira] [Resolved] (SPARK-42246) Reach Full Parity with Vanilla PySpark's UDF

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-42246. -- Resolution: Resolved > Reach Full Parity with Vanilla PySpark's UDF >

[jira] [Updated] (SPARK-42263) Implement `spark.catalog.registerFunction`

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42263: - Parent: (was: SPARK-42246) Issue Type: Improvement (was: Sub-task) > Implement

[jira] [Updated] (SPARK-42247) Standardize `returnType` property of UserDefinedFunction

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42247: - Parent: (was: SPARK-42246) Issue Type: Improvement (was: Sub-task) > Standardize

[jira] [Updated] (SPARK-42269) Support complex return types in DDL strings

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42269: - Parent: SPARK-41661 Issue Type: Sub-task (was: Improvement) > Support complex return

[jira] [Updated] (SPARK-42269) Support complex return types in DDL strings

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42269: - Parent: (was: SPARK-42246) Issue Type: Improvement (was: Sub-task) > Support

[jira] [Updated] (SPARK-42247) Standardize `returnType` property of UserDefinedFunction

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42247: - Description: There are checks (was: The inconsistency can be reproduced as shown below:

[jira] [Updated] (SPARK-42246) Reach Full Parity with Vanilla PySpark's UDF

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42246: - Summary: Reach Full Parity with Vanilla PySpark's UDF (was: Reach Full Parity with Vanilla

[jira] [Updated] (SPARK-42269) Support complex return types in DDL strings

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42269: - Summary: Support complex return types in DDL strings (was: Support user-specified return type

[jira] [Updated] (SPARK-42247) Standardize `returnType` property of UserDefinedFunction

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42247: - Summary: Standardize `returnType` property of UserDefinedFunction (was: `returnType` attribute

[jira] [Resolved] (SPARK-42211) Python UDFs with inconsistent client and server versions

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-42211. -- Resolution: Duplicate > Python UDFs with inconsistent client and server versions >

[jira] [Resolved] (SPARK-42210) Standardize registered pickled Python UDFs

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-42210. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39860

[jira] [Assigned] (SPARK-42210) Standardize registered pickled Python UDFs

2023-02-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reassigned SPARK-42210: Assignee: Xinrong Meng > Standardize registered pickled Python UDFs >

[jira] [Updated] (SPARK-42340) Implement GroupedData.applyInPandas

2023-02-05 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42340: - Epic Link: SPARK-39375 > Implement GroupedData.applyInPandas >

[jira] [Updated] (SPARK-42340) Implement GroupedData.applyInPandas

2023-02-05 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42340: - Parent: (was: SPARK-41661) Issue Type: New Feature (was: Sub-task) > Implement

[jira] [Resolved] (SPARK-42125) Pandas UDF in Spark Connect

2023-01-31 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-42125. -- Resolution: Resolved Resolved by https://github.com/apache/spark/pull/39753 > Pandas UDF in

[jira] [Assigned] (SPARK-42125) Pandas UDF in Spark Connect

2023-01-31 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reassigned SPARK-42125: Assignee: Xinrong Meng > Pandas UDF in Spark Connect > --- > >

[jira] [Created] (SPARK-42271) Reuse UDF test cases under `pyspark.sql.tests`

2023-01-31 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42271: Summary: Reuse UDF test cases under `pyspark.sql.tests` Key: SPARK-42271 URL: https://issues.apache.org/jira/browse/SPARK-42271 Project: Spark Issue Type:

[jira] [Updated] (SPARK-42269) Support user-specified return type as a collection DataType in DDL strings

2023-01-31 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42269: - Summary: Support user-specified return type as a collection DataType in DDL strings (was:

[jira] [Created] (SPARK-42269) Support user-specified return type as a collection DataType

2023-01-31 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42269: Summary: Support user-specified return type as a collection DataType Key: SPARK-42269 URL: https://issues.apache.org/jira/browse/SPARK-42269 Project: Spark

[jira] [Created] (SPARK-42267) Support left_outer join

2023-01-31 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42267: Summary: Support left_outer join Key: SPARK-42267 URL: https://issues.apache.org/jira/browse/SPARK-42267 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-42265) DataFrame.createTempView - SparkConnectGrpcException: requirement failed

2023-01-31 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42265: Summary: DataFrame.createTempView - SparkConnectGrpcException: requirement failed Key: SPARK-42265 URL: https://issues.apache.org/jira/browse/SPARK-42265 Project:

[jira] [Assigned] (SPARK-42208) Reuse UDF test cases under `pyspark.sql.tests`

2023-01-31 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reassigned SPARK-42208: Assignee: Xinrong Meng > Reuse UDF test cases under `pyspark.sql.tests` >

[jira] [Resolved] (SPARK-42208) Reuse UDF test cases under `pyspark.sql.tests`

2023-01-31 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-42208. -- Resolution: Duplicate Duplicated by https://issues.apache.org/jira/browse/SPARK-42264 >

[jira] [Created] (SPARK-42264) Test Parity: pyspark.sql.tests.test_udf and pyspark.sql.tests.pandas.test_pandas_udf

2023-01-31 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42264: Summary: Test Parity: pyspark.sql.tests.test_udf and pyspark.sql.tests.pandas.test_pandas_udf Key: SPARK-42264 URL: https://issues.apache.org/jira/browse/SPARK-42264

[jira] [Updated] (SPARK-42210) Standardize registered pickled Python UDFs

2023-01-31 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42210: - Description: Implement spark.udf. > Standardize registered pickled Python UDFs >

[jira] [Updated] (SPARK-42263) Implement `spark.catalog.registerFunction`

2023-01-31 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42263: - Summary: Implement `spark.catalog.registerFunction` (was: Implement `spark.udf`) > Implement

[jira] [Updated] (SPARK-42263) Implement `spark.udf`

2023-01-31 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42263: - Summary: Implement `spark.udf` (was: Implement `spark.catalog.registerFunction`) > Implement

[jira] [Created] (SPARK-42263) Implement `spark.catalog.registerFunction`

2023-01-31 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42263: Summary: Implement `spark.catalog.registerFunction` Key: SPARK-42263 URL: https://issues.apache.org/jira/browse/SPARK-42263 Project: Spark Issue Type:

[jira] [Created] (SPARK-42247) `returnType` attribute of UDF when the user-specified return type has column name embeded

2023-01-30 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42247: Summary: `returnType` attribute of UDF when the user-specified return type has column name embeded Key: SPARK-42247 URL: https://issues.apache.org/jira/browse/SPARK-42247

[jira] [Created] (SPARK-42246) Reach Full Parity with Vanilla PySpark's UDF in Python

2023-01-30 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42246: Summary: Reach Full Parity with Vanilla PySpark's UDF in Python Key: SPARK-42246 URL: https://issues.apache.org/jira/browse/SPARK-42246 Project: Spark Issue

[jira] [Commented] (SPARK-41661) Support for User-defined Functions in Python

2023-01-27 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681221#comment-17681221 ] Xinrong Meng commented on SPARK-41661: -- Hi [~grundprinzip-db], I renamed the ticket for clarity to

[jira] [Updated] (SPARK-41661) Support for User-defined Functions in Python

2023-01-27 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-41661: - Summary: Support for User-defined Functions in Python (was: Support for Python UDFs) >

[jira] [Created] (SPARK-42211) Python UDFs with inconsistent client and server versions

2023-01-27 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42211: Summary: Python UDFs with inconsistent client and server versions Key: SPARK-42211 URL: https://issues.apache.org/jira/browse/SPARK-42211 Project: Spark

[jira] [Created] (SPARK-42210) Standardize registered pickled Python UDFs

2023-01-26 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42210: Summary: Standardize registered pickled Python UDFs Key: SPARK-42210 URL: https://issues.apache.org/jira/browse/SPARK-42210 Project: Spark Issue Type:

[jira] [Created] (SPARK-42208) Reuse UDF test cases under `pyspark.sql.tests`

2023-01-26 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42208: Summary: Reuse UDF test cases under `pyspark.sql.tests` Key: SPARK-42208 URL: https://issues.apache.org/jira/browse/SPARK-42208 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-41662) Minimal support for pickled Python UDFs

2023-01-26 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-41662. -- Resolution: Duplicate > Minimal support for pickled Python UDFs >

[jira] [Commented] (SPARK-42126) Accept return type in DDL strings for Python Scalar UDFs in Spark Connect

2023-01-26 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680954#comment-17680954 ] Xinrong Meng commented on SPARK-42126: -- Resolved by https://github.com/apache/spark/pull/39739. >

[jira] [Resolved] (SPARK-42126) Accept return type in DDL strings for Python Scalar UDFs in Spark Connect

2023-01-26 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-42126. -- Resolution: Resolved > Accept return type in DDL strings for Python Scalar UDFs in Spark

[jira] [Assigned] (SPARK-42126) Accept return type in DDL strings for Python Scalar UDFs in Spark Connect

2023-01-26 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reassigned SPARK-42126: Assignee: Xinrong Meng > Accept return type in DDL strings for Python Scalar UDFs in

[jira] [Updated] (SPARK-42126) Accept return type in DDL strings for Python Scalar UDFs in Spark Connect

2023-01-25 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42126: - Summary: Accept return type in DDL strings for Python Scalar UDFs in Spark Connect (was:

[jira] [Updated] (SPARK-35996) Setting version to 3.3.0-SNAPSHOT

2023-01-25 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35996: - Description: (was: Start to prepare Apache Spark 3.5.0 and the published snapshot version

[jira] [Updated] (SPARK-42184) Setting version to 3.5.0-SNAPSHOT

2023-01-25 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42184: - Description: Start to prepare Apache Spark 3.5.0 and the published snapshot version should not

[jira] [Updated] (SPARK-35996) Setting version to 3.3.0-SNAPSHOT

2023-01-25 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-35996: - Description: Start to prepare Apache Spark 3.5.0 and the published snapshot version should not

[jira] [Created] (SPARK-42184) Setting version to 3.5.0-SNAPSHOT

2023-01-25 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42184: Summary: Setting version to 3.5.0-SNAPSHOT Key: SPARK-42184 URL: https://issues.apache.org/jira/browse/SPARK-42184 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-42126) Support user-specified return type in DDL-formatted string

2023-01-19 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42126: Summary: Support user-specified return type in DDL-formatted string Key: SPARK-42126 URL: https://issues.apache.org/jira/browse/SPARK-42126 Project: Spark

[jira] [Created] (SPARK-42125) Pandas UDF in Spark Connect

2023-01-19 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42125: Summary: Pandas UDF in Spark Connect Key: SPARK-42125 URL: https://issues.apache.org/jira/browse/SPARK-42125 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-42124) Scalar Inline Python UDF in Spark Connect

2023-01-19 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42124: Summary: Scalar Inline Python UDF in Spark Connect Key: SPARK-42124 URL: https://issues.apache.org/jira/browse/SPARK-42124 Project: Spark Issue Type:

[jira] [Updated] (SPARK-42095) Fix gRPC check in tests

2023-01-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42095: - Description: Fix gRPC check in tests, including variables and error messages. > Fix gRPC check

[jira] [Updated] (SPARK-42095) Fix gRPC check in tests

2023-01-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-42095: - Summary: Fix gRPC check in tests (was: gRPC check in tests) > Fix gRPC check in tests >

[jira] [Created] (SPARK-42095) gRPC check in tests

2023-01-16 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-42095: Summary: gRPC check in tests Key: SPARK-42095 URL: https://issues.apache.org/jira/browse/SPARK-42095 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-40307) Introduce Arrow-optimized Python UDFs

2023-01-12 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-40307. -- Assignee: Xinrong Meng Resolution: Resolved > Introduce Arrow-optimized Python UDFs >

[jira] [Commented] (SPARK-40307) Introduce Arrow-optimized Python UDFs

2023-01-12 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675983#comment-17675983 ] Xinrong Meng commented on SPARK-40307: -- Resolved by https://github.com/apache/spark/pull/39384. >

[jira] [Updated] (SPARK-40307) Introduce Arrow-optimized Python UDFs

2023-01-03 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40307: - Description: Python user-defined function (UDF) enables users to run arbitrary code against

[jira] [Updated] (SPARK-40307) Introduce Arrow-optimized Python UDFs

2023-01-03 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40307: - Description: Python user-defined function (UDF) enables users to run arbitrary code against

[jira] [Updated] (SPARK-40307) Introduce Arrow-optimized Python UDFs

2023-01-03 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40307: - Description: Python user-defined function (UDF) enables users to run arbitrary code against

[jira] [Updated] (SPARK-40307) Introduce Arrow-optimized Python UDFs

2023-01-03 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40307: - Summary: Introduce Arrow-optimized Python UDFs (was: Optimize (De)Serialization of Python UDFs

[jira] [Updated] (SPARK-40307) Optimize (De)Serialization of Python UDFs by Arrow

2023-01-03 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40307: - Description: Python user-defined function (UDF) enables users to run arbitrary code against

[jira] [Updated] (SPARK-40307) Optimize (De)Serialization of Python UDFs by Arrow

2023-01-03 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-40307: - Summary: Optimize (De)Serialization of Python UDFs by Arrow (was: Optimize (De)Serialization

[jira] [Created] (SPARK-41473) Implement `functions.format_number`

2022-12-09 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-41473: Summary: Implement `functions.format_number` Key: SPARK-41473 URL: https://issues.apache.org/jira/browse/SPARK-41473 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-41283) Feature parity: Functions API in Spark Connect

2022-12-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reassigned SPARK-41283: Assignee: Ruifeng Zheng (was: Xinrong Meng) > Feature parity: Functions API in Spark

[jira] [Created] (SPARK-41472) Implement the rest of string/binary functions

2022-12-09 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-41472: Summary: Implement the rest of string/binary functions Key: SPARK-41472 URL: https://issues.apache.org/jira/browse/SPARK-41472 Project: Spark Issue Type:

[jira] [Updated] (SPARK-41455) Resolve dtypes inconsistencies of date/timestamp functions

2022-12-08 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-41455: - Description: When implementing date/timestamp functions, we notice inconsistent dtypes with

[jira] [Updated] (SPARK-41455) Resolve dtypes inconsistencies of date/timestamp functions

2022-12-08 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-41455: - Description: When implementing date/timestamp functions, we notice inconsistent dtypes with

[jira] [Updated] (SPARK-41455) Resolve dtypes inconsistencies of date/timestamp functions

2022-12-08 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-41455: - Description: When implementing date/timestamp functions, we notice inconsistent dtypes with

[jira] [Created] (SPARK-41455) Resolve dtypes inconsistencies of date/timestamp functions

2022-12-08 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-41455: Summary: Resolve dtypes inconsistencies of date/timestamp functions Key: SPARK-41455 URL: https://issues.apache.org/jira/browse/SPARK-41455 Project: Spark

[jira] [Updated] (SPARK-41414) Implement date/timestamp functions

2022-12-06 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-41414: - Summary: Implement date/timestamp functions (was: Implement data/timestamp functions) >

[jira] [Created] (SPARK-41414) Implement data/timestamp functions

2022-12-06 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-41414: Summary: Implement data/timestamp functions Key: SPARK-41414 URL: https://issues.apache.org/jira/browse/SPARK-41414 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41402) Override nodeName of StringDecode

2022-12-05 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-41402: Summary: Override nodeName of StringDecode Key: SPARK-41402 URL: https://issues.apache.org/jira/browse/SPARK-41402 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-41397) Implement part of String/Binary functions

2022-12-05 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-41397: Summary: Implement part of String/Binary functions Key: SPARK-41397 URL: https://issues.apache.org/jira/browse/SPARK-41397 Project: Spark Issue Type:

[jira] [Updated] (SPARK-41397) Implement part of string/binary functions

2022-12-05 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-41397: - Summary: Implement part of string/binary functions (was: Implement part of String/Binary

[jira] [Commented] (SPARK-41372) Support DataFrame TempView

2022-12-05 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643473#comment-17643473 ] Xinrong Meng commented on SPARK-41372: -- Resolved by https://github.com/apache/spark/pull/38891. >

[jira] [Resolved] (SPARK-41372) Support DataFrame TempView

2022-12-05 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-41372. -- Resolution: Resolved > Support DataFrame TempView > -- > >

[jira] [Updated] (SPARK-41354) Implement `DataFrame.repartitionByRange`

2022-12-01 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-41354: - Summary: Implement `DataFrame.repartitionByRange` (was: Support

[jira] [Created] (SPARK-41354) Support `DataFrame.repartitionByRange`

2022-12-01 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-41354: Summary: Support `DataFrame.repartitionByRange` Key: SPARK-41354 URL: https://issues.apache.org/jira/browse/SPARK-41354 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-41227) Implement DataFrame cross join

2022-11-28 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-41227: - Description: Implement DataFrame cross join for Spark Connect. That consists of -

[jira] [Updated] (SPARK-41227) Implement DataFrame cross join

2022-11-28 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-41227: - Summary: Implement DataFrame cross join (was: Implement `DataFrame.crossJoin`) > Implement

[jira] [Updated] (SPARK-41227) Implement DataFrame cross join

2022-11-28 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-41227: - Description: Implement DataFrame cross join for Spark Connect. That consists of -

[jira] [Created] (SPARK-41243) Update the protobuf version in README

2022-11-23 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-41243: Summary: Update the protobuf version in README Key: SPARK-41243 URL: https://issues.apache.org/jira/browse/SPARK-41243 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-41227) Implement `DataFrame.crossJoin`

2022-11-22 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-41227: Summary: Implement `DataFrame.crossJoin` Key: SPARK-41227 URL: https://issues.apache.org/jira/browse/SPARK-41227 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-41150) Document debugging with PySpark memory profiler

2022-11-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-41150: - Component/s: Documentation > Document debugging with PySpark memory profiler >

[jira] [Updated] (SPARK-41150) Document debugging with PySpark memory profiler

2022-11-15 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-41150: - Description: Document how to debug with PySpark memory profiler on

[jira] [Updated] (SPARK-41150) Document debugging with PySpark memory profiler

2022-11-15 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-41150: - Summary: Document debugging with PySpark memory profiler (was: Document PySpark memory

<    1   2   3   4   5   6   7   8   9   10   >