[jira] [Resolved] (SPARK-48553) Cache more properties

[ https://issues.apache.org/jira/browse/SPARK-48553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48553. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46896

[jira] [Created] (SPARK-48553) Cache more properties

Ruifeng Zheng created SPARK-48553: - Summary: Cache more properties Key: SPARK-48553 URL: https://issues.apache.org/jira/browse/SPARK-48553 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-48550) Directly use the parent Window class

Ruifeng Zheng created SPARK-48550: - Summary: Directly use the parent Window class Key: SPARK-48550 URL: https://issues.apache.org/jira/browse/SPARK-48550 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-48536) Cache user specified schema in applyInPandas and applyInArrow

[ https://issues.apache.org/jira/browse/SPARK-48536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48536. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46877

[jira] [Updated] (SPARK-48508) Client Side RPC optimization for Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-48508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-48508: -- Labels: (was: pull-request-available) > Client Side RPC optimization for Spark Connect >

[jira] [Created] (SPARK-48533) Add test for cached schema

Ruifeng Zheng created SPARK-48533: - Summary: Add test for cached schema Key: SPARK-48533 URL: https://issues.apache.org/jira/browse/SPARK-48533 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-48512) Refactor Python tests

[ https://issues.apache.org/jira/browse/SPARK-48512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48512. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46852

[jira] [Created] (SPARK-48509) Cache user specified schema in DataFrame.{to, mapInPandas, mapInArrow}

Ruifeng Zheng created SPARK-48509: - Summary: Cache user specified schema in DataFrame.{to, mapInPandas, mapInArrow} Key: SPARK-48509 URL: https://issues.apache.org/jira/browse/SPARK-48509 Project:

[jira] [Created] (SPARK-48508) Client Side RPC optimization for Spark Connect

Ruifeng Zheng created SPARK-48508: - Summary: Client Side RPC optimization for Spark Connect Key: SPARK-48508 URL: https://issues.apache.org/jira/browse/SPARK-48508 Project: Spark Issue Type:

[jira] [Updated] (SPARK-48508) Client Side RPC optimization for Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-48508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-48508: -- Issue Type: Umbrella (was: Improvement) > Client Side RPC optimization for Spark Connect >

[jira] [Created] (SPARK-48504) Parent Window class for Spark Connect and Spark Classic

Ruifeng Zheng created SPARK-48504: - Summary: Parent Window class for Spark Connect and Spark Classic Key: SPARK-48504 URL: https://issues.apache.org/jira/browse/SPARK-48504 Project: Spark

[jira] [Reopened] (SPARK-48415) TypeName support parameterized datatypes

[ https://issues.apache.org/jira/browse/SPARK-48415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reopened SPARK-48415: --- > TypeName support parameterized datatypes > > >

[jira] [Reopened] (SPARK-48322) Drop internal metadata in `DataFrame.schema`

[ https://issues.apache.org/jira/browse/SPARK-48322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reopened SPARK-48322: --- > Drop internal metadata in `DataFrame.schema` > >

[jira] [Assigned] (SPARK-48322) Drop internal metadata in `DataFrame.schema`

[ https://issues.apache.org/jira/browse/SPARK-48322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-48322: - Assignee: (was: Ruifeng Zheng) > Drop internal metadata in `DataFrame.schema` >

[jira] [Reopened] (SPARK-42965) metadata mismatch for StructField when running some tests.

[ https://issues.apache.org/jira/browse/SPARK-42965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reopened SPARK-42965: --- Assignee: (was: Ruifeng Zheng) > metadata mismatch for StructField when running some

[jira] [Created] (SPARK-48434) Make printSchema use the cached schema

Ruifeng Zheng created SPARK-48434: - Summary: Make printSchema use the cached schema Key: SPARK-48434 URL: https://issues.apache.org/jira/browse/SPARK-48434 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-48434) Make printSchema use the cached schema

[ https://issues.apache.org/jira/browse/SPARK-48434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-48434: -- Issue Type: Improvement (was: New Feature) > Make printSchema use the cached schema >

[jira] [Resolved] (SPARK-48415) TypeName support parameterized datatypes

[ https://issues.apache.org/jira/browse/SPARK-48415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48415. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46738

[jira] [Created] (SPARK-48415) TypeName support parameterized datatypes

Ruifeng Zheng created SPARK-48415: - Summary: TypeName support parameterized datatypes Key: SPARK-48415 URL: https://issues.apache.org/jira/browse/SPARK-48415 Project: Spark Issue Type: New

[jira] [Resolved] (SPARK-48412) Refactor data type json parse

[ https://issues.apache.org/jira/browse/SPARK-48412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48412. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46733

[jira] [Created] (SPARK-48412) Refactor data type json parse

Ruifeng Zheng created SPARK-48412: - Summary: Refactor data type json parse Key: SPARK-48412 URL: https://issues.apache.org/jira/browse/SPARK-48412 Project: Spark Issue Type: New Feature

[jira] [Resolved] (SPARK-48395) Fix StructType.treeString for parameterized types

[ https://issues.apache.org/jira/browse/SPARK-48395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48395. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46711

[jira] [Created] (SPARK-48393) Move a group of constants to `pyspark.util`

Ruifeng Zheng created SPARK-48393: - Summary: Move a group of constants to `pyspark.util` Key: SPARK-48393 URL: https://issues.apache.org/jira/browse/SPARK-48393 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-45716) Python parity method StructType.treeString

[ https://issues.apache.org/jira/browse/SPARK-45716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-45716. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46685

[jira] [Resolved] (SPARK-48372) Implement `StructType.treeString`

[ https://issues.apache.org/jira/browse/SPARK-48372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48372. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46685

[jira] [Created] (SPARK-48372) Implement `StructType.treeString`

Ruifeng Zheng created SPARK-48372: - Summary: Implement `StructType.treeString` Key: SPARK-48372 URL: https://issues.apache.org/jira/browse/SPARK-48372 Project: Spark Issue Type: New Feature

[jira] [Resolved] (SPARK-48369) Add function timestamp_add

[ https://issues.apache.org/jira/browse/SPARK-48369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48369. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46680

[jira] [Created] (SPARK-48369) Add function timestamp_add

Ruifeng Zheng created SPARK-48369: - Summary: Add function timestamp_add Key: SPARK-48369 URL: https://issues.apache.org/jira/browse/SPARK-48369 Project: Spark Issue Type: New Feature

[jira] [Resolved] (SPARK-48335) Make `_parse_datatype_string` compatible with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-48335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48335. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46657

[jira] [Assigned] (SPARK-48335) Make `_parse_datatype_string` compatible with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-48335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-48335: - Assignee: Ruifeng Zheng > Make `_parse_datatype_string` compatible with Spark Connect

[jira] [Created] (SPARK-48335) Make `_parse_datatype_string` compatible with Spark Connect

Ruifeng Zheng created SPARK-48335: - Summary: Make `_parse_datatype_string` compatible with Spark Connect Key: SPARK-48335 URL: https://issues.apache.org/jira/browse/SPARK-48335 Project: Spark

[jira] [Updated] (SPARK-48335) Make `_parse_datatype_string` compatible with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-48335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-48335: -- Issue Type: Improvement (was: Bug) > Make `_parse_datatype_string` compatible with Spark

[jira] [Created] (SPARK-48333) Test `test_sorting_functions_with_column` with same `Column`

Ruifeng Zheng created SPARK-48333: - Summary: Test `test_sorting_functions_with_column` with same `Column` Key: SPARK-48333 URL: https://issues.apache.org/jira/browse/SPARK-48333 Project: Spark

[jira] [Resolved] (SPARK-48321) Avoid using deprecated methods in dsl

[ https://issues.apache.org/jira/browse/SPARK-48321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48321. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46635

[jira] [Created] (SPARK-48322) Drop internal metadata in `DataFrame.schema`

Ruifeng Zheng created SPARK-48322: - Summary: Drop internal metadata in `DataFrame.schema` Key: SPARK-48322 URL: https://issues.apache.org/jira/browse/SPARK-48322 Project: Spark Issue Type:

[jira] [Created] (SPARK-48321) Avoid using deprecated methods in dsl

Ruifeng Zheng created SPARK-48321: - Summary: Avoid using deprecated methods in dsl Key: SPARK-48321 URL: https://issues.apache.org/jira/browse/SPARK-48321 Project: Spark Issue Type:

[jira] [Created] (SPARK-48319) Test `assert_true` and `raise_error` with more specific error class

Ruifeng Zheng created SPARK-48319: - Summary: Test `assert_true` and `raise_error` with more specific error class Key: SPARK-48319 URL: https://issues.apache.org/jira/browse/SPARK-48319 Project: Spark

[jira] [Updated] (SPARK-48319) Test `assert_true` and `raise_error` with the same error class as Spark Classic

[ https://issues.apache.org/jira/browse/SPARK-48319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-48319: -- Summary: Test `assert_true` and `raise_error` with the same error class as Spark Classic

[jira] [Resolved] (SPARK-48301) Rename CREATE_FUNC_WITH_IF_NOT_EXISTS_AND_REPLACE to CREATE_ROUTINE_WITH_IF_NOT_EXISTS_AND_REPLACE

[ https://issues.apache.org/jira/browse/SPARK-48301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48301. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46608

[jira] [Assigned] (SPARK-48287) Apply the builtin `timestamp_diff` method

[ https://issues.apache.org/jira/browse/SPARK-48287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-48287: - Assignee: Ruifeng Zheng > Apply the builtin `timestamp_diff` method >

[jira] [Resolved] (SPARK-48287) Apply the builtin `timestamp_diff` method

[ https://issues.apache.org/jira/browse/SPARK-48287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48287. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46595

[jira] [Created] (SPARK-48295) Turn on compute.ops_on_diff_frames by default

Ruifeng Zheng created SPARK-48295: - Summary: Turn on compute.ops_on_diff_frames by default Key: SPARK-48295 URL: https://issues.apache.org/jira/browse/SPARK-48295 Project: Spark Issue Type:

[jira] [Created] (SPARK-48287) Apply the builtin `timestamp_diff` method

Ruifeng Zheng created SPARK-48287: - Summary: Apply the builtin `timestamp_diff` method Key: SPARK-48287 URL: https://issues.apache.org/jira/browse/SPARK-48287 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-48278) Refine the string representation of `Cast`

[ https://issues.apache.org/jira/browse/SPARK-48278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48278. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46585

[jira] [Assigned] (SPARK-48272) Add function `timestamp_diff`

[ https://issues.apache.org/jira/browse/SPARK-48272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-48272: - Assignee: Ruifeng Zheng > Add function `timestamp_diff` >

[jira] [Resolved] (SPARK-48272) Add function `timestamp_diff`

[ https://issues.apache.org/jira/browse/SPARK-48272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48272. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46576

[jira] [Updated] (SPARK-48276) Add the missing __repr__ method for SQLExpression

[ https://issues.apache.org/jira/browse/SPARK-48276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-48276: -- Fix Version/s: 4.0.0 > Add the missing __repr__ method for SQLExpression >

[jira] [Created] (SPARK-48278) Refine the string representation of `Cast`

Ruifeng Zheng created SPARK-48278: - Summary: Refine the string representation of `Cast` Key: SPARK-48278 URL: https://issues.apache.org/jira/browse/SPARK-48278 Project: Spark Issue Type:

[jira] [Created] (SPARK-48272) Add function `timestamp_diff`

Ruifeng Zheng created SPARK-48272: - Summary: Add function `timestamp_diff` Key: SPARK-48272 URL: https://issues.apache.org/jira/browse/SPARK-48272 Project: Spark Issue Type: New Feature

[jira] [Resolved] (SPARK-48259) Add 3 missing methods in dsl

[ https://issues.apache.org/jira/browse/SPARK-48259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48259. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46559

[jira] [Created] (SPARK-48259) Add 3 missing methods in dsl

Ruifeng Zheng created SPARK-48259: - Summary: Add 3 missing methods in dsl Key: SPARK-48259 URL: https://issues.apache.org/jira/browse/SPARK-48259 Project: Spark Issue Type: Test

[jira] [Created] (SPARK-48228) Implement the missing function validation in ApplyInXXX

Ruifeng Zheng created SPARK-48228: - Summary: Implement the missing function validation in ApplyInXXX Key: SPARK-48228 URL: https://issues.apache.org/jira/browse/SPARK-48228 Project: Spark

[jira] [Created] (SPARK-48227) Document the requirement of seed in protos

Ruifeng Zheng created SPARK-48227: - Summary: Document the requirement of seed in protos Key: SPARK-48227 URL: https://issues.apache.org/jira/browse/SPARK-48227 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-48190) Introduce a helper function to drop metadata

[ https://issues.apache.org/jira/browse/SPARK-48190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48190. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46466

[jira] [Updated] (SPARK-48184) Always set the seed of dataframe.sample in Client side

[ https://issues.apache.org/jira/browse/SPARK-48184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-48184: -- Description: In Spark Classic: In [1]: df = spark.range(1).sample(0.1) In [2]:

[jira] [Assigned] (SPARK-48142) Enable `CogroupedApplyInPandasTests.test_wrong_args`

[ https://issues.apache.org/jira/browse/SPARK-48142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-48142: - Assignee: Ruifeng Zheng > Enable `CogroupedApplyInPandasTests.test_wrong_args` >

[jira] [Resolved] (SPARK-48142) Enable `CogroupedApplyInPandasTests.test_wrong_args`

[ https://issues.apache.org/jira/browse/SPARK-48142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48142. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46397

[jira] [Created] (SPARK-48142) Enable `CogroupedApplyInPandasTests.test_wrong_args`

Ruifeng Zheng created SPARK-48142: - Summary: Enable `CogroupedApplyInPandasTests.test_wrong_args` Key: SPARK-48142 URL: https://issues.apache.org/jira/browse/SPARK-48142 Project: Spark Issue

[jira] [Created] (SPARK-48058) `UserDefinedFunction.returnType` parse the DDL string

Ruifeng Zheng created SPARK-48058: - Summary: `UserDefinedFunction.returnType` parse the DDL string Key: SPARK-48058 URL: https://issues.apache.org/jira/browse/SPARK-48058 Project: Spark

[jira] [Resolved] (SPARK-48055) Enable PandasUDFScalarParityTests.{test_vectorized_udf_empty_partition, test_vectorized_udf_struct_with_empty_partition}

[ https://issues.apache.org/jira/browse/SPARK-48055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48055. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46296

[jira] [Updated] (SPARK-47129) Make ResolveRelations cache connect plan properly

[ https://issues.apache.org/jira/browse/SPARK-47129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47129: -- Issue Type: Bug (was: Improvement) > Make ResolveRelations cache connect plan properly >

[jira] [Updated] (SPARK-47129) Make ResolveRelations cache connect plan properly

[ https://issues.apache.org/jira/browse/SPARK-47129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47129: -- Affects Version/s: 3.4.3 3.5.1 > Make ResolveRelations cache connect

[jira] [Created] (SPARK-48020) Pin 'pandas==2.2.2'

Ruifeng Zheng created SPARK-48020: - Summary: Pin 'pandas==2.2.2' Key: SPARK-48020 URL: https://issues.apache.org/jira/browse/SPARK-48020 Project: Spark Issue Type: Bug Components:

[jira] [Created] (SPARK-48005) Enable `DefaultIndexParityTests. test_index_distributed_sequence_cleanup`

Ruifeng Zheng created SPARK-48005: - Summary: Enable `DefaultIndexParityTests. test_index_distributed_sequence_cleanup` Key: SPARK-48005 URL: https://issues.apache.org/jira/browse/SPARK-48005 Project:

[jira] [Resolved] (SPARK-47986) [CONNECT][PYTHON] Unable to create a new session when the default session is closed by the server

[ https://issues.apache.org/jira/browse/SPARK-47986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47986. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46221

[jira] [Resolved] (SPARK-47985) Simplify functions with `lit`

[ https://issues.apache.org/jira/browse/SPARK-47985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47985. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46219

[jira] [Assigned] (SPARK-47985) Simplify functions with `lit`

[ https://issues.apache.org/jira/browse/SPARK-47985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47985: - Assignee: Ruifeng Zheng > Simplify functions with `lit` >

[jira] [Updated] (SPARK-47623) Enable `QuietTest` in parity tests

[ https://issues.apache.org/jira/browse/SPARK-47623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47623: -- Summary: Enable `QuietTest` in parity tests (was: Use `QuietTest` in parity tests) > Enable

[jira] [Created] (SPARK-47987) Reenable `ArrowParityTests.test_createDataFrame_empty_partition`

Ruifeng Zheng created SPARK-47987: - Summary: Reenable `ArrowParityTests.test_createDataFrame_empty_partition` Key: SPARK-47987 URL: https://issues.apache.org/jira/browse/SPARK-47987 Project: Spark

[jira] [Updated] (SPARK-47970) Revisit skipped parity tests for PySpark Connect

[ https://issues.apache.org/jira/browse/SPARK-47970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47970: -- Summary: Revisit skipped parity tests for PySpark Connect (was: Revisit skipped parity tests

[jira] [Created] (SPARK-47985) Simplify functions with `lit`

Ruifeng Zheng created SPARK-47985: - Summary: Simplify functions with `lit` Key: SPARK-47985 URL: https://issues.apache.org/jira/browse/SPARK-47985 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-47499) Reuse `test_help_command` in Connect

[ https://issues.apache.org/jira/browse/SPARK-47499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47499: -- Parent: SPARK-47970 Issue Type: Sub-task (was: Test) > Reuse `test_help_command` in

[jira] [Created] (SPARK-47970) Revisit skipped parity tests for PySpark

Ruifeng Zheng created SPARK-47970: - Summary: Revisit skipped parity tests for PySpark Key: SPARK-47970 URL: https://issues.apache.org/jira/browse/SPARK-47970 Project: Spark Issue Type:

[jira] [Created] (SPARK-47937) Fix docstring of `hll_sketch_agg`

Ruifeng Zheng created SPARK-47937: - Summary: Fix docstring of `hll_sketch_agg` Key: SPARK-47937 URL: https://issues.apache.org/jira/browse/SPARK-47937 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-47935) Pin pandas==2.0.3 for pypy3.8

Ruifeng Zheng created SPARK-47935: - Summary: Pin pandas==2.0.3 for pypy3.8 Key: SPARK-47935 URL: https://issues.apache.org/jira/browse/SPARK-47935 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-47845) Support column type in split function in scala and python

[ https://issues.apache.org/jira/browse/SPARK-47845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47845. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46045

[jira] [Assigned] (SPARK-47845) Support column type in split function in scala and python

[ https://issues.apache.org/jira/browse/SPARK-47845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47845: - Assignee: Liu Cao > Support column type in split function in scala and python >

[jira] [Resolved] (SPARK-47906) Fix docstring and type hint of `hll_union_agg`

[ https://issues.apache.org/jira/browse/SPARK-47906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47906. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46128

[jira] [Assigned] (SPARK-47883) Make CollectTailExec execute lazily

[ https://issues.apache.org/jira/browse/SPARK-47883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47883: - Assignee: Ruifeng Zheng > Make CollectTailExec execute lazily >

[jira] [Resolved] (SPARK-47883) Make CollectTailExec execute lazily

[ https://issues.apache.org/jira/browse/SPARK-47883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47883. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46101

[jira] [Updated] (SPARK-47883) Make CollectTailExec execute lazily

[ https://issues.apache.org/jira/browse/SPARK-47883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47883: -- Summary: Make CollectTailExec execute lazily (was: Make CollectTailExec lazily execute) >

[jira] [Created] (SPARK-47883) Make CollectTailExec lazily execute

Ruifeng Zheng created SPARK-47883: - Summary: Make CollectTailExec lazily execute Key: SPARK-47883 URL: https://issues.apache.org/jira/browse/SPARK-47883 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-47868) Recursion Limit Error in SparkSession and SparkConnectPlanner

[ https://issues.apache.org/jira/browse/SPARK-47868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47868. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46075

[jira] [Created] (SPARK-47855) Warn `spark.sql.execution.arrow.pyspark.fallback.enabled` in Connect

Ruifeng Zheng created SPARK-47855: - Summary: Warn `spark.sql.execution.arrow.pyspark.fallback.enabled` in Connect Key: SPARK-47855 URL: https://issues.apache.org/jira/browse/SPARK-47855 Project:

[jira] [Resolved] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

[ https://issues.apache.org/jira/browse/SPARK-47828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47828. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46023

[jira] [Assigned] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

[ https://issues.apache.org/jira/browse/SPARK-47828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47828: - Assignee: Ruifeng Zheng > DataFrameWriterV2.overwrite fails with invalid plan >

[jira] [Assigned] (SPARK-47816) Document the lazy evaluation of views in spark.{sql, table}

[ https://issues.apache.org/jira/browse/SPARK-47816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47816: - Assignee: Ruifeng Zheng > Document the lazy evaluation of views in spark.{sql, table}

[jira] [Resolved] (SPARK-47816) Document the lazy evaluation of views in spark.{sql, table}

[ https://issues.apache.org/jira/browse/SPARK-47816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47816. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46007

[jira] [Updated] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

[ https://issues.apache.org/jira/browse/SPARK-47828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47828: -- Affects Version/s: 3.4.2 > DataFrameWriterV2.overwrite fails with invalid plan >

[jira] [Updated] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

[ https://issues.apache.org/jira/browse/SPARK-47828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47828: -- Issue Type: Bug (was: Improvement) > DataFrameWriterV2.overwrite fails with invalid plan >

[jira] [Created] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

Ruifeng Zheng created SPARK-47828: - Summary: DataFrameWriterV2.overwrite fails with invalid plan Key: SPARK-47828 URL: https://issues.apache.org/jira/browse/SPARK-47828 Project: Spark Issue

[jira] [Resolved] (SPARK-47815) Unify the user agent with json

[ https://issues.apache.org/jira/browse/SPARK-47815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47815. --- Resolution: Not A Problem > Unify the user agent with json > --

[jira] [Created] (SPARK-47816) Document the lazy evaluation of views in spark.{sql, table}

Ruifeng Zheng created SPARK-47816: - Summary: Document the lazy evaluation of views in spark.{sql, table} Key: SPARK-47816 URL: https://issues.apache.org/jira/browse/SPARK-47816 Project: Spark

[jira] [Updated] (SPARK-47815) Unify the user agent string with json

[ https://issues.apache.org/jira/browse/SPARK-47815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47815: -- Summary: Unify the user agent string with json (was: Unify the user agent string

[jira] [Created] (SPARK-47815) Unify the user agent string representation with json

Ruifeng Zheng created SPARK-47815: - Summary: Unify the user agent string representation with json Key: SPARK-47815 URL: https://issues.apache.org/jira/browse/SPARK-47815 Project: Spark Issue

[jira] [Updated] (SPARK-47815) Unify the user agent with json

[ https://issues.apache.org/jira/browse/SPARK-47815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47815: -- Summary: Unify the user agent with json (was: Unify the user agent string with json) >

[jira] [Assigned] (SPARK-47779) Add a helper function to sort PS Frame/Series

[ https://issues.apache.org/jira/browse/SPARK-47779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47779: - Assignee: Ruifeng Zheng > Add a helper function to sort PS Frame/Series >

[jira] [Resolved] (SPARK-47779) Add a helper function to sort PS Frame/Series

[ https://issues.apache.org/jira/browse/SPARK-47779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47779. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45952

[jira] [Created] (SPARK-47779) Add a helper function to sort PS Frame/Series

Ruifeng Zheng created SPARK-47779: - Summary: Add a helper function to sort PS Frame/Series Key: SPARK-47779 URL: https://issues.apache.org/jira/browse/SPARK-47779 Project: Spark Issue Type:

[jira] [Updated] (SPARK-47779) Add a helper function to sort PS Frame/Series

[ https://issues.apache.org/jira/browse/SPARK-47779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47779: -- Component/s: PS Tests (was: PySpark) > Add a helper

  1   2   3   4   5   6   7   8   9   10   >