[jira] [Commented] (SPARK-35473) Enable disallow_untyped_defs mypy check for pyspark.pandas.groupby.

[ https://issues.apache.org/jira/browse/SPARK-35473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17366949#comment-17366949 ] Takuya Ueshin commented on SPARK-35473: --- I'm working on this. > Enable disallow_u

[jira] [Resolved] (SPARK-35849) Make astype data-type-based for DecimalOps

[ https://issues.apache.org/jira/browse/SPARK-35849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35849. --- Fix Version/s: 3.2.0 Assignee: Yikun Jiang Resolution: Fixed Issue resolved

[jira] [Resolved] (SPARK-35847) Manage InternalField in DataTypeOps.isnull

[ https://issues.apache.org/jira/browse/SPARK-35847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35847. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue resolve

[jira] [Created] (SPARK-35859) Cleanup type hints.

Takuya Ueshin created SPARK-35859: - Summary: Cleanup type hints. Key: SPARK-35859 URL: https://issues.apache.org/jira/browse/SPARK-35859 Project: Spark Issue Type: Sub-task Componen

[jira] [Commented] (SPARK-35476) Enable disallow_untyped_defs mypy check for pyspark.pandas.series.

[ https://issues.apache.org/jira/browse/SPARK-35476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368476#comment-17368476 ] Takuya Ueshin commented on SPARK-35476: --- I'm working on this. > Enable disallow_u

[jira] [Commented] (SPARK-35471) Enable disallow_untyped_defs mypy check for pyspark.pandas.frame.

[ https://issues.apache.org/jira/browse/SPARK-35471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17368970#comment-17368970 ] Takuya Ueshin commented on SPARK-35471: --- I'm working on this. > Enable disallow_u

[jira] [Created] (SPARK-35901) Refine type hints in pyspark.pandas.window

Takuya Ueshin created SPARK-35901: - Summary: Refine type hints in pyspark.pandas.window Key: SPARK-35901 URL: https://issues.apache.org/jira/browse/SPARK-35901 Project: Spark Issue Type: Sub-

[jira] [Resolved] (SPARK-35466) Enable disallow_untyped_defs mypy check for pyspark.pandas.data_type_ops.

[ https://issues.apache.org/jira/browse/SPARK-35466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35466. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue resolve

[jira] [Resolved] (SPARK-35344) Support creating a Column of numpy literal value in pandas-on-Spark

[ https://issues.apache.org/jira/browse/SPARK-35344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35344. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue resolved

[jira] [Updated] (SPARK-35936) Deprecate Python 3.6 support

[ https://issues.apache.org/jira/browse/SPARK-35936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-35936: -- Description: According to [https://endoflife.date/python], Python 3.6 will be EOL on 23 Dec,

[jira] [Resolved] (SPARK-35859) Cleanup type hints.

[ https://issues.apache.org/jira/browse/SPARK-35859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35859. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue resolve

[jira] [Resolved] (SPARK-35464) pandas APIs on Spark: Enable mypy check "disallow_untyped_defs" for main codes.

[ https://issues.apache.org/jira/browse/SPARK-35464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35464. --- Fix Version/s: 3.2.0 Resolution: Fixed > pandas APIs on Spark: Enable mypy check "dis

[jira] [Reopened] (SPARK-35721) Path level discover for python unittests

[ https://issues.apache.org/jira/browse/SPARK-35721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reopened SPARK-35721: --- Assignee: (was: Yikun Jiang) The commit was reverted https://github.com/apache/spark/

[jira] [Updated] (SPARK-35721) Path level discover for python unittests

[ https://issues.apache.org/jira/browse/SPARK-35721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-35721: -- Fix Version/s: (was: 3.2.0) > Path level discover for python unittests > -

[jira] [Created] (SPARK-35941) pandas API on Spark: Improve type hints.

Takuya Ueshin created SPARK-35941: - Summary: pandas API on Spark: Improve type hints. Key: SPARK-35941 URL: https://issues.apache.org/jira/browse/SPARK-35941 Project: Spark Issue Type: Umbrel

[jira] [Created] (SPARK-35942) Refine Scalar type alias and reuse it

Takuya Ueshin created SPARK-35942: - Summary: Refine Scalar type alias and reuse it Key: SPARK-35942 URL: https://issues.apache.org/jira/browse/SPARK-35942 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-35943) Introduce Axis type alias.

Takuya Ueshin created SPARK-35943: - Summary: Introduce Axis type alias. Key: SPARK-35943 URL: https://issues.apache.org/jira/browse/SPARK-35943 Project: Spark Issue Type: Sub-task C

[jira] [Created] (SPARK-35944) Introduce type aliases for names or labels.

Takuya Ueshin created SPARK-35944: - Summary: Introduce type aliases for names or labels. Key: SPARK-35944 URL: https://issues.apache.org/jira/browse/SPARK-35944 Project: Spark Issue Type: Sub

[jira] [Updated] (SPARK-35464) pandas API on Spark: Enable mypy check "disallow_untyped_defs" for main codes.

[ https://issues.apache.org/jira/browse/SPARK-35464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-35464: -- Summary: pandas API on Spark: Enable mypy check "disallow_untyped_defs" for main codes. (was:

[jira] [Updated] (SPARK-35337) pandas API on Spark: Separate basic operations into data type based structures

[ https://issues.apache.org/jira/browse/SPARK-35337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-35337: -- Summary: pandas API on Spark: Separate basic operations into data type based structures (was:

[jira] [Created] (SPARK-35981) Use check_exact=False in StatsTest.test_cov_corr_meta to loosen the check precision

Takuya Ueshin created SPARK-35981: - Summary: Use check_exact=False in StatsTest.test_cov_corr_meta to loosen the check precision Key: SPARK-35981 URL: https://issues.apache.org/jira/browse/SPARK-35981

[jira] [Resolved] (SPARK-35339) Improve unit tests for data-type-based basic operations

[ https://issues.apache.org/jira/browse/SPARK-35339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35339. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue resolved

[jira] [Resolved] (SPARK-35615) Make unary and comparison operators data-type-based

[ https://issues.apache.org/jira/browse/SPARK-35615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35615. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue resolved

[jira] [Resolved] (SPARK-35340) Standardize TypeError messages for unsupported basic operations

[ https://issues.apache.org/jira/browse/SPARK-35340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35340. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue resolved

[jira] [Created] (SPARK-36062) Try to capture faulthanlder when a Python worker crashes.

Takuya Ueshin created SPARK-36062: - Summary: Try to capture faulthanlder when a Python worker crashes. Key: SPARK-36062 URL: https://issues.apache.org/jira/browse/SPARK-36062 Project: Spark I

[jira] [Created] (SPARK-36064) Manage InternalField more in DataTypeOps.

Takuya Ueshin created SPARK-36064: - Summary: Manage InternalField more in DataTypeOps. Key: SPARK-36064 URL: https://issues.apache.org/jira/browse/SPARK-36064 Project: Spark Issue Type: Sub-t

[jira] [Resolved] (SPARK-36002) Consolidate tests for data-type-based operations of decimal Series

[ https://issues.apache.org/jira/browse/SPARK-36002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36002. --- Fix Version/s: 3.2.0 Assignee: Yikun Jiang Resolution: Fixed Issue resolved

[jira] [Created] (SPARK-36103) Manage InternalField in DataTypeOps.invert

Takuya Ueshin created SPARK-36103: - Summary: Manage InternalField in DataTypeOps.invert Key: SPARK-36103 URL: https://issues.apache.org/jira/browse/SPARK-36103 Project: Spark Issue Type: Impr

[jira] [Resolved] (SPARK-36125) Implement non-equality comparison operators between two Categoricals

[ https://issues.apache.org/jira/browse/SPARK-36125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36125. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue resolved

[jira] [Created] (SPARK-36167) Revisit more InternalField managements.

Takuya Ueshin created SPARK-36167: - Summary: Revisit more InternalField managements. Key: SPARK-36167 URL: https://issues.apache.org/jira/browse/SPARK-36167 Project: Spark Issue Type: Improve

[jira] [Resolved] (SPARK-36167) Revisit more InternalField managements.

[ https://issues.apache.org/jira/browse/SPARK-36167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36167. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue resolve

[jira] [Created] (SPARK-36185) Implement functions in CategoricalAccessor/CategoricalIndex

Takuya Ueshin created SPARK-36185: - Summary: Implement functions in CategoricalAccessor/CategoricalIndex Key: SPARK-36185 URL: https://issues.apache.org/jira/browse/SPARK-36185 Project: Spark

[jira] [Created] (SPARK-36186) Add as_ordered/as_unordered to CategoricalAccessor and CategoricalIndex.

Takuya Ueshin created SPARK-36186: - Summary: Add as_ordered/as_unordered to CategoricalAccessor and CategoricalIndex. Key: SPARK-36186 URL: https://issues.apache.org/jira/browse/SPARK-36186 Project: S

[jira] [Created] (SPARK-36188) Add categories setter to CategoricalAccessor and CategoricalIndex.

Takuya Ueshin created SPARK-36188: - Summary: Add categories setter to CategoricalAccessor and CategoricalIndex. Key: SPARK-36188 URL: https://issues.apache.org/jira/browse/SPARK-36188 Project: Spark

[jira] [Commented] (SPARK-36188) Add categories setter to CategoricalAccessor and CategoricalIndex.

[ https://issues.apache.org/jira/browse/SPARK-36188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17382378#comment-17382378 ] Takuya Ueshin commented on SPARK-36188: --- I'm working on this. > Add categories se

[jira] [Resolved] (SPARK-36127) Support comparison between a Categorical and a scalar

[ https://issues.apache.org/jira/browse/SPARK-36127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36127. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng (was: Apache Spark) Resolution:

[jira] [Created] (SPARK-36214) Add add_categories to CategoricalAccessor and CategoricalIndex.

Takuya Ueshin created SPARK-36214: - Summary: Add add_categories to CategoricalAccessor and CategoricalIndex. Key: SPARK-36214 URL: https://issues.apache.org/jira/browse/SPARK-36214 Project: Spark

[jira] [Commented] (SPARK-36214) Add add_categories to CategoricalAccessor and CategoricalIndex.

[ https://issues.apache.org/jira/browse/SPARK-36214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17383641#comment-17383641 ] Takuya Ueshin commented on SPARK-36214: --- I'm working on this. > Add add_categorie

[jira] [Resolved] (SPARK-36186) Add as_ordered/as_unordered to CategoricalAccessor and CategoricalIndex.

[ https://issues.apache.org/jira/browse/SPARK-36186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36186. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue resolve

[jira] [Resolved] (SPARK-36188) Add categories setter to CategoricalAccessor and CategoricalIndex.

[ https://issues.apache.org/jira/browse/SPARK-36188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36188. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue resolve

[jira] [Created] (SPARK-36249) Add remove_categories to CategoricalAccessor and CategoricalIndex

Takuya Ueshin created SPARK-36249: - Summary: Add remove_categories to CategoricalAccessor and CategoricalIndex Key: SPARK-36249 URL: https://issues.apache.org/jira/browse/SPARK-36249 Project: Spark

[jira] [Commented] (SPARK-36249) Add remove_categories to CategoricalAccessor and CategoricalIndex

[ https://issues.apache.org/jira/browse/SPARK-36249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17385069#comment-17385069 ] Takuya Ueshin commented on SPARK-36249: --- I'm working on this. > Add remove_catego

[jira] [Resolved] (SPARK-36214) Add add_categories to CategoricalAccessor and CategoricalIndex.

[ https://issues.apache.org/jira/browse/SPARK-36214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36214. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue resolve

[jira] [Created] (SPARK-36261) Add remove_unused_categories to CategoricalAccessor and CategoricalIndex.

Takuya Ueshin created SPARK-36261: - Summary: Add remove_unused_categories to CategoricalAccessor and CategoricalIndex. Key: SPARK-36261 URL: https://issues.apache.org/jira/browse/SPARK-36261 Project:

[jira] [Commented] (SPARK-36261) Add remove_unused_categories to CategoricalAccessor and CategoricalIndex.

[ https://issues.apache.org/jira/browse/SPARK-36261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17385690#comment-17385690 ] Takuya Ueshin commented on SPARK-36261: --- I'm working on this. > Add remove_unused

[jira] [Created] (SPARK-36264) Add reorder_categories to CategoricalAccessor and CategoricalIndex.

Takuya Ueshin created SPARK-36264: - Summary: Add reorder_categories to CategoricalAccessor and CategoricalIndex. Key: SPARK-36264 URL: https://issues.apache.org/jira/browse/SPARK-36264 Project: Spark

[jira] [Commented] (SPARK-36264) Add reorder_categories to CategoricalAccessor and CategoricalIndex.

[ https://issues.apache.org/jira/browse/SPARK-36264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17385785#comment-17385785 ] Takuya Ueshin commented on SPARK-36264: --- I'm working on this. > Add reorder_categ

[jira] [Created] (SPARK-36265) Use __getitem__ instead of getItem to suppress warnings.

Takuya Ueshin created SPARK-36265: - Summary: Use __getitem__ instead of getItem to suppress warnings. Key: SPARK-36265 URL: https://issues.apache.org/jira/browse/SPARK-36265 Project: Spark Is

[jira] [Created] (SPARK-36267) Clean up CategoricalAccessor and CategoricalIndex.

Takuya Ueshin created SPARK-36267: - Summary: Clean up CategoricalAccessor and CategoricalIndex. Key: SPARK-36267 URL: https://issues.apache.org/jira/browse/SPARK-36267 Project: Spark Issue Ty

[jira] [Updated] (SPARK-36267) Clean up CategoricalAccessor and CategoricalIndex.

[ https://issues.apache.org/jira/browse/SPARK-36267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-36267: -- Description: - Clean up the classes - Add deprecation warnings > Clean up CategoricalAccessor

[jira] [Commented] (SPARK-36267) Clean up CategoricalAccessor and CategoricalIndex.

[ https://issues.apache.org/jira/browse/SPARK-36267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386534#comment-17386534 ] Takuya Ueshin commented on SPARK-36267: --- I'll work on this. > Clean up Categorica

[jira] [Created] (SPARK-36279) Fix lint-python to work with Python 3.9

Takuya Ueshin created SPARK-36279: - Summary: Fix lint-python to work with Python 3.9 Key: SPARK-36279 URL: https://issues.apache.org/jira/browse/SPARK-36279 Project: Spark Issue Type: Sub-tas

[jira] [Resolved] (SPARK-36264) Add reorder_categories to CategoricalAccessor and CategoricalIndex.

[ https://issues.apache.org/jira/browse/SPARK-36264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36264. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue resolve

[jira] [Resolved] (SPARK-36274) Fix equality comparison of unordered Categoricals

[ https://issues.apache.org/jira/browse/SPARK-36274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36274. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue resolved

[jira] [Resolved] (SPARK-36260) Add set_categories to CategoricalAccessor and CategoricalIndex

[ https://issues.apache.org/jira/browse/SPARK-36260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36260. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue resolved

[jira] [Resolved] (SPARK-36185) Implement functions in CategoricalAccessor/CategoricalIndex

[ https://issues.apache.org/jira/browse/SPARK-36185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36185. --- Fix Version/s: 3.2.0 Resolution: Done I'd close this since we've done the tasks we pl

[jira] [Created] (SPARK-36320) Fix Series/Index.copy() to drop extra columns.

Takuya Ueshin created SPARK-36320: - Summary: Fix Series/Index.copy() to drop extra columns. Key: SPARK-36320 URL: https://issues.apache.org/jira/browse/SPARK-36320 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-36143) Adjust `astype` of fractional Series with missing values to follow pandas

[ https://issues.apache.org/jira/browse/SPARK-36143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36143. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue resolved

[jira] [Resolved] (SPARK-36190) Improve the rest of DataTypeOps tests by avoiding joins

[ https://issues.apache.org/jira/browse/SPARK-36190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36190. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue resolved

[jira] [Created] (SPARK-36333) Reuse isnull where the nullability check is needed.

Takuya Ueshin created SPARK-36333: - Summary: Reuse isnull where the nullability check is needed. Key: SPARK-36333 URL: https://issues.apache.org/jira/browse/SPARK-36333 Project: Spark Issue T

[jira] [Updated] (SPARK-36333) Reuse isnull where the null check is needed.

[ https://issues.apache.org/jira/browse/SPARK-36333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-36333: -- Summary: Reuse isnull where the null check is needed. (was: Reuse isnull where the nullabilit

[jira] [Created] (SPARK-36350) Make nanvl work with DataTypeOps

Takuya Ueshin created SPARK-36350: - Summary: Make nanvl work with DataTypeOps Key: SPARK-36350 URL: https://issues.apache.org/jira/browse/SPARK-36350 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-36333) Reuse isnull where the null check is needed.

[ https://issues.apache.org/jira/browse/SPARK-36333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36333. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue resolve

[jira] [Resolved] (SPARK-36350) Make nanvl work with DataTypeOps

[ https://issues.apache.org/jira/browse/SPARK-36350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36350. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed Issue resolve

[jira] [Created] (SPARK-36365) Remove old workarounds related to ordering.

Takuya Ueshin created SPARK-36365: - Summary: Remove old workarounds related to ordering. Key: SPARK-36365 URL: https://issues.apache.org/jira/browse/SPARK-36365 Project: Spark Issue Type: Imp

[jira] [Updated] (SPARK-36365) Remove old workarounds related to null ordering.

[ https://issues.apache.org/jira/browse/SPARK-36365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-36365: -- Summary: Remove old workarounds related to null ordering. (was: Remove old workarounds relate

[jira] [Created] (SPARK-36367) Fix the behavior to follow pandas >= 1.3

Takuya Ueshin created SPARK-36367: - Summary: Fix the behavior to follow pandas >= 1.3 Key: SPARK-36367 URL: https://issues.apache.org/jira/browse/SPARK-36367 Project: Spark Issue Type: Improv

[jira] [Created] (SPARK-36368) Fix CategoricalOps.astype to follow pandas 1.3

Takuya Ueshin created SPARK-36368: - Summary: Fix CategoricalOps.astype to follow pandas 1.3 Key: SPARK-36368 URL: https://issues.apache.org/jira/browse/SPARK-36368 Project: Spark Issue Type:

[jira] [Updated] (SPARK-36367) Fix the behavior to follow pandas >= 1.3

[ https://issues.apache.org/jira/browse/SPARK-36367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-36367: -- Issue Type: Umbrella (was: Improvement) > Fix the behavior to follow pandas >= 1.3 >

[jira] [Created] (SPARK-36369) Fix Index.union to follow pandas 1.3

Takuya Ueshin created SPARK-36369: - Summary: Fix Index.union to follow pandas 1.3 Key: SPARK-36369 URL: https://issues.apache.org/jira/browse/SPARK-36369 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36370) Avoid using SelectionMixin._builtin_table which is removed in pandas 1.3

Takuya Ueshin created SPARK-36370: - Summary: Avoid using SelectionMixin._builtin_table which is removed in pandas 1.3 Key: SPARK-36370 URL: https://issues.apache.org/jira/browse/SPARK-36370 Project: S

[jira] [Created] (SPARK-36386) Fix expanding to follow pandas 1.3

Takuya Ueshin created SPARK-36386: - Summary: Fix expanding to follow pandas 1.3 Key: SPARK-36386 URL: https://issues.apache.org/jira/browse/SPARK-36386 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-36386) Fix groupby-expanding to follow pandas 1.3

[ https://issues.apache.org/jira/browse/SPARK-36386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-36386: -- Summary: Fix groupby-expanding to follow pandas 1.3 (was: Fix expanding to follow pandas 1.3)

[jira] [Created] (SPARK-36387) Fix Series.astype from datetime to nullable string

Takuya Ueshin created SPARK-36387: - Summary: Fix Series.astype from datetime to nullable string Key: SPARK-36387 URL: https://issues.apache.org/jira/browse/SPARK-36387 Project: Spark Issue Ty

[jira] [Updated] (SPARK-36386) Fix DataFrame groupby-expanding to follow pandas 1.3

[ https://issues.apache.org/jira/browse/SPARK-36386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-36386: -- Summary: Fix DataFrame groupby-expanding to follow pandas 1.3 (was: Fix groupby-expanding to

[jira] [Created] (SPARK-36388) Fix DataFrame groupby-rolling to follow pandas 1.3

Takuya Ueshin created SPARK-36388: - Summary: Fix DataFrame groupby-rolling to follow pandas 1.3 Key: SPARK-36388 URL: https://issues.apache.org/jira/browse/SPARK-36388 Project: Spark Issue Ty

[jira] [Resolved] (SPARK-36370) Avoid using SelectionMixin._builtin_table which is removed in pandas 1.3

[ https://issues.apache.org/jira/browse/SPARK-36370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36370. --- Fix Version/s: 3.2.0 Assignee: Takuya Ueshin Resolution: Fixed This was incl

[jira] [Resolved] (SPARK-36469) Implement Index.map

[ https://issues.apache.org/jira/browse/SPARK-36469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36469. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng (was: Apache Spark) Resolution:

[jira] [Resolved] (SPARK-36387) Fix Series.astype from datetime to nullable string

[ https://issues.apache.org/jira/browse/SPARK-36387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36387. --- Fix Version/s: 3.3.0 Assignee: Haejoon Lee Resolution: Fixed Issue resolved

[jira] [Created] (SPARK-36537) Take care of other tests disabled.

Takuya Ueshin created SPARK-36537: - Summary: Take care of other tests disabled. Key: SPARK-36537 URL: https://issues.apache.org/jira/browse/SPARK-36537 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-36537) Take care of other tests disabled related to inplace updates with CategoricalDtype.

[ https://issues.apache.org/jira/browse/SPARK-36537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-36537: -- Summary: Take care of other tests disabled related to inplace updates with CategoricalDtype.

[jira] [Updated] (SPARK-36537) Take care of other tests disabled related to inplace updates with CategoricalDtype.

[ https://issues.apache.org/jira/browse/SPARK-36537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-36537: -- Description: There are some more tests disabled related to inplace updates with CategoricalDt

[jira] [Resolved] (SPARK-36368) Fix CategoricalOps.astype to follow pandas 1.3

[ https://issues.apache.org/jira/browse/SPARK-36368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36368. --- Fix Version/s: 3.3.0 Assignee: Haejoon Lee Resolution: Fixed Issue resolved

[jira] [Resolved] (SPARK-36399) Implement DataFrame.combine_first

[ https://issues.apache.org/jira/browse/SPARK-36399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36399. --- Fix Version/s: 3.3.0 Assignee: Xinrong Meng Resolution: Fixed Issue resolved

[jira] [Created] (SPARK-36653) Implement Series.__xor__

Takuya Ueshin created SPARK-36653: - Summary: Implement Series.__xor__ Key: SPARK-36653 URL: https://issues.apache.org/jira/browse/SPARK-36653 Project: Spark Issue Type: Sub-task Com

[jira] [Resolved] (SPARK-36401) Implement Series.cov

[ https://issues.apache.org/jira/browse/SPARK-36401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36401. --- Fix Version/s: 3.3.0 Assignee: dgd_contributor Resolution: Fixed Issue resol

[jira] [Resolved] (SPARK-36655) Add `versionadded` for API added in Spark 3.3.0

[ https://issues.apache.org/jira/browse/SPARK-36655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36655. --- Fix Version/s: 3.3.0 Assignee: Xinrong Meng Resolution: Fixed Issue resolved

[jira] [Created] (SPARK-36696) spark.read.parquet loads empty dataset

Takuya Ueshin created SPARK-36696: - Summary: spark.read.parquet loads empty dataset Key: SPARK-36696 URL: https://issues.apache.org/jira/browse/SPARK-36696 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-36696) spark.read.parquet loads empty dataset

[ https://issues.apache.org/jira/browse/SPARK-36696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-36696: -- Attachment: example.parquet > spark.read.parquet loads empty dataset > ---

[jira] [Commented] (SPARK-34276) Check the unreleased/unresolved JIRAs/PRs of Parquet 1.11 and 1.12

[ https://issues.apache.org/jira/browse/SPARK-34276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17412183#comment-17412183 ] Takuya Ueshin commented on SPARK-34276: --- SPARK-36696 seems to be the actual issue

[jira] [Resolved] (SPARK-36653) Implement Series.__xor__

[ https://issues.apache.org/jira/browse/SPARK-36653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36653. --- Fix Version/s: 3.3.0 Assignee: dgd_contributor Resolution: Fixed Issue resol

[jira] [Resolved] (SPARK-36722) Problems with update function in koalas - pyspark pandas.

[ https://issues.apache.org/jira/browse/SPARK-36722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36722. --- Fix Version/s: 3.2.0 Assignee: dgd_contributor Resolution: Fixed Issue resol

[jira] [Resolved] (SPARK-36696) spark.read.parquet loads empty dataset

[ https://issues.apache.org/jira/browse/SPARK-36696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36696. --- Fix Version/s: 3.2.0 Resolution: Fixed I confirmed the example file can be read after

[jira] [Commented] (SPARK-34276) Check the unreleased/unresolved JIRAs/PRs of Parquet 1.11 and 1.12

[ https://issues.apache.org/jira/browse/SPARK-34276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17415725#comment-17415725 ] Takuya Ueshin commented on SPARK-34276: --- SPARK-36696 was fixed by upgrading parque

[jira] [Created] (SPARK-41753) Add tests for ArrayZip to check the result size and nullability.

Takuya Ueshin created SPARK-41753: - Summary: Add tests for ArrayZip to check the result size and nullability. Key: SPARK-41753 URL: https://issues.apache.org/jira/browse/SPARK-41753 Project: Spark

[jira] [Created] (SPARK-41778) Add an alias "reduce" to ArrayAggregate

Takuya Ueshin created SPARK-41778: - Summary: Add an alias "reduce" to ArrayAggregate Key: SPARK-41778 URL: https://issues.apache.org/jira/browse/SPARK-41778 Project: Spark Issue Type: Improve

[jira] [Created] (SPARK-42295) Tear down the test cleanly

Takuya Ueshin created SPARK-42295: - Summary: Tear down the test cleanly Key: SPARK-42295 URL: https://issues.apache.org/jira/browse/SPARK-42295 Project: Spark Issue Type: Sub-task C

[jira] [Commented] (SPARK-42017) Different error type AnalysisException vs SparkConnectAnalysisException

[ https://issues.apache.org/jira/browse/SPARK-42017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17683594#comment-17683594 ] Takuya Ueshin commented on SPARK-42017: --- The error class hierarchy is one of the i

[jira] [Created] (SPARK-42338) Different exception in DataFrame.sample

Takuya Ueshin created SPARK-42338: - Summary: Different exception in DataFrame.sample Key: SPARK-42338 URL: https://issues.apache.org/jira/browse/SPARK-42338 Project: Spark Issue Type: Sub-tas

[jira] [Created] (SPARK-42340) Implement GroupedData.applyInPandas

Takuya Ueshin created SPARK-42340: - Summary: Implement GroupedData.applyInPandas Key: SPARK-42340 URL: https://issues.apache.org/jira/browse/SPARK-42340 Project: Spark Issue Type: Sub-task

<    3   4   5   6   7   8   9   >