[jira] [Updated] (SPARK-36628) Implement __getitem__ of label-based MultiIndex

2021-09-01 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36628: - Summary: Implement __getitem__ of label-based MultiIndex (was: Implement label-based

[jira] [Created] (SPARK-36628) Implement label-based MultiIndex select

2021-08-31 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36628: Summary: Implement label-based MultiIndex select Key: SPARK-36628 URL: https://issues.apache.org/jira/browse/SPARK-36628 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-36618) Support axis=0 for drop API

2021-08-30 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36618: - Summary: Support axis=0 for drop API (was: Support axis=0 in for drop API) > Support axis=0

[jira] [Created] (SPARK-36618) Support axis=0 in for drop API

2021-08-30 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36618: Summary: Support axis=0 in for drop API Key: SPARK-36618 URL: https://issues.apache.org/jira/browse/SPARK-36618 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36570) Investigate native support for raw data containing commas

2021-08-23 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36570: Summary: Investigate native support for raw data containing commas Key: SPARK-36570 URL: https://issues.apache.org/jira/browse/SPARK-36570 Project: Spark

[jira] [Created] (SPARK-36569) Support errors='coerce' for ps.to_numeric

2021-08-23 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36569: Summary: Support errors='coerce' for ps.to_numeric Key: SPARK-36569 URL: https://issues.apache.org/jira/browse/SPARK-36569 Project: Spark Issue Type:

[jira] [Updated] (SPARK-36470) Implement `CategoricalIndex.map` and `DatetimeIndx.map`

2021-08-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36470: - Summary: Implement `CategoricalIndex.map` and `DatetimeIndx.map` (was: Implement

[jira] [Issue Comment Deleted] (SPARK-36471) Implement MultiIndex.map

2021-08-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36471: - Comment: was deleted (was: Map is not implemented for DatetimeIndex in pandas yet.) >

[jira] [Reopened] (SPARK-36471) Implement MultiIndex.map

2021-08-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reopened SPARK-36471: -- > Implement MultiIndex.map > > > Key: SPARK-36471 >

[jira] [Updated] (SPARK-36471) Implement MultiIndex.map

2021-08-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36471: - Summary: Implement MultiIndex.map (was: Implement DatetimeIndex.map) > Implement

[jira] [Updated] (SPARK-36470) Implement CategoricalIndex.map and DatetimeIndx.map

2021-08-16 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36470: - Summary: Implement CategoricalIndex.map and DatetimeIndx.map (was: Implement

[jira] [Resolved] (SPARK-36471) Implement DatetimeIndex.map

2021-08-13 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-36471. -- Resolution: Won't Do Map is not implemented for DatetimeIndex in pandas yet. > Implement

[jira] [Created] (SPARK-36471) Implement DatetimeIndex.map

2021-08-10 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36471: Summary: Implement DatetimeIndex.map Key: SPARK-36471 URL: https://issues.apache.org/jira/browse/SPARK-36471 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36469) Implement Index.map

2021-08-10 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36469: Summary: Implement Index.map Key: SPARK-36469 URL: https://issues.apache.org/jira/browse/SPARK-36469 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36470) Implement CategoricalIndex.map

2021-08-10 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36470: Summary: Implement CategoricalIndex.map Key: SPARK-36470 URL: https://issues.apache.org/jira/browse/SPARK-36470 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-36395) Implement map for indexes

2021-08-10 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36395: - Summary: Implement map for indexes (was: Implement Index.map) > Implement map for indexes >

[jira] [Updated] (SPARK-36395) Implement Index.map

2021-08-10 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36395: - Parent: (was: SPARK-36394) Issue Type: Umbrella (was: Sub-task) > Implement

[jira] [Created] (SPARK-36439) Implement DataFrame.join on key column

2021-08-05 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36439: Summary: Implement DataFrame.join on key column Key: SPARK-36439 URL: https://issues.apache.org/jira/browse/SPARK-36439 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36438) Support list-like Python objects for Series comparison

2021-08-05 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36438: Summary: Support list-like Python objects for Series comparison Key: SPARK-36438 URL: https://issues.apache.org/jira/browse/SPARK-36438 Project: Spark Issue

[jira] [Created] (SPARK-36437) Enable binary operations with list-like Python objects

2021-08-05 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36437: Summary: Enable binary operations with list-like Python objects Key: SPARK-36437 URL: https://issues.apache.org/jira/browse/SPARK-36437 Project: Spark Issue

[jira] [Created] (SPARK-36436) Implement 'weights' and 'axis' in sample at DataFrame and Series

2021-08-05 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36436: Summary: Implement 'weights' and 'axis' in sample at DataFrame and Series Key: SPARK-36436 URL: https://issues.apache.org/jira/browse/SPARK-36436 Project: Spark

[jira] [Created] (SPARK-36435) Implement MultIndex.equal_levels

2021-08-05 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36435: Summary: Implement MultIndex.equal_levels Key: SPARK-36435 URL: https://issues.apache.org/jira/browse/SPARK-36435 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36434) Implement DataFrame.lookup

2021-08-05 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36434: Summary: Implement DataFrame.lookup Key: SPARK-36434 URL: https://issues.apache.org/jira/browse/SPARK-36434 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36403) Implement Index.putmask

2021-08-03 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36403: Summary: Implement Index.putmask Key: SPARK-36403 URL: https://issues.apache.org/jira/browse/SPARK-36403 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36402) Implement Series.combine

2021-08-03 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36402: Summary: Implement Series.combine Key: SPARK-36402 URL: https://issues.apache.org/jira/browse/SPARK-36402 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36401) Implement Series.cov

2021-08-03 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36401: Summary: Implement Series.cov Key: SPARK-36401 URL: https://issues.apache.org/jira/browse/SPARK-36401 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36399) Implement DataFrame.combine_first

2021-08-03 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36399: Summary: Implement DataFrame.combine_first Key: SPARK-36399 URL: https://issues.apache.org/jira/browse/SPARK-36399 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-36395) Implement Index.map

2021-08-03 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36395: - Summary: Implement Index.map (was: Implement Index.map functionality) > Implement Index.map >

[jira] [Updated] (SPARK-36396) Implement DataFrame.cov

2021-08-03 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36396: - Summary: Implement DataFrame.cov (was: Implement dataframe.cov) > Implement DataFrame.cov >

[jira] [Created] (SPARK-36397) Implement DataFrame.mode

2021-08-03 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36397: Summary: Implement DataFrame.mode Key: SPARK-36397 URL: https://issues.apache.org/jira/browse/SPARK-36397 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-36394) Increase pandas API coverage in PySpark

2021-08-03 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36394: - Description: Increase pandas API coverage in PySpark.   Especially, pending PRs

[jira] [Created] (SPARK-36396) Implement dataframe.cov

2021-08-03 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36396: Summary: Implement dataframe.cov Key: SPARK-36396 URL: https://issues.apache.org/jira/browse/SPARK-36396 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36395) Implement Index.map functionality

2021-08-03 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36395: Summary: Implement Index.map functionality Key: SPARK-36395 URL: https://issues.apache.org/jira/browse/SPARK-36395 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-36394) Increase pandas API coverage in PySpark

2021-08-03 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36394: Summary: Increase pandas API coverage in PySpark Key: SPARK-36394 URL: https://issues.apache.org/jira/browse/SPARK-36394 Project: Spark Issue Type: Umbrella

[jira] [Created] (SPARK-36364) Move window and aggregate functions to DataTypeOps

2021-07-30 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36364: Summary: Move window and aggregate functions to DataTypeOps Key: SPARK-36364 URL: https://issues.apache.org/jira/browse/SPARK-36364 Project: Spark Issue

[jira] [Resolved] (SPARK-36140) Replace DataTypeOps tests that have operations on different Series

2021-07-30 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-36140. -- Resolution: Done > Replace DataTypeOps tests that have operations on different Series >

[jira] [Updated] (SPARK-36192) Better error messages when comparing against list

2021-07-29 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36192: - Environment: (was: We shall throw TypeError messages rather than Spark exceptions.) >

[jira] [Updated] (SPARK-36192) Better error messages for DataTypeOps against list

2021-07-29 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36192: - Summary: Better error messages for DataTypeOps against list (was: Better error messages when

[jira] [Resolved] (SPARK-35997) Implement comparison operators for CategoricalDtype in pandas API on Spark

2021-07-27 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-35997. -- Resolution: Done > Implement comparison operators for CategoricalDtype in pandas API on Spark

[jira] [Updated] (SPARK-36190) Improve the rest of DataTypeOps tests by avoiding joins

2021-07-27 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36190: - Summary: Improve the rest of DataTypeOps tests by avoiding joins (was: Improve all DataTypeOps

[jira] [Updated] (SPARK-36190) Improve all DataTypeOps tests by avoiding joins

2021-07-27 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36190: - Summary: Improve all DataTypeOps tests by avoiding joins (was: Improve the rest of DataTypeOps

[jira] [Updated] (SPARK-36310) Fix hasnan() window function in IndexOpsMixin

2021-07-27 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36310: - Description:   {code:java} File "/__w/spark/spark/python/pyspark/pandas/groupby.py", line 1497,

[jira] [Updated] (SPARK-36310) Fix hasnan() window function in IndexOpsMixin

2021-07-27 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36310: - Summary: Fix hasnan() window function in IndexOpsMixin (was: Fix hasnan(), any(), and all()

[jira] [Updated] (SPARK-36310) Fix hasnan(), any(), and all() window function in IndexOpsMixin

2021-07-26 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36310: - Summary: Fix hasnan(), any(), and all() window function in IndexOpsMixin (was: Fix hasnan(),

[jira] [Created] (SPARK-36310) Fix hasnan(), any(), and all() window function

2021-07-26 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36310: Summary: Fix hasnan(), any(), and all() window function Key: SPARK-36310 URL: https://issues.apache.org/jira/browse/SPARK-36310 Project: Spark Issue Type:

[jira] [Updated] (SPARK-36143) Adjust `astype` of fractional Series with missing values to follow pandas

2021-07-26 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36143: - Description: {code:java} >>> pser = pd.Series([1, 2, np.nan], dtype=float) >>> psser =

[jira] [Updated] (SPARK-36143) Adjust `astype` of fractional Series with missing values to follow pandas

2021-07-26 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36143: - Summary: Adjust `astype` of fractional Series with missing values to follow pandas (was:

[jira] [Issue Comment Deleted] (SPARK-36230) hasnans for Series of Decimal(`NaN`)

2021-07-26 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36230: - Comment: was deleted (was: https://issues.apache.org/jira/browse/SPARK-36231) > hasnans for

[jira] [Commented] (SPARK-36230) hasnans for Series of Decimal(`NaN`)

2021-07-26 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17387596#comment-17387596 ] Xinrong Meng commented on SPARK-36230: -- https://issues.apache.org/jira/browse/SPARK-36231 >

[jira] [Created] (SPARK-36274) Fix equality comparison of unordered Categoricals

2021-07-23 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36274: Summary: Fix equality comparison of unordered Categoricals Key: SPARK-36274 URL: https://issues.apache.org/jira/browse/SPARK-36274 Project: Spark Issue

[jira] [Reopened] (SPARK-35997) Implement comparison operators for CategoricalDtype in pandas API on Spark

2021-07-23 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reopened SPARK-35997: -- > Implement comparison operators for CategoricalDtype in pandas API on Spark >

[jira] [Commented] (SPARK-36260) Add set_categories to CategoricalAccessor and CategoricalIndex

2021-07-22 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385668#comment-17385668 ] Xinrong Meng commented on SPARK-36260: -- I am working on this. > Add set_categories to

[jira] [Created] (SPARK-36260) Add set_categories to CategoricalAccessor and CategoricalIndex

2021-07-22 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36260: Summary: Add set_categories to CategoricalAccessor and CategoricalIndex Key: SPARK-36260 URL: https://issues.apache.org/jira/browse/SPARK-36260 Project: Spark

[jira] [Commented] (SPARK-36248) Add rename_categories to CategoricalAccessor and CategoricalIndex

2021-07-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17385063#comment-17385063 ] Xinrong Meng commented on SPARK-36248: -- I'm working on this. > Add rename_categories to

[jira] [Created] (SPARK-36248) Add rename_categories to CategoricalAccessor and CategoricalIndex

2021-07-21 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36248: Summary: Add rename_categories to CategoricalAccessor and CategoricalIndex Key: SPARK-36248 URL: https://issues.apache.org/jira/browse/SPARK-36248 Project: Spark

[jira] [Updated] (SPARK-36142) Adjust exponentiation between Series with missing values and bool literal to follow pandas

2021-07-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36142: - Summary: Adjust exponentiation between Series with missing values and bool literal to follow

[jira] [Updated] (SPARK-36143) Adjust astype of Series with missing values to follow pandas

2021-07-21 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36143: - Description: {code:java} >>> pser = pd.Series([1, 2, np.nan], dtype=float) >>> psser =

[jira] [Updated] (SPARK-36000) Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled

2021-07-20 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36000: - Description: The creation and operations of ps.Series/Index with Decimal('NaN') doesn't work

[jira] [Updated] (SPARK-36000) Support creation and operations of ps.Series/Index with Decimal('NaN')

2021-07-20 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36000: - Summary: Support creation and operations of ps.Series/Index with Decimal('NaN') (was: Support

[jira] [Updated] (SPARK-36000) Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled

2021-07-20 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36000: - Description: The creation and operations of ps.Series/Index have bugs. Please refer to

[jira] [Updated] (SPARK-36232) Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled

2021-07-20 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36232: - Description:   {code:java} >>> import decimal as d >>> import pyspark.pandas as ps >>> import

[jira] [Commented] (SPARK-36141) Support arithmetic operations of Series containing Decimal(np.nan)

2021-07-20 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384485#comment-17384485 ] Xinrong Meng commented on SPARK-36141: -- It is closed because it is duplicated from 

[jira] [Created] (SPARK-36232) Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled

2021-07-20 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36232: Summary: Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled Key: SPARK-36232 URL: https://issues.apache.org/jira/browse/SPARK-36232

[jira] [Created] (SPARK-36231) Support arithmetic operations of Series containing Decimal(np.nan)

2021-07-20 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36231: Summary: Support arithmetic operations of Series containing Decimal(np.nan) Key: SPARK-36231 URL: https://issues.apache.org/jira/browse/SPARK-36231 Project: Spark

[jira] [Resolved] (SPARK-36141) Support arithmetic operations of Series containing Decimal(np.nan)

2021-07-20 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-36141. -- Resolution: Duplicate > Support arithmetic operations of Series containing Decimal(np.nan) >

[jira] [Created] (SPARK-36230) hasnans for Series of Decimal(`NaN`)

2021-07-20 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36230: Summary: hasnans for Series of Decimal(`NaN`) Key: SPARK-36230 URL: https://issues.apache.org/jira/browse/SPARK-36230 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-36143) Adjust astype of Series with missing values to follow pandas

2021-07-20 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36143: - Summary: Adjust astype of Series with missing values to follow pandas (was: Adjust astype of

[jira] [Commented] (SPARK-36000) Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled

2021-07-19 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383667#comment-17383667 ] Xinrong Meng commented on SPARK-36000: -- We might want to support

[jira] [Resolved] (SPARK-35997) Implement comparison operators for CategoricalDtype in pandas API on Spark

2021-07-19 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-35997. -- Resolution: Done > Implement comparison operators for CategoricalDtype in pandas API on Spark

[jira] [Created] (SPARK-36192) Better error messages when comparing against list

2021-07-16 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36192: Summary: Better error messages when comparing against list Key: SPARK-36192 URL: https://issues.apache.org/jira/browse/SPARK-36192 Project: Spark Issue

[jira] [Created] (SPARK-36190) Improve the rest of DataTypeOps tests by avoiding joins

2021-07-16 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36190: Summary: Improve the rest of DataTypeOps tests by avoiding joins Key: SPARK-36190 URL: https://issues.apache.org/jira/browse/SPARK-36190 Project: Spark

[jira] [Created] (SPARK-36189) Improve bool, string, numeric DataTypeOps tests by avoiding joins

2021-07-16 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36189: Summary: Improve bool, string, numeric DataTypeOps tests by avoiding joins Key: SPARK-36189 URL: https://issues.apache.org/jira/browse/SPARK-36189 Project: Spark

[jira] [Updated] (SPARK-36127) Support comparison between a Categorical and a scalar

2021-07-15 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36127: - Summary: Support comparison between a Categorical and a scalar (was: Adjust non-equality

[jira] [Comment Edited] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

2021-07-14 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380902#comment-17380902 ] Xinrong Meng edited comment on SPARK-36031 at 7/14/21, 11:52 PM: - Hi

[jira] [Commented] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

2021-07-14 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380902#comment-17380902 ] Xinrong Meng commented on SPARK-36031: -- Hi [~yikunkero], I dispatched the examples in the

[jira] [Comment Edited] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

2021-07-14 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380902#comment-17380902 ] Xinrong Meng edited comment on SPARK-36031 at 7/14/21, 11:51 PM: - Hi

[jira] [Created] (SPARK-36143) Adjust astype of Series of ExtensionDtype to follow pandas

2021-07-14 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36143: Summary: Adjust astype of Series of ExtensionDtype to follow pandas Key: SPARK-36143 URL: https://issues.apache.org/jira/browse/SPARK-36143 Project: Spark

[jira] [Updated] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

2021-07-14 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36031: - Description: There are many operations for series containing nan values that don't follow

[jira] [Created] (SPARK-36142) Adjust exponentiation between ExtentionDtypes and bools to follow pandas

2021-07-14 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36142: Summary: Adjust exponentiation between ExtentionDtypes and bools to follow pandas Key: SPARK-36142 URL: https://issues.apache.org/jira/browse/SPARK-36142 Project:

[jira] [Updated] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

2021-07-14 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36031: - Description: There are many operations for series doen't follow the pandas, such as:

[jira] [Created] (SPARK-36141) Support arithmetic operations of Series containing Decimal(np.nan)

2021-07-14 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36141: Summary: Support arithmetic operations of Series containing Decimal(np.nan) Key: SPARK-36141 URL: https://issues.apache.org/jira/browse/SPARK-36141 Project: Spark

[jira] [Updated] (SPARK-36031) Keep same behavior with pandas for operations of series with nan

2021-07-14 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36031: - Description: There are many operations for series doen't follow the pandas, such as:

[jira] [Created] (SPARK-36140) Replace DataTypeOps tests that have operations on different Series

2021-07-14 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36140: Summary: Replace DataTypeOps tests that have operations on different Series Key: SPARK-36140 URL: https://issues.apache.org/jira/browse/SPARK-36140 Project: Spark

[jira] [Reopened] (SPARK-36127) Adjust non-equality comparison operators to accept scalar

2021-07-13 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng reopened SPARK-36127: -- > Adjust non-equality comparison operators to accept scalar >

[jira] [Updated] (SPARK-36125) Implement non-equality comparison operators of Categoricals

2021-07-13 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36125: - Summary: Implement non-equality comparison operators of Categoricals (was: Implement

[jira] [Updated] (SPARK-36125) Implement non-equality comparison operators between two Categoricals

2021-07-13 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36125: - Summary: Implement non-equality comparison operators between two Categoricals (was: Implement

[jira] [Resolved] (SPARK-36127) Adjust non-equality comparison operators to accept scalar

2021-07-13 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-36127. -- Resolution: Duplicate > Adjust non-equality comparison operators to accept scalar >

[jira] [Updated] (SPARK-36125) Implement non-equality comparison operators between two Categoricals

2021-07-13 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36125: - Description: Implement non-equality comparison operators between two Categoricals (was:

[jira] [Updated] (SPARK-36125) Implement non-equality comparison operators between two Categoricals

2021-07-13 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36125: - Summary: Implement non-equality comparison operators between two Categoricals (was: Implement

[jira] [Created] (SPARK-36127) Adjust non-equality comparison operators to accept scalar

2021-07-13 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36127: Summary: Adjust non-equality comparison operators to accept scalar Key: SPARK-36127 URL: https://issues.apache.org/jira/browse/SPARK-36127 Project: Spark

[jira] [Created] (SPARK-36126) Adjust equality comparison operators of Categorical to follow pandas

2021-07-13 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36126: Summary: Adjust equality comparison operators of Categorical to follow pandas Key: SPARK-36126 URL: https://issues.apache.org/jira/browse/SPARK-36126 Project: Spark

[jira] [Created] (SPARK-36125) Implement non-equality comparison operators between two categories

2021-07-13 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36125: Summary: Implement non-equality comparison operators between two categories Key: SPARK-36125 URL: https://issues.apache.org/jira/browse/SPARK-36125 Project: Spark

[jira] [Updated] (SPARK-36104) Manage InternalField in DataTypeOps.neg/abs

2021-07-12 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36104: - Description: The spark data type and nullability must be the same as the original when 

[jira] [Created] (SPARK-36104) Manage InternalField in DataTypeOps.neg/abs

2021-07-12 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36104: Summary: Manage InternalField in DataTypeOps.neg/abs Key: SPARK-36104 URL: https://issues.apache.org/jira/browse/SPARK-36104 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-35337) pandas API on Spark: Separate basic operations into data type based structures

2021-07-12 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng resolved SPARK-35337. -- Resolution: Done > pandas API on Spark: Separate basic operations into data type based

[jira] [Updated] (SPARK-36003) Implement unary operator `invert` of integral ps.Series/Index

2021-07-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36003: - Description:   {code:java} >>> ~ps.Series([1, 2, 3]) Traceback (most recent call last): ...

[jira] [Updated] (SPARK-36003) Implement unary operator `invert` of integral ps.Series/Index

2021-07-09 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36003: - Summary: Implement unary operator `invert` of integral ps.Series/Index (was: Implement unary

[jira] [Created] (SPARK-36078) Complete mappings between numpy literals and Spark data types

2021-07-09 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36078: Summary: Complete mappings between numpy literals and Spark data types Key: SPARK-36078 URL: https://issues.apache.org/jira/browse/SPARK-36078 Project: Spark

[jira] [Created] (SPARK-36077) Support numpy literals as input for pandas-on-Spark APIs

2021-07-09 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-36077: Summary: Support numpy literals as input for pandas-on-Spark APIs Key: SPARK-36077 URL: https://issues.apache.org/jira/browse/SPARK-36077 Project: Spark

[jira] [Updated] (SPARK-36035) Adjust `test_astype`, `test_neg` for old pandas versions

2021-07-07 Thread Xinrong Meng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng updated SPARK-36035: - Description: * test_astype For pandas < 1.1.0, declaring or converting to StringDtype was in

<    3   4   5   6   7   8   9   10   >