Hyukjin Kwon created SPARK-51287:
------------------------------------
Summary: Fix pyspark.pandas.base.IndexOpsMixin.value_counts to
pass with Arrow enabled
Key: SPARK-51287
URL: https://issues.apache.org/jira/browse/SPARK-51287
Project: Spark
Issue Type: Sub-task
Components: Pandas API on Spark, PySpark
Affects Versions: 4.0.0
Reporter: Hyukjin Kwon
{code}
warnings.warn(
**********************************************************************
File "/__w/spark/spark/python/pyspark/pandas/base.py", line 1326, in
pyspark.pandas.base.IndexOpsMixin.value_counts
Failed example:
s.index.value_counts().sort_index()
Expected:
(cow, length) 1
(cow, weight) 2
(falcon, length) 2
(falcon, weight) 1
(lama, weight) 3
Name: count, dtype: int64
Got:
{'__index_level_0__': 'cow', '__index_level_1__': 'length'} 1
{'__index_level_0__': 'cow', '__index_level_1__': 'weight'} 2
{'__index_level_0__': 'falcon', '__index_level_1__': 'length'} 2
{'__index_level_0__': 'falcon', '__index_level_1__': 'weight'} 1
{'__index_level_0__': 'lama', '__index_level_1__': 'weight'} 3
Name: count, dtype: int64
**********************************************************************
File "/__w/spark/spark/python/pyspark/pandas/base.py", line 1334, in
pyspark.pandas.base.IndexOpsMixin.value_counts
Failed example:
s.index.value_counts(normalize=True).sort_index()
Expected:
(cow, length) 0.111111
(cow, weight) 0.222222
(falcon, length) 0.222222
(falcon, weight) 0.111111
(lama, weight) 0.333333
Name: proportion, dtype: float64
Got:
{'__index_level_0__': 'cow', '__index_level_1__': 'length'} 0.111111
{'__index_level_0__': 'cow', '__index_level_1__': 'weight'} 0.222222
{'__index_level_0__': 'falcon', '__index_level_1__': 'length'} 0.222222
{'__index_level_0__': 'falcon', '__index_level_1__': 'weight'} 0.111111
{'__index_level_0__': 'lama', '__index_level_1__': 'weight'} 0.333333
Name: proportion, dtype: float64
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
