This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new b8e7e0d [SPARK-36930][PYTHON][FOLLOW-UP] Fix test case for
MultiIndex.dtypes with pandas version < 1.3
b8e7e0d is described below
commit b8e7e0d06447b02c30616078145eb01b1186f5db
Author: dch nguyen <[email protected]>
AuthorDate: Thu Oct 7 15:54:07 2021 +0900
[SPARK-36930][PYTHON][FOLLOW-UP] Fix test case for MultiIndex.dtypes with
pandas version < 1.3
### What changes were proposed in this pull request?
This PR is a minor followup of https://github.com/apache/spark/pull/34179
to fix test case for ```MultiIndex.dtypes``` with pandas version < 1.3.
### Why are the changes needed?
Fix test case for ```MultiIndex.dtypes``` with pandas version < 1.3.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
manual test
Closes #34206 from dchvn/SPARK-36930-FU.
Authored-by: dch nguyen <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
python/pyspark/pandas/tests/test_dataframe.py | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/python/pyspark/pandas/tests/test_dataframe.py
b/python/pyspark/pandas/tests/test_dataframe.py
index 800fa46..701052e 100644
--- a/python/pyspark/pandas/tests/test_dataframe.py
+++ b/python/pyspark/pandas/tests/test_dataframe.py
@@ -6006,13 +6006,24 @@ class DataFrameTest(PandasOnSparkTestCase,
SQLTestUtils):
pmidx = pd.MultiIndex.from_arrays(arrays, names=("number", "color"))
psmidx = ps.from_pandas(pmidx)
- self.assert_eq(psmidx.dtypes, pmidx.dtypes)
+ if LooseVersion(pd.__version__) >= LooseVersion("1.3"):
+ self.assert_eq(psmidx.dtypes, pmidx.dtypes)
+ else:
+ expected = pd.Series([np.dtype("int64"), np.dtype("O")],
index=["number", "color"])
+ self.assert_eq(psmidx.dtypes, expected)
# multiple labels
pmidx = pd.MultiIndex.from_arrays(arrays, names=[("zero", "first"),
("one", "second")])
psmidx = ps.from_pandas(pmidx)
- self.assert_eq(psmidx.dtypes, pmidx.dtypes)
+ if LooseVersion(pd.__version__) >= LooseVersion("1.3"):
+ self.assert_eq(psmidx.dtypes, pmidx.dtypes)
+ else:
+ expected = pd.Series(
+ [np.dtype("int64"), np.dtype("O")],
+ index=pd.Index([("zero", "first"), ("one", "second")]),
+ )
+ self.assert_eq(psmidx.dtypes, expected)
if __name__ == "__main__":
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
