allisonwang-db commented on code in PR #42332:
URL: https://github.com/apache/spark/pull/42332#discussion_r1284896049
##########
python/pyspark/testing/pandasutils.py:
##########
@@ -450,6 +477,15 @@ def assert_eq(self, left, right, check_exact=True,
almost=False):
import pandas as pd
from pandas.api.types import is_list_like
+ # for pandas-on-Spark, ignore row order by default because row
ordering not guaranteed
+ if isinstance(left, (ps.DataFrame, ps.Series, ps.Index)):
+ return assertPandasOnSparkEqual(
+ left, right, checkExact=check_exact, almost=almost,
checkRowOrder=check_row_order
Review Comment:
Should the default value for check_row_order be false? since we want to
ignore the row ordering here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
