itholic opened a new pull request, #37739: URL: https://github.com/apache/spark/pull/37739
### What changes were proposed in this pull request? This PR proposes to fix the inconsistent behavior for `Index.intersection` function as below: When `other` is list of tuple, the behavior of pandas API on Spark is difference from pandas. - pandas API on Spark ```python >>> psidx Int64Index([1, 2, 3, 4], dtype='int64', name='Koalas') >>> psidx.intersection([(1, 2), (3, 4)]).sort_values() MultiIndex([], ) ``` - pandas ```python >>> pidx Int64Index([1, 2, 3, 4], dtype='int64', name='Koalas') >>> pidx.intersection([(1, 2), (3, 4)]).sort_values() Traceback (most recent call last): ... ValueError: Names should be list-like for a MultiIndex ``` ### Why are the changes needed? To reach parity with pandas. ### Does this PR introduce _any_ user-facing change? Yes, the behavior of `Index.intersection` is chaged, when the `other` is list of tuple: - Before ```python >>> psidx Int64Index([1, 2, 3, 4], dtype='int64', name='Koalas') >>> psidx.intersection([(1, 2), (3, 4)]).sort_values() MultiIndex([], ) ``` - After ```python >>> psidx Int64Index([1, 2, 3, 4], dtype='int64', name='Koalas') >>> psidx.intersection([(1, 2), (3, 4)]).sort_values() Traceback (most recent call last): ... ValueError: Names should be list-like for a MultiIndex ``` ### How was this patch tested? Added a unit test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
