linzch3 opened a new pull request, #60092:
URL: https://github.com/apache/spark/pull/60092
### What changes were proposed in this pull request?
In `pyspark.pandas.Series.__init__`, when `data` is a pandas-on-Spark
`Series`, the `index` argument was checked with `if index:`. This PR changes it
to `if index is not None:`.
### Why are the changes needed?
`if index:` evaluates the truthiness of the argument, which breaks in two
ways:
1. Passing a pandas-on-Spark `Index` (for example the Series' own
`df.index`) raises, because `Index.__bool__` is ambiguous:
```python
>>> psdf = ps.DataFrame({'c0': [1]})
>>> ps.Series(psdf['c0'], index=psdf.index)
ValueError: The truth value of a Index is ambiguous. Use a.empty, a.bool(),
a.item(), a.any() or a.all().
```
pandas returns the Series reindexed by that index (`pd.Series(pdf['c0'],
index=pdf.index)`).
2. Passing an empty index (`index=[]`) skips the reindex entirely and
returns the full Series. pandas returns an empty Series in that case.
### Does this PR introduce _any_ user-facing change?
Yes, it's a bug fix:
- `ps.Series(psser, index=<pandas-on-Spark Index>)` now reindexes like
pandas instead of raising `ValueError`.
- `ps.Series(psser, index=[])` now returns an empty Series like pandas,
instead of the unchanged Series.
### How was this patch tested?
Extended `SeriesTests.test_series_from_series` in
`python/pyspark/pandas/tests/series/test_series.py` with a Series' own index,
another pandas-on-Spark `Index`, and an empty index, all compared against
pandas. The new assertions fail without the fix (the `ValueError` above) and
pass with it. The full `test_series.py` passes locally (44 passed). `ruff
check` and `ruff format --check` pass on the changed files.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Grok Bot (AI coding assistant)
The fix and test were written with AI assistance; the commit carries the
same `Generated-by` trailer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
