viirya opened a new pull request, #157: URL: https://github.com/apache/spark-connect-rust/pull/157
### What changes were proposed in this pull request? JIRA: [SPARK-60919](https://issues.apache.org/jira/browse/SPARK-60919) Add `__hash__` to the drop-in's `Row` (`crates/pyspark-rs/src/row.rs`). It hashes the tuple of the row's values. ### Why are the changes needed? `Row` defines `__eq__` but not `__hash__`, so `hash(Row(a=1))` raises `TypeError: unhashable type: 'builtins.Row'`. In PySpark, `Row` subclasses `tuple` and overrides neither method, so it hashes like a tuple of its values: `hash(Row(a=1)) == hash((1,))`, and Rows work in sets and as dict keys. This stays consistent with `__eq__`, since equal rows have equal values. A nested struct value comes back as a `Row`, so it hashes too. An array value (a `list`) or a map value (a `dict`) still raises `TypeError`, as it does inside a tuple. ### Does this PR introduce _any_ user-facing change? Yes. `Row` objects can now be hashed, so they can go in sets and be used as dict keys, matching PySpark. ### How was this patch tested? New offline tests in `python/tests/test_dropin_offline.py`: - the hash equals the hash of the values tuple, including nested rows and an empty row - equal rows collapse in a set, and a row works as a dict key - rows holding list or dict values raise `TypeError` ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Anthropic) This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
