Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/17160
It does not need for `__contains__` but need for `bool` because I guess we
would not want to return bool as other operators return `Column`.
For example,
```python
>>> not spark.range(1).id
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File ".../spark/python/pyspark/sql/column.py", line 452, in __nonzero__
raise ValueError("Cannot convert column into bool: please use '&' for
'and', '|' for 'or', "
ValueError: Cannot convert column into bool: please use '&' for 'and', '|'
for 'or', '~' for 'not' when building DataFrame boolean expressions.
```
If we remove this, then
```
>>> not spark.range(1).id
False
```
I think we want `Column` as below:
```python
>>> 1 < spark.range(1).id
Column<(id > 1)>
```
but for `bool`, it seems not.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
