Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/18926
> Basically, the change just wants to ensure the type of length is int.
Yes, but to be more correct, I think this makes sure if both are same
types, `Column` or `int`. I think this throws an exception for a better error
message, (rather than, for example, "Unexpected types: startPos :<type 'int'>
length: <class 'pyspark.sql.column.Column'>"). I know this sounds rather
excessive checking but this is still a valid checking.
Possible way I think is with keeping this checking (which I believe is not
shorter):
```python
if isinstance(startPos, int) and isinstance(length, int):
jc = self._jc.substr(startPos, length)
elif isinstance(startPos, Column) and isinstance(length, Column)::
jc = self._jc.substr(startPos._jc, length._jc)
else:
if type(startPos) != type(length):
raise TypeError(...)
else:
raise TypeError("Unexpected type: %s" % type(startPos))
```
or, with removing this excessive checking:
```python
if isinstance(startPos, int) and isinstance(length, int):
jc = self._jc.substr(startPos, length)
elif isinstance(startPos, Column) and isinstance(length, Column)::
jc = self._jc.substr(startPos._jc, length._jc)
else:
raise TypeError(...)
```
I agree this is excessive (the former) but I wonder if we should remove
already existing one. Please correct me if I am mistaken here.
`long` does not work already but with unexpected exception message, which
this PR fixes.
The current state fixes the JIRA specified here.
I will stop staying against if you guys here feel strongly with fixing
together but if you are not, could we go without that issue?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
