Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20163
Hey @rednaxelafx that's fine. We all make mistake and I usually think it's
always better then not trying. I also made a mistake at the first time. It was
easier to debug this with your comments and details in the PR description.
Thank you.
> I'd like to wait for more discussions / suggestions on whether or not we
want a behavior change that makes this reproducer work, or a simple document
change that'll just say PySpark doesn't support mismatching returnType.
So, few options might be ...
1. Simple document this
2. `str` logics in `type.StringType` - in this case, I think we should do a
small banchmark. It'd would be so hard and I think you could reuse commands I
used here - https://github.com/apache/spark/pull/19246#discussion_r139874732
3. Investigate the way to register a custom Pyrolite unpickler that
converts `datetime.date*` to `Timestamp` or `Date`. I believe we already have
some custom fixes there.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
