gaogaotiantian commented on PR #53161: URL: https://github.com/apache/spark/pull/53161#issuecomment-3624246960
Well Python will fail when naive and aware datetimes are mixed - there's no way for spark to avoid it. It is a breaking change I agree, that's why I propose to but it under an optional flag. The reason we should have this is what I've mentioned above - we simply can't make it right when we mix them. Say the user creates a datetime then changes the session config, vs they create a session config then create a datetime - should the behavior be the same? We won't even be able to distinguish. The only way to make it work properly with easy to explain rules, is to strictly not mix them. The existing code might work in some naive cases, we can keep it, but when the user tries to do something complicated with their timestamps, they will have issues. We should at least provide one way for them to always get the correct and consistent result. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
