gaogaotiantian commented on PR #53161: URL: https://github.com/apache/spark/pull/53161#issuecomment-3624345492
Okay sure. This won't be a trivial change but I can draft a PR. The tricky part is where to enforce this. I had a few thoughts in my mind: 1. Check conf directly in `fromInternal` and `toInternal` - but this has to access the session conf of spark in datatype conversion, which is a new pattern and will couple the data type and the existence of a session too much 2. Pass it as an argument to `fromInternal` and `toInternal` - still a new pattern and will have to change every container. 3. (Might go this way) Add a class-level switch variable for `TimestampType` and `TimestampNTZType` and set it when we config session (probably will be a hook somewhere) - we need to pass the confs to workers as well to make it work properly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
