alamb commented on issue #10602: URL: https://github.com/apache/datafusion/issues/10602#issuecomment-2131254591
Thank you @tustvold and @Abdullahsab3 and @mhilton and @appletreeisyellow for the thoughts. From my perspective, the current (non timezone aware) `date_bin` function has the benefit of being (relatively) simple to implement and fast (as it is some arithmetic calculation on the underlying integer value without having to do DST conversions per row) Given the differences in underlying timestamp representation between arrow and postgres I do think some difference is likely inevitable and thus likely not a deal breaker. Here are my suggested next steps @appletreeisyellow tries to prototype one or both proposals and see if we can get it to produce the desired results: 1. create a `to_local_time` function 2. Modify to the `date_bin` function to make it timezone aware I think the `to_local_time` might be the simplest approach. * A ScalarUDF * a `Signature` that takes `Timestamp(.., *)` * produces a `Timestamp(.., None)`. * The `invoke` function would do the oppositte of whatever `cast(Timestamp(.., None) --> Timestamp(.., TZ))` does -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
