MaxGekk commented on issue #28067: [WIP][SPARK-31297][SQL] Speed up dates rebasing URL: https://github.com/apache/spark/pull/28067#issuecomment-605807798 > can you briefly explain your idea to optimize it? @cloud-fan The difference in days between Proleptic Gregorian and the hybrid calendar (Julian+Gregorian) doesn't change so often. If you look at the JIRA ticket [SPARK-31297](https://issues.apache.org/jira/browse/SPARK-31297?focusedCommentId=17070457&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17070457), you can see that it changed 14 times on the interval `1001-01-01`-`2030-01-01`. The idea is to build an array of days when the diff was changed, and for the given date, find the interval to which the date belongs to. > and what's the benchmark numbers before your optimization? The benchmark has not been merged yet. It waits for you approval. You can find numbers there https://github.com/apache/spark/pull/28057
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
