MaxGekk opened a new pull request #28067: [WIP][SPARK-31297][SQL] Speed up dates rebasing URL: https://github.com/apache/spark/pull/28067 ### What changes were proposed in this pull request? Here is the benchmark results: ``` Java HotSpot(TM) 64-Bit Server VM 1.8.0_231-b11 on Mac OS X 10.15.3 Intel(R) Core(TM) i7-4850HQ CPU @ 2.30GHz Save dates to parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ after 1582, noop 7703 7703 0 13.0 77.0 1.0X before 1582, noop 7679 7679 0 13.0 76.8 1.0X after 1582, rebase off 17668 17668 0 5.7 176.7 0.4X after 1582, rebase on 18527 18527 0 5.4 185.3 0.4X before 1582, rebase off 17526 17526 0 5.7 175.3 0.4X before 1582, rebase on 18189 18189 0 5.5 181.9 0.4X Java HotSpot(TM) 64-Bit Server VM 1.8.0_231-b11 on Mac OS X 10.15.3 Intel(R) Core(TM) i7-4850HQ CPU @ 2.30GHz Load dates from parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------ after 1582, vec off, rebase off 10582 10694 192 9.5 105.8 1.0X after 1582, vec off, rebase on 11611 11620 10 8.6 116.1 0.9X after 1582, vec on, rebase off 2982 3010 38 33.5 29.8 3.5X after 1582, vec on, rebase on 4448 4538 82 22.5 44.5 2.4X before 1582, vec off, rebase off 10559 10614 71 9.5 105.6 1.0X before 1582, vec off, rebase on 11487 11572 74 8.7 114.9 0.9X before 1582, vec on, rebase off 2894 2951 83 34.6 28.9 3.7X before 1582, vec on, rebase on 4505 4614 102 22.2 45.1 2.3X ``` ### Why are the changes needed? <!-- Please clarify why the changes are needed. For instance, 1. If you propose a new API, clarify the use case for a new API. 2. If you fix a bug, you can clarify why it is a bug. --> ### Does this PR introduce any user-facing change? <!-- If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible. If no, write 'No'. --> ### How was this patch tested? <!-- If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible. If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future. If tests were not added, please describe why they were not added and/or why it was difficult to add. -->
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
