Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/22293 )
Change subject: IMPALA-13627: Handle legacy Hive timezone conversion ...................................................................... Patch Set 5: (3 comments) http://gerrit.cloudera.org:8080/#/c/22293/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/22293/3//COMMIT_MSG@37 PS3, Line 37: impact on performance > Done thx for the benchmarks, it is less horrible than what I guessed - can you also mention the results in the commit message? http://gerrit.cloudera.org:8080/#/c/22293/5/be/src/benchmarks/convert-timestamp-benchmark.cc File be/src/benchmarks/convert-timestamp-benchmark.cc: http://gerrit.cloudera.org:8080/#/c/22293/5/be/src/benchmarks/convert-timestamp-benchmark.cc@156 PS5, Line 156: jvm speedup: 1.8607 it's interesting that cctz is 50x faster in single threaded mode while just 10x faster with 8 thread. My guess is that this is caused by the atomic member used during timezone lookups. http://gerrit.cloudera.org:8080/#/c/22293/3/be/src/exec/parquet/parquet-common.h File be/src/exec/parquet/parquet-common.h: http://gerrit.cloudera.org:8080/#/c/22293/3/be/src/exec/parquet/parquet-common.h@815 PS3, Line 815: /// If timestamp t >= v before conversion, then this function converts v in such a : /// way that the same will be true after t is converted. : void ConvertMinStatToLocalTime(TimestampValue* v) const; : : /// If timestamp t <= v before conversion, then this function converts v in such a : /// way that the same will be true after t is converted. : void ConvertMaxStatToLocalTime(TimestampValue* v) const; > I think one problem is that the JVM method I'm mimicking here doesn't handl >I could hard-code adding/subtracting 1hr all the time That should help near DST changes, but theoretically there can be larger gaps during historical timezone changes. Using +-1 day instead of hour should be really enough though. -- To view, visit http://gerrit.cloudera.org:8080/22293 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I1271ed1da0b74366ab8315e7ec2d4ee47111e067 Gerrit-Change-Number: 22293 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Smith <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Comment-Date: Thu, 16 Jan 2025 09:54:49 +0000 Gerrit-HasComments: Yes
