[
https://issues.apache.org/jira/browse/CALCITE-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16812871#comment-16812871
]
Valeriy Trofimov edited comment on CALCITE-2974 at 4/9/19 12:47 AM:
--------------------------------------------------------------------
Our perf numbers are based on running selected code from
SqlOperatrBaseTest.testCastDateTime() thousands of times on an instrumented
RexSimplify.simplifyCast() method. For the previously mentioned timestamp
conversion the difference is 4-7 fold perf improvement:
|| ||Current Code||Improved Code||Improvement Ratio||
||Tp50|51,403|11,454|4.48|
||Tp90|119,010|17,880|6.65|
||Tp99|174,603|37,435|4.66|
If you add similar code for string-to-int conversion you'll get 7-14 fold perf
improvement:
|| ||Current Code||Improved Code||Improvement Ratio||
||Tp50|32,406|4,470|7.24|
||Tp90|85,486|6,146|13.90|
||Tp99|132,698|10,337|12.83|
The numbers for the "Current Code" and "Improved Code" are time in nanoseconds
how long it took to execute the current code line in question - call to
reduce() and the improved code where we manually convert string to timestamp or
int.
was (Author: valtroffuture):
Our perf numbers are based on running selected code from
SqlOperatrBaseTest.testCastDateTime() thousands of times on an instrumented
RexSimplify.simplifyCast() method. For the previously mentioned timestamp
conversion the difference is 4-7 fold perf improvement:
|| ||Current Code||Improved Code||Improvement Ratio||
||Tp50|51,403|11,454|4.48|
||Tp90|119,010|17,880|6.65|
||Tp99|174,603|37,435|4.66|
If you add similar code for string-to-int conversion you'll get 7-14 fold perf
improvement:
|| ||Current Code||Improved Code||Improvement Ratio||
||Tp50|32,406|4,470|7.24|
||Tp90|85,486|6,146|13.90|
||Tp99|132,698|10,337|12.83|
The numbers for the "Current Code" and "Improved Code" are time in nanoseconds
how long it took to execute the current code line in question - call to
reduce()) and the improved code where we manually convert string to timestamp
or int.
> Timestamp conversion performance can be improved
> ------------------------------------------------
>
> Key: CALCITE-2974
> URL: https://issues.apache.org/jira/browse/CALCITE-2974
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Affects Versions: 1.19.0
> Reporter: Valeriy Trofimov
> Priority: Major
> Labels: easyfix, performance
> Fix For: next
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> RexSimplify.simplifyCast() is slow when converting a string to SQL Timestamp
> value. The slowness is caused by this line:
> {code:java}
> executor.reduce(rexBuilder, ImmutableList.of(e), reducedValues);
> {code}
> Debugging this code line with my team showed that for timestamp conversion it
> loads a pre-compiled conversion code, which makes it slow. My team proposes
> to replace this line with the following code that greately improves
> performance:
> {code:java}
> if (typeName == SqlTypeName.CHAR && e.getType().getSqlTypeName() ==
> SqlTypeName.TIMESTAMP_WITH_LOCAL_TIME_ZONE) {
> if (literal.getValue() instanceof NlsString) {
> String timestampStr = ((NlsString) literal.getValue()).getValue();
> Long timestampLong =
> SqlFunctions.toTimestampWithLocalTimeZone(timestampStr);
> reducedValues.add(rexBuilder.makeLiteral(timestampLong, e.getType(),
> true));
> }
> }
> if (reducedValues.isEmpty()) {
> executor.reduce(rexBuilder, ImmutableList.of(e), reducedValues);
> }
> {code}
> Let us know if we can submit a pull request with this code change or if we've
> missed anything. Do you know the reason behind using a pre-compiled code for
> timestamp conversion?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
