[
https://issues.apache.org/jira/browse/CALCITE-4994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17483671#comment-17483671
]
Jay Narale edited comment on CALCITE-4994 at 1/28/22, 10:37 AM:
----------------------------------------------------------------
However previous commit 0f14355323ace283b1189cf0c62cb19c38e7479e *Optimize null
values* has similar perf in convert to rel ( SqlNode to rel) and faster in
parsing than previous (but slower than map)or maybe just using lambda is fine
{noformat}
Benchmark
(columnLength) (length) Mode Cnt Score Error Units
RelConversionBenchmark.convertToRel
10 10000 avgt 10 263.109 ± 30.978 ms/op
RelConversionBenchmark.convertToRel
100 10000 avgt 10 486.431 ± 35.005 ms/op
RelConversionBenchmark.convertToRel
1000 10000 avgt 10 2671.751 ± 52.201 ms/op
RelConversionBenchmark.parse
10 10000 avgt 10 814.821 ± 34.034 ms/op
RelConversionBenchmark.parse
100 10000 avgt 10 1481.561 ± 16.361 ms/op
RelConversionBenchmark.parse
1000 10000 avgt 10 10165.776 ± 103.432 ms/op
{noformat}
was (Author: jaynarale):
However previous commit 0f14355323ace283b1189cf0c62cb19c38e7479e *Optimize null
values* has similar perf in convert to rel ( SqlNode to rel) and faster in
parsing than previous (but slower than map)
{noformat}
Benchmark
(columnLength) (length) Mode Cnt Score Error Units
RelConversionBenchmark.convertToRel
10 10000 avgt 10 263.109 ± 30.978 ms/op
RelConversionBenchmark.convertToRel
100 10000 avgt 10 486.431 ± 35.005 ms/op
RelConversionBenchmark.convertToRel
1000 10000 avgt 10 2671.751 ± 52.201 ms/op
RelConversionBenchmark.parse
10 10000 avgt 10 814.821 ± 34.034 ms/op
RelConversionBenchmark.parse
100 10000 avgt 10 1481.561 ± 16.361 ms/op
RelConversionBenchmark.parse
1000 10000 avgt 10 10165.776 ± 103.432 ms/op{noformat}
> SqlToRelConverter creates FieldMap for every Identifier Instead of Memoizing
> it
> -------------------------------------------------------------------------------
>
> Key: CALCITE-4994
> URL: https://issues.apache.org/jira/browse/CALCITE-4994
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Reporter: Jay Narale
> Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When converting from Sql To Rel, In SqlToRelConverter for every single
> instance of an identifier we create a new map in
> *_org.apache.calcite.sql2rel.SqlToRelConverter.Blackboard#lookupExp_*
>
> {code:java}
> final Map<String, Integer> fieldOffsets = new HashMap<>();
> for (RelDataTypeField f : resolve.rowType().getFieldList()) {
> if (!fieldOffsets.containsKey(f.getName())) {
> fieldOffsets.put(f.getName(), f.getIndex());
> }
> }
> final Map<String, Integer> map = ImmutableMap.copyOf(fieldOffsets);{code}
>
> So for a Sql Query
> {code:java}
> SELECT name, nation FROM customer{code}
> We would do the above operation twice.
> Memoization of this information will improve performance.
> In my database, I had observed that for a large table involving 1200 columns
> and a huge select having multiple expressions and operators, this part was a
> bottleneck.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
