[
https://issues.apache.org/jira/browse/CALCITE-4994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17483643#comment-17483643
]
Jay Narale edited comment on CALCITE-4994 at 1/28/22, 8:28 AM:
---------------------------------------------------------------
[~julianhyde] I did all the benchmarks again. Unfortunately your changes had no
perf boost. I am currently looking into the reason if it lies in my benchmark
or something else.
Previous
{noformat}
Benchmark
(columnLength) (length) Mode Cnt Score Error Units
RelConversionBenchmark.convertToRel
10 10000 avgt 10 297.593 ± 31.218 ms/op
RelConversionBenchmark.convertToRel
100 10000 avgt 10 629.753 ± 41.458 ms/op
RelConversionBenchmark.convertToRel
1000 10000 avgt 10 4595.974 ± 86.444 ms/op
RelConversionBenchmark.parse
10 10000 avgt 10 831.633 ± 43.468 ms/op
RelConversionBenchmark.parse
100 10000 avgt 10 1642.692 ± 63.149 ms/op
RelConversionBenchmark.parse
1000 10000 avgt 10 11443.488 ± 782.940 ms/op{noformat}
Map backed List
{noformat}
Benchmark
(columnLength) (length) Mode Cnt Score Error Units
RelConversionBenchmark.convertToRel
10 10000 avgt 10 269.442 ± 22.770 ms/op
RelConversionBenchmark.convertToRel
100 10000 avgt 10 642.567 ± 49.528 ms/op
RelConversionBenchmark.convertToRel
1000 10000 avgt 10 4505.452 ± 168.228 ms/op
RelConversionBenchmark.parse
10 10000 avgt 10 821.649 ± 29.873 ms/op
RelConversionBenchmark.parse
100 10000 avgt 10 1908.704 ± 47.444 ms/op
RelConversionBenchmark.parse
1000 10000 avgt 10 13214.205 ± 96.598 ms/op{noformat}
Map At blackboard
{noformat}
Benchmark
(columnLength) (length) Mode Cnt Score Error Units
RelConversionBenchmark.convertToRel
10 10000 avgt 10 289.291 ± 23.526 ms/op
RelConversionBenchmark.convertToRel
100 10000 avgt 10 498.313 ± 51.517 ms/op
RelConversionBenchmark.convertToRel
1000 10000 avgt 10 2697.696 ± 26.089 ms/op
RelConversionBenchmark.parse
10 10000 avgt 10 811.910 ± 43.344 ms/op
RelConversionBenchmark.parse
100 10000 avgt 10 1549.019 ± 38.436 ms/op
RelConversionBenchmark.parse
1000 10000 avgt 10 8808.414 ± 313.369 ms/op{noformat}
was (Author: jaynarale):
[~julianhyde] I did all the benchmark results. Unfortunately your changes had
no perf boost. I am currently looking into the reason if it lies in my
benchmark or something else.
Previous
{noformat}
Benchmark
(columnLength) (length) Mode Cnt Score Error Units
RelConversionBenchmark.convertToRel
10 10000 avgt 10 297.593 ± 31.218 ms/op
RelConversionBenchmark.convertToRel
100 10000 avgt 10 629.753 ± 41.458 ms/op
RelConversionBenchmark.convertToRel
1000 10000 avgt 10 4595.974 ± 86.444 ms/op
RelConversionBenchmark.parse
10 10000 avgt 10 831.633 ± 43.468 ms/op
RelConversionBenchmark.parse
100 10000 avgt 10 1642.692 ± 63.149 ms/op
RelConversionBenchmark.parse
1000 10000 avgt 10 11443.488 ± 782.940 ms/op{noformat}
Map backed List
{noformat}
Benchmark
(columnLength) (length) Mode Cnt Score Error Units
RelConversionBenchmark.convertToRel
10 10000 avgt 10 269.442 ± 22.770 ms/op
RelConversionBenchmark.convertToRel
100 10000 avgt 10 642.567 ± 49.528 ms/op
RelConversionBenchmark.convertToRel
1000 10000 avgt 10 4505.452 ± 168.228 ms/op
RelConversionBenchmark.parse
10 10000 avgt 10 821.649 ± 29.873 ms/op
RelConversionBenchmark.parse
100 10000 avgt 10 1908.704 ± 47.444 ms/op
RelConversionBenchmark.parse
1000 10000 avgt 10 13214.205 ± 96.598 ms/op{noformat}
Map At blackboard
{noformat}
Benchmark
(columnLength) (length) Mode Cnt Score Error Units
RelConversionBenchmark.convertToRel
10 10000 avgt 10 289.291 ± 23.526 ms/op
RelConversionBenchmark.convertToRel
100 10000 avgt 10 498.313 ± 51.517 ms/op
RelConversionBenchmark.convertToRel
1000 10000 avgt 10 2697.696 ± 26.089 ms/op
RelConversionBenchmark.parse
10 10000 avgt 10 811.910 ± 43.344 ms/op
RelConversionBenchmark.parse
100 10000 avgt 10 1549.019 ± 38.436 ms/op
RelConversionBenchmark.parse
1000 10000 avgt 10 8808.414 ± 313.369 ms/op{noformat}
> SqlToRelConverter creates FieldMap for every Identifier Instead of Memoizing
> it
> -------------------------------------------------------------------------------
>
> Key: CALCITE-4994
> URL: https://issues.apache.org/jira/browse/CALCITE-4994
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Reporter: Jay Narale
> Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When converting from Sql To Rel, In SqlToRelConverter for every single
> instance of an identifier we create a new map in
> *_org.apache.calcite.sql2rel.SqlToRelConverter.Blackboard#lookupExp_*
>
> {code:java}
> final Map<String, Integer> fieldOffsets = new HashMap<>();
> for (RelDataTypeField f : resolve.rowType().getFieldList()) {
> if (!fieldOffsets.containsKey(f.getName())) {
> fieldOffsets.put(f.getName(), f.getIndex());
> }
> }
> final Map<String, Integer> map = ImmutableMap.copyOf(fieldOffsets);{code}
>
> So for a Sql Query
> {code:java}
> SELECT name, nation FROM customer{code}
> We would do the above operation twice.
> Memoization of this information will improve performance.
> In my database, I had observed that for a large table involving 1200 columns
> and a huge select having multiple expressions and operators, this part was a
> bottleneck.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
