[
https://issues.apache.org/jira/browse/CALCITE-4994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482648#comment-17482648
]
Julian Hyde edited comment on CALCITE-4994 at 1/26/22, 5:51 PM:
----------------------------------------------------------------
Thanks for writing the benchmark. It proves that your change is beneficial.
(I have started asking for benchmarks for all changes like this. It allows us
to focus on changes that have a real impact, and skip changes that make the
code a little faster but much more complex. Also, we will be able to see
whether future proposed changes help or hurt your benchmark.)
Yes, please benchmark against the function. The function I have in mind is
simply {{name -> rowType.getFieldNames().indexOf(name)}}. Calling it is O\(n),
and it is called O\(n) times, but we save the O\(n) cost of building a Map.
was (Author: julianhyde):
Thanks for writing the benchmark. It proves that your change is beneficial.
(I have started asking for benchmarks for all changes like this. It allows us
to focus on changes that have a real impact, and skip changes that make the
code a little faster but much more complex. Also, we will be able to see
whether future proposed changes help or hurt your benchmark.)
Yes, please benchmark against the function. The function I have in mind is
simply {{name -> rowType.getFieldNames().indexOf(name)}}. Calling it is O(n),
and it is called O(n) times, but we save the O(n) cost of building a Map.
> SqlToRelConverter creates FieldMap for every Identifier Instead of Memoizing
> it
> -------------------------------------------------------------------------------
>
> Key: CALCITE-4994
> URL: https://issues.apache.org/jira/browse/CALCITE-4994
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Reporter: Jay Narale
> Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When converting from Sql To Rel, In SqlToRelConverter for every single
> instance of an identifier we create a new map in
> *_org.apache.calcite.sql2rel.SqlToRelConverter.Blackboard#lookupExp_*
>
> {code:java}
> final Map<String, Integer> fieldOffsets = new HashMap<>();
> for (RelDataTypeField f : resolve.rowType().getFieldList()) {
> if (!fieldOffsets.containsKey(f.getName())) {
> fieldOffsets.put(f.getName(), f.getIndex());
> }
> }
> final Map<String, Integer> map = ImmutableMap.copyOf(fieldOffsets);{code}
>
> So for a Sql Query
> {code:java}
> SELECT name, nation FROM customer{code}
> We would do the above operation twice.
> Memoization of this information will improve performance.
> In my database, I had observed that for a large table involving 1200 columns
> and a huge select having multiple expressions and operators, this part was a
> bottleneck.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
