[ 
https://issues.apache.org/jira/browse/CALCITE-3784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17035034#comment-17035034
 ] 

Ravi Kapoor commented on CALCITE-3784:
--------------------------------------

[~julianhyde] [~hyuan] It seems shocking to me as well unless I monitored the 
memory consumption and the thread dump.
There is no multiple conditions as there is only one RexCall is created which 
is used in filter expression but this RexCall further uses  similar RexCalls 
multiple times as an expression and that uses further RexCalls and son on upto 
a depth of around 60. Actually I cant have a variable to hold the result but to 
evaluate at runtime so I had to embed the repeated expression as I am creating 
a sql query at the end.

Let me show you how my RexCall looks like. Take a look at !OOM Screenshot.png! 

The last variable StgValidationFlag is used in the filter which uses all the 
previous evaluated RexCalls
blows out of memory as the string formation is too large to load even in the 
IDE.

Moreover, these RexCalls are all if else conditions exprs.






> RexCall toString digest gives OOM while huge expression is evaluated
> --------------------------------------------------------------------
>
>                 Key: CALCITE-3784
>                 URL: https://issues.apache.org/jira/browse/CALCITE-3784
>             Project: Calcite
>          Issue Type: Bug
>          Components: core
>            Reporter: Ravi Kapoor
>            Priority: Critical
>         Attachments: OOM Screenshot.png
>
>
> I have some complex query which has tens of thousands of rex expressions and 
> this expression is used in filter expression in the query.
> On creating a filter below code gets called:
> {code}
> public RelBuilder filter(Iterable<CorrelationId> variablesSet,
>   Iterable<? extends RexNode> predicates) {
>   final RexNode simplifiedPredicates =
>   simplifier.simplifyFilterPredicates(predicates);
>   if (simplifiedPredicates == null) {
>     return empty();
>  }
> {code}
> RexSimplify further adds the rexnode in the Set<RexNode> calling hashcode() 
> internally which calls toString()
> Is there any way to avoid this computeDigest Call which creates complex 
> string object and blows up the  RAM about 14GB?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to