[
https://issues.apache.org/jira/browse/DRILL-7372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964033#comment-16964033
]
ASF GitHub Bot commented on DRILL-7372:
---------------------------------------
vvysotskyi commented on pull request #1887: DRILL-7372: MethodAnalyzer consumes
too much memory
URL: https://github.com/apache/drill/pull/1887#discussion_r341122302
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/compile/bytecode/MethodAnalyzer.java
##########
@@ -174,14 +171,22 @@ public void initJumpTarget(int opcode, LabelNode target)
{
case IF_ICMPLE:
case IF_ACMPEQ:
case IF_ACMPNE:
+ case IFNULL:
case IFNONNULL:
// for the case when conditional block is handled, creates new
variables set
// to store local variables declared inside current conditional
block and
// stores its target LabelNode to restore previous variables set
after conditional block is ended
- localVariablesSet.push(new HashSet<>());
+ localVariablesSet.push(new BitSet());
labelsStack.push(target);
}
}
}
+
+ @Override
+ public void clearStack() {
+ super.clearStack();
Review comment:
Thanks, done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> MethodAnalyzer consumes too much memory
> ---------------------------------------
>
> Key: DRILL-7372
> URL: https://issues.apache.org/jira/browse/DRILL-7372
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.16.0
> Reporter: Vova Vysotskyi
> Assignee: Vova Vysotskyi
> Priority: Major
> Fix For: 1.17.0
>
>
> In the scope of DRILL-6524 was added logic for determining whether a variable
> is assigned in conditional block to prevent incorrect scalar replacement for
> such cases.
> But for some queries, this logic consumes too many memory, for example, for
> the following query:
> {code:sql}
> SELECT *
> FROM cp.`employee.json`
> WHERE employee_id+0 < employee_id
> OR employee_id+1 < employee_id
> AND employee_id+2 < employee_id
> OR employee_id+3 < employee_id
> AND employee_id+4 < employee_id
> OR employee_id+5 < employee_id
> AND employee_id+6 < employee_id
> OR employee_id+7 < employee_id
> AND employee_id+8 < employee_id
> OR employee_id+9 < employee_id
> AND employee_id+10 < employee_id
> OR employee_id+11 < employee_id
> AND employee_id+12 < employee_id
> OR employee_id+13 < employee_id
> AND employee_id+14 < employee_id
> OR employee_id+15 < employee_id
> AND employee_id+16 < employee_id
> OR employee_id+17 < employee_id
> AND employee_id+18 < employee_id
> OR employee_id+19 < employee_id
> AND employee_id+20 < employee_id
> OR employee_id+21 < employee_id
> AND employee_id+22 < employee_id
> OR employee_id+23 < employee_id
> AND employee_id+24 < employee_id
> OR employee_id+25 < employee_id
> AND employee_id+26 < employee_id
> OR employee_id+27 < employee_id
> AND employee_id+28 < employee_id
> OR employee_id+29 < employee_id
> AND employee_id+30 < employee_id
> OR employee_id+31 < employee_id
> AND employee_id+32 < employee_id
> OR employee_id+33 < employee_id
> AND employee_id+34 < employee_id
> OR employee_id+35 < employee_id
> AND employee_id+36 < employee_id
> OR employee_id+37 < employee_id
> AND employee_id+38 < employee_id
> OR employee_id+39 < employee_id
> AND employee_id+40 < employee_id
> OR employee_id+41 < employee_id
> AND employee_id+42 < employee_id
> OR employee_id+43 < employee_id
> AND employee_id+44 < employee_id
> OR employee_id+45 < employee_id
> AND employee_id+46 < employee_id
> OR employee_id+47 < employee_id
> AND employee_id+48 < employee_id
> OR employee_id+49 < employee_id
> AND TRUE;
> {code}
> Drill consumes more than 6 GB memory.
> One of the issues to fix is to replace {{Deque<Set<Integer>>
> localVariablesSet;}} with {{Deque<BitSet>}}, it will reduce memory usage
> significantly.
> Additionally should be investigated why these objects cannot be collected by
> GC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
