[
https://issues.apache.org/jira/browse/DRILL-7372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-7372:
------------------------------------
Reviewer: Arina Ielchiieva
> MethodAnalyzer consumes too many memory
> ---------------------------------------
>
> Key: DRILL-7372
> URL: https://issues.apache.org/jira/browse/DRILL-7372
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.16.0
> Reporter: Vova Vysotskyi
> Assignee: Vova Vysotskyi
> Priority: Major
> Fix For: 1.17.0
>
>
> In the scope of DRILL-6524 was added logic for determining whether a variable
> is assigned in conditional block to prevent incorrect scalar replacement for
> such cases.
> But for some queries, this logic consumes too many memory, for example, for
> the following query:
> {code:sql}
> SELECT *
> FROM cp.`employee.json`
> WHERE employee_id+0 < employee_id
> OR employee_id+1 < employee_id
> AND employee_id+2 < employee_id
> OR employee_id+3 < employee_id
> AND employee_id+4 < employee_id
> OR employee_id+5 < employee_id
> AND employee_id+6 < employee_id
> OR employee_id+7 < employee_id
> AND employee_id+8 < employee_id
> OR employee_id+9 < employee_id
> AND employee_id+10 < employee_id
> OR employee_id+11 < employee_id
> AND employee_id+12 < employee_id
> OR employee_id+13 < employee_id
> AND employee_id+14 < employee_id
> OR employee_id+15 < employee_id
> AND employee_id+16 < employee_id
> OR employee_id+17 < employee_id
> AND employee_id+18 < employee_id
> OR employee_id+19 < employee_id
> AND employee_id+20 < employee_id
> OR employee_id+21 < employee_id
> AND employee_id+22 < employee_id
> OR employee_id+23 < employee_id
> AND employee_id+24 < employee_id
> OR employee_id+25 < employee_id
> AND employee_id+26 < employee_id
> OR employee_id+27 < employee_id
> AND employee_id+28 < employee_id
> OR employee_id+29 < employee_id
> AND employee_id+30 < employee_id
> OR employee_id+31 < employee_id
> AND employee_id+32 < employee_id
> OR employee_id+33 < employee_id
> AND employee_id+34 < employee_id
> OR employee_id+35 < employee_id
> AND employee_id+36 < employee_id
> OR employee_id+37 < employee_id
> AND employee_id+38 < employee_id
> OR employee_id+39 < employee_id
> AND employee_id+40 < employee_id
> OR employee_id+41 < employee_id
> AND employee_id+42 < employee_id
> OR employee_id+43 < employee_id
> AND employee_id+44 < employee_id
> OR employee_id+45 < employee_id
> AND employee_id+46 < employee_id
> OR employee_id+47 < employee_id
> AND employee_id+48 < employee_id
> OR employee_id+49 < employee_id
> AND TRUE;
> {code}
> Drill consumes more than 6 GB memory.
> One of the issues to fix is to replace {{Deque<Set<Integer>>
> localVariablesSet;}} with {{Deque<BitSet>}}, it will reduce memory usage
> significantly.
> Additionally should be investigated why these objects cannot be collected by
> GC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
