Re: java.lang.OutOfMemoryError: GC overhead limit exceeded

2019-09-11 Thread Jiang Wu
Thanks Vova. Let me know if you want any more information from me? I can share the SQL that produced this particular set of data objects in memory if you need it. -- Jiang On Wed, Sep 11, 2019 at 7:23 AM Vova Vysotskyi wrote: > Hi Jiang, > > Thanks for catching this issue. It was caused by t

Re: java.lang.OutOfMemoryError: GC overhead limit exceeded

2019-09-11 Thread Vova Vysotskyi
Hi Jiang, Thanks for catching this issue. It was caused by the fix for DRILL-6524 , and this additional structure is required to control method execution flow to be able to do some useful things. I have created DRILL-7372

Re: java.lang.OutOfMemoryError: GC overhead limit exceeded

2019-09-10 Thread Jiang Wu
Hi Paul, thanks for the tip. I will set it up to see if that makes a difference. Looking at the heap dump, it appears that there are 19,971 O(20k) AssignmentTrackingFrame and the first AssignmentTrackingFrame has 1,260 localVariablesSet size. And each localVariableSet is made of many individual I

Re: java.lang.OutOfMemoryError: GC overhead limit exceeded

2019-09-10 Thread Paul Rogers
Hi Jiang, Many factors influence memory usage; the trick is to tease them apart. An obvious question is the memory use of your custom storage plugin. This will depend on any buffering done by the plugin, the number of threads (minor fragments) per node, and the number of concurrent queries. Sin