[
https://issues.apache.org/jira/browse/DERBY-6111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13845425#comment-13845425
]
johannes stadler commented on DERBY-6111:
-----------------------------------------
-----------------------------------------------------------------------
Johannes Stadler
--------------- DE ---------------
Ich bin vom 29.11.2013 17:00 - 02.01.2014 07:00 abwesend.
Ich werde Ihre Nachricht nach meiner Rückkehr beantworten.
In dringenden Fällen wenden Sie sich bitte an unseren Support:
E-Mail: [email protected]
Telefon: +49 (0)8541 90390-390
--------------- EN ---------------
I'm out of office 29.11.2013 17:00 - 02.01.2014 07:00.
I will respond to your message after having returned.
In urgent cases please contact our support department:
eMail: [email protected]
Phone: +49 (0)8541 90390-390
----------------------------------
---------------------------------------------------------------------
FOCONIS ... mehr Zeit fürs Wesentliche
----------------------------------------------------------------------
Anschrift und Sitz:
FOCONIS AG, Furtgasse 2, 94474 Vilshofen
Telefon: +49 (0) 8541 90390-0
Telefax: +49 (0) 8541 90390-900
Vorstand: Werner Strauß, Olaf Pulwey
Aufsichtsrat: Johann Mühlbauer (Vors.)
Amtsgericht Passau: HRB 6572
USt-ID-Nr: DE 211 388 566
----------------------------------------------------------------------
> OutOfMemoryError using CREATE INDEX on large tables
> ---------------------------------------------------
>
> Key: DERBY-6111
> URL: https://issues.apache.org/jira/browse/DERBY-6111
> Project: Derby
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 10.8.2.2, 10.9.1.0, 10.10.1.1
> Environment: Windows 7, different JREs (1.4.2_09, 1.5.0_11, 7)
> Reporter: Johannes Stadler
> Priority: Critical
> Labels: CREATE, INDEX, OutOfMemory, OutOfMemoryError,
> derby_triage10_11
> Attachments: createIndexOOM.zip, java_pid3236.zip
>
>
> I'm experiencing OutOfMemoryErorrs when performing a simple CREATE INDEX
> command on tables with more than 500,000 rows.
> The crashes occured not deterministically in our standard environment using
> 64MByte heap space. But you can easily reproduce the error using the the
> repro database attached, when running it with 12MByte heap space.
> Just start ij with the -Xmx12M JVM argument, connect to the sample db and
> execute
> CREATE INDEX IDX_CUSTOMER_NAME ON CUSTOMER (NAME).
> I've done some investigation and i was able to track down the error. It
> occurs in SortBuffer.insert(), but not as expected in NodeAllocator.newNode()
> (there is a handler for the OOE), but already in the call of
> sortObserver.insertDuplicateKey() or .insertNonDuplicateKey() (where the data
> value descriptors are cloned).
> Unfortunately this is not the point to fix it. As i caused the MergeRun (that
> spills the buffer to disk) to happen earlier, it did not significantly lower
> the memory consumption. Instead it created about 13,000 temp files with only
> 1KByte size (because of the many files, performance was inacceptable).
> So i analyzed the heap (using the HeapDumpOnOutOfMemory option) and saw that
> it's not the sortbuffer that consumes most of the memory (just few KBytes and
> about 6% of the memory), but the ConcurrentCache. Even though the maxSize of
> the ConcurrentCache was set 1000, the cache contained about 2,500 elements.
> I've also attached the heapdump.
> If i'm understanding the concept right, the cache elements are added without
> regarding the maxSize and there's a worker thread that runs on low prio, that
> shrinks the cache from time to time to 10% of its size.
> I think in this particular case, where memory is getting low, it would be a
> better idea to have the cache cleared synchronously and provide more space to
> the sortBuffer. Maybe that could be done in the ClockPolicy.insertEntry() in
> case, that the current size is increasing the max size by 50%. I'm not
> already very familiar with the code, so i failed to do so.
> I hope you got all the information you need, if you require any further
> information, please let me know.
> Greetings
> Johannes Stadler
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
