Hi Noel, our users typically deal with huge amounts of data, which often do not fit into memory. Tables might have hundreds of columns, so that already a low amount of rows held in memory can occupy a lot of it. Additionally, queries are often issued in parallel, therefore a low value was set so that the users do not need to take care of advanced H2 settings and guarantee there will not be an out of memory error at any time. The performance was still good.
Anyway, the problem I still have is, that a high value of MAX_MEMORY_ROWS will require a small amount of memory during index creation, but lots of memory for result sets. a low value of MAX_MEMORY_ROWS will require a huge amount of memory during index creation, but only a small amount of memory for result sets. Would it be possible to introduce a new database property for the index creation? This one could default to the current default for MAX_MEMORY_ROWS. This would help us a lot. On 2019/05/21 9:52 AM, [email protected] wrote: > > I tracked this down to the *rebuildIndexBlockMerge *method of the *MVTable *class (see below). > As I saw that the *MAX_MEMORY_ROWS *parameter is used in the method, I changed its values and tried again. > With set to 1000 the index creation still required a maximum heap of about 800M, but the OOM Error did not occur anymore. > This sounds a bit strange to me, because as far as I understood *lowering *the value of that parameter should *decrease > *memory consumption. > > Is anything wrong with my configuration? Or might this be a bug? > > <Chuckle> Now that is an interesting failure mode. Given how much memory you seem to have, you should be setting MAX_MEMORY_ROWS *higher*, not lower. Running with MAX_MEMORY_ROWS set to 10 is something we only expect to see in unit tests, when we're trying to test our on-disk temporary-data code-paths. So what is happening is that we create a new temporary map for each block of MAX_MEMORY_ROWS rows, which is your case means we end up creating 8,000,000/10 = 800,000 maps! So I would suggest either leaving MAX_MEMORY_ROWS alone, which will auto-configure to a reasonable number, or configuring it such that virtually all your queries run in-memory without needing to spill temporary data to disk. Which will also mean that your application will run a whole lot faster. -- You received this message because you are subscribed to the Google Groups "H2 Database" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/h2-database. To view this discussion on the web visit https://groups.google.com/d/msgid/h2-database/277d5c3d-aeff-0a9a-0751-1694d8a0a80c%40gmail.com . For more options, visit https://groups.google.com/d/optout. ================================================================================================================ Disclaimer The information contained in this e - mail and any attachments ( together the "message") is intended for the addressee only and may contain confidential and/or privileged information. If you have received the message by mistake please delete it and notify the sender and do not copy or distribute it or disclose its contents to anyone. FINARIS Financial Software Partner GmbH, Sömmerringstr. 23, 60322 Frankfurt/Main, Germany Registered at Frankfurt/Main, HRB 52873, Managing Directors: Dipl. Inf. Hermann Friebel, Dipl. Ing. Kai Bächle, Dipl. Inf. Werner Märkl ================================================================================================================ -- You received this message because you are subscribed to the Google Groups "H2 Database" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/h2-database. To view this discussion on the web visit https://groups.google.com/d/msgid/h2-database/OFC621AD22.2AF3302E-ONC1258401.004F778A-C1258401.00513315%40finaris.de. For more options, visit https://groups.google.com/d/optout.
