Re: [h2] ALTER TABLE ADD uses too much memory
On 2015-01-18 10:03 AM, Steve McLeod wrote: Noel, you mentioned that you have some suggestions for making the data copying faster? I'm keen to hear them. The quicker the ALTER TABLE ADD ... statements run, the less chance my users experience power outage, or decide to force quit the process, etc. During the startup phase of my application, I open a connection to the DB with the following settings: LOG=0;UNDO_LOG=0;LOCK_MODE=0;EXCLUSIVE=TRUE Then I perform the necessary DDL statements to make the schema up to date, Then I close the connection, Then I re-open the connection normally. Respectively, these settings: - disable transaction log - disable session undo log - set lock mode to READ_UNCOMMITTED - get an exclusive connection This means that your DDL will run at max speed, but these are settings are ONLY valid for a single, exclusive connection, and they remove a fair chunk of the safety net, so it's not something you should do on a network drive, for example. If you want to be really paranoid, make a copy of the H2 file before running your DDL. -- You received this message because you are subscribed to the Google Groups "H2 Database" group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to this group, send email to h2-database@googlegroups.com. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout.
Re: [h2] ALTER TABLE ADD uses too much memory
Hi Noel, Once I investigated deeper, I discovered that I was wrong, that the culprit is not H2's data copying. The memory issue was, of course, completely in my own code. Noel, you mentioned that you have some suggestions for making the data copying faster? I'm keen to hear them. The quicker the ALTER TABLE ADD ... statements run, the less chance my users experience power outage, or decide to force quit the process, etc. On Tuesday, 6 January 2015 19:11:10 UTC+8, Noel Grandin wrote: > > > On 2015-01-06 12:50 PM, Steve McLeod wrote: > > When our users update to the latest version of our desktop software that > uses H2, we need to add several columns to a > > table with many columns that may have millions of rows. Some people have > been getting an OutOfMemoryException during > > this process. > > > > That is rather odd, since H2 should be creating a temporary table if the > result-set gets too big. > I do this all the time, and I don't have OOM problems. It sure can take a > long time though, which I have some > suggestions for, but first lets sort out this bug. > > What does your H2 URL look like, and what does your ALTER TABLE statement > look like? > > > I've analysed this problem, and checked the H2 source code for ALTER > TABLE ADD. It seems that the high RAM usage is > > occurring during the data copying step. > > > > Can you be more specific - what data-structure is using up memory and what > section of H2 code? > > -- You received this message because you are subscribed to the Google Groups "H2 Database" group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to this group, send email to h2-database@googlegroups.com. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout.
Re: [h2] ALTER TABLE ADD uses too much memory
On 2015-01-06 12:50 PM, Steve McLeod wrote: When our users update to the latest version of our desktop software that uses H2, we need to add several columns to a table with many columns that may have millions of rows. Some people have been getting an OutOfMemoryException during this process. That is rather odd, since H2 should be creating a temporary table if the result-set gets too big. I do this all the time, and I don't have OOM problems. It sure can take a long time though, which I have some suggestions for, but first lets sort out this bug. What does your H2 URL look like, and what does your ALTER TABLE statement look like? I've analysed this problem, and checked the H2 source code for ALTER TABLE ADD. It seems that the high RAM usage is occurring during the data copying step. Can you be more specific - what data-structure is using up memory and what section of H2 code? -- You received this message because you are subscribed to the Google Groups "H2 Database" group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to this group, send email to h2-database@googlegroups.com. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout.
[h2] ALTER TABLE ADD uses too much memory
When our users update to the latest version of our desktop software that uses H2, we need to add several columns to a table with many columns that may have millions of rows. Some people have been getting an OutOfMemoryException during this process. I've analysed this problem, and checked the H2 source code for ALTER TABLE ADD. It seems that the high RAM usage is occurring during the data copying step. I'm looking for an alternative solution. Two I've thought of: 1) Manually create a replacement table, then using JDBC, fetch each existing row, inserting it into the new table. 2) Dump the current table to CSV file, then create the new table with the additional columns, and import from the CSV file Before I try these different approaches, do you have any thoughts on which might be better, or whether an alternative approach may work? -- You received this message because you are subscribed to the Google Groups "H2 Database" group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to this group, send email to h2-database@googlegroups.com. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout.
