I do it with a bunch of inserts. The tables already exist. It happens even if I use UNDO_LOG=0, which I think disables the undo log.
I'll try jmap. On Tuesday, September 10, 2013 11:50:43 AM UTC-7, Thomas Mueller wrote: > > Hi, > > It depends on how you do the import. Could you get a heap histogram for a > large import (jmap -histo:live <pid>), and could you tell us how you do the > import exactly? > > If you use "create table ... as select" then it can't run out of memory > (as far as I know), because no undo log is needed. > > If you execute many insert or update statements, then the undo log is > used, and at least a few bytes are needed per record (the actual data is > stored to disk at some point, but the position of the data is not). So in > theory it could run out of memory, even thought the transaction would need > to be huge. > > By the way, with the new MVStore storage (not production ready yet), there > should no longer be a limit on the size of a transaction, as no undo log is > needed. > > Regards, > Thomas > > > > > > > On Tue, Sep 10, 2013 at 10:51 AM, Ryan How <[email protected]<javascript:> > > wrote: > >> Howcome it is getting OOM on import? It shouldn't do that should it? I've >> imported several GB of data before, I did get OOM, but increasing heap size >> to around 1GB worked for me. I didn't need to go to crazy sizes... >> >> >> On 10/09/2013 4:32 PM, Noel Grandin wrote: >> >>> >>> On 2013-09-06 20:15, Brian Craft wrote: >>> >>>> I need to load about 1G of data into an existing db, while maintaining >>>> data coherence. Wrapping the inserts in one transaction results in >>>> out-of-memory problems in the jvm. I increased the max heap size to 8g w/o >>>> improvement. I can split it into a bunch of smaller commits, which works >>>> fine, but then on error I need a bunch of application code to delete the >>>> transactions which succeeded. The deletes will need their own >>>> transactions, >>>> which could also fail. >>>> >>>> Is there any better way to do this? >>>> >>>> >>> Not really. >>> One strategy would be to copy the DB, since with H2 it's just a single >>> file, and then run your import process. >>> If it fails, just replace the modified DB with the backup. >>> >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "H2 Database" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to h2-database...@**googlegroups.com <javascript:>. >> To post to this group, send email to [email protected]<javascript:> >> . >> Visit this group at >> http://groups.google.com/**group/h2-database<http://groups.google.com/group/h2-database> >> . >> For more options, visit >> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> >> . >> > > -- You received this message because you are subscribed to the Google Groups "H2 Database" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/groups/opt_out.
