I replied you some days ago from gmail but I'm not sure you received it cause I don't see the relevant message on nabble, so I repost it there. In the meantime Noel fixed the Recover tool issue due to max int size overflow.
/Sorry for spreading info in so many replies. I'll try to recap here. About setCharacterStream it was introduced several years ago for buggy JDBC drivers compatibility (maybe could be safely removed in most cases). About the negative numbers, I shared them at http://h2-database.66688.n3.nabble.com/h2-Continuous-Increase-in-H2-db-size-after-dropping-and-loading-same-data-repeatedly-tp4026836p4027139.html I repost it here for your convenience About the pending transaction, I suspect you are right: it's likely the cause of most of the issues. However the code is not mine, but it seems ok to me: it's single threaded, it acquires a single connection at the begin, sets the autocommit to false and then starts adding batches: it commits every N batch inserts and at the end of every table. At the end it restores the old autocommit (thought I don't remember if it actually closes the connection at the end... I'll check it). I'm going to add some log (just to see if there's some corner case that I didn't notice) and eventually debug. It would indeed be very useful if I had some sort of API (even H2 internal) exposing the connection transactional status (pending statements and so on)./ Thomas Mueller-6 wrote > Hi, > > About setCharacterStream: I would use setString for strings that easily > fit > in memory. > > About the negative numbers: why don't you post which numbers where > negative? I guess it's a bug in the Recover tool (using int instead of > long). > > Regards, > Thomas > > > > On Wed, Aug 14, 2013 at 6:16 PM, davide.cavestro > < > davide.cavestro@ > >wrote: > >> I ran the drop/import process on a smaller data set, actually the same >> used >> on the tests I've used to reply to Thomas some hours ago (please note >> those >> tests used /PreparedStatement.setCharacterStream()/). >> >> I did some raw measurements launching Tomcat from the Eclipse IDE: the >> entire drop/import process took 20-25% less using >> /PreparedStatement.setString()/ instead of >> /PreparedStatement.setCharacterStream()/. >> Using /PreparedStatement.setCharacterStream()/ Tomcat takes ~7 seconds to >> go >> down, both after a single drop/import and two sequential ones, but in the >> latter case Eclipse complains that /Terminate failed/. With a single >> drop/import I got no error messages, but with two drop/import it seems >> systematic. From Eclipse I have no way to get more details on the error >> cause, however I've seen the tomcat JVM process disappears after 7 >> seconds. >> Using /PreparedStatement.setString()/ Tomcat takes ~4 seconds to go down, >> both after a single drop/import and two sequential ones. >> >> Using /PreparedStatement.setCharacterStream()/, a Tomcat shutdown after a >> single drop/import reduces the DB size from 755MB (with Tomcat alive) to >> 584MB (after Tomcat shutdown). Shutting down Tomcat after 2 drop/import >> within the same JVM instance the DB size of 1267MB (with Tomcat alive) is >> reduced to 1156MB (Tomcat stopped). >> Using /PreparedStatement.setString()/, a Tomcat shutdown after a single >> drop/import reduces the DB size from 755MB (with Tomcat alive) to 556MB >> (after Tomcat shutdown). Shutting down Tomcat after 2 drop/import within >> the >> same JVM instance the DB size of 1267MB (with Tomcat alive) is reduced to >> 1096MB (Tomcat stopped). >> >> With /PreparedStatement.setCharacterStream()/, running the Recover on the >> database after two drop/import within the same JVM instance I got the >> following statistics >> >> >> >> With /PreparedStatement.setString()/, running the Recover on the database >> after two drop/import within the same JVM instance I got the following >> statistics >> >> >> PS: about the negative numbers I've seen on the statistics for the 17GB >> db >> I've posted some hours ago... could it be that the row count is kept into >> an >> integer variable, and I simply exceeded /Integer.MAX_SIZE/? >> >> >> Noel Grandin wrote >> > On Tue, Aug 13, 2013 at 5:56 PM, davide.cavestro >> > < >> >> > davide.cavestro@ >> >> > > wrote: >> >> I've tried replacing setCharacterStream() with setString() but the db >> >> size >> >> still continues to increase. Is there any way to get an idea of actual >> >> fragmentation of internal data-structures? >> >> >> > >> > Sorry, no, no idea. >> > But I thought we were chasing a bug where shutdown was taking too >> > long, in which case making this change should make a difference. >> > Did you measure your new shutdown time with this change? >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups >> > "H2 Database" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an >> > email to >> >> > h2-database+unsubscribe@ >> >> > . >> > To post to this group, send email to >> >> > h2-database@ >> >> > . >> > Visit this group at http://groups.google.com/group/h2-database. >> > For more options, visit https://groups.google.com/groups/opt_out. >> >> >> >> >> >> -- >> View this message in context: >> http://h2-database.66688.n3.nabble.com/h2-Continuous-Increase-in-H2-db-size-after-dropping-and-loading-same-data-repeatedly-tp4026836p4027141.html >> Sent from the H2 Database mailing list archive at Nabble.com. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "H2 Database" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to > h2-database+unsubscribe@ > . >> To post to this group, send email to > h2-database@ > . >> Visit this group at http://groups.google.com/group/h2-database. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > -- > You received this message because you are subscribed to the Google Groups > "H2 Database" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to > h2-database+unsubscribe@ > . > To post to this group, send email to > h2-database@ > . > Visit this group at http://groups.google.com/group/h2-database. > For more options, visit https://groups.google.com/groups/opt_out. -- View this message in context: http://h2-database.66688.n3.nabble.com/h2-Continuous-Increase-in-H2-db-size-after-dropping-and-loading-same-data-repeatedly-tp4026836p4027161.html Sent from the H2 Database mailing list archive at Nabble.com. -- You received this message because you are subscribed to the Google Groups "H2 Database" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/groups/opt_out.
