[h2] Re: Corrupted database (again) with h2-1.4.199

It is getting monotonic but we had another database corruption. It happened on the same database I have been doing these tests on but since this is one of the bigger databases we have that may not be remarkable. Today we used a different recovery approach: instead of restoring a backup (which

[h2] Re: Corrupted database (again) with h2-1.4.199

Hi Andrei, Did you get anywhere from here using my feedback? Or do you need more information? I would be very interested to hear more about this problem. We just had a very peculiar database corruption: the corrupted database (~5Mb) can be accessed from 199 but it appears to be as good as

[h2] Re: Corrupted database (again) with h2-1.4.199

Hi Andrei, Thank you again for delving into this. I built from master branch and first ran the recover tool. You are definitely on to something here because where 199 trips over the database with an exception during the generation of database.mv.txt master generates a similarly sized

[h2] Re: Corrupted database (again) with h2-1.4.199

Hi Silvio, I just merged into master presumable fix for your issue , but I can not really verify that it works. If you can rebuild h2 from master branch again and try to open that corrupted file... -- You received this message because you

[h2] Re: Corrupted database (again) with h2-1.4.199

I managed to build h2 with openjdk8. The recover tool fails in the same way with that version as it does with current 199. I tried with 198 and that fails also. Then I tried 197 and that succeeds, just like 196. -- You received this message because you are subscribed to the Google Groups "H2

[h2] Re: Corrupted database (again) with h2-1.4.199

I just tried to build but I am running openjdk13 (openjdk12 on the servers) and the build fails (package com.sun.javadoc does not exist). I used to be able to build H2 earlier when I was running openjdk11. I will try building with an older JDK tomorrow. Any preference for one? -- You received

[h2] Re: Corrupted database (again) with h2-1.4.199

Too bad 8-( I was thinking about running recovery tool under debugger to see what's going on. BTW, error from recovery is different from the one database it throwing. Recovery is saying that file was kind of illegally truncated and is too short now, while database is complaining about

[h2] Re: Corrupted database (again) with h2-1.4.199

Unfortunately not. The data belongs to a customer. I would be happy to do any kind of testing for you though. -- You received this message because you are subscribed to the Google Groups "H2 Database" group. To unsubscribe from this group and stop receiving emails from it, send an email to

[h2] Re: Corrupted database (again) with h2-1.4.199

Are there any chance, you can share original corrupted file? -- You received this message because you are subscribed to the Google Groups "H2 Database" group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to

[h2] Re: Corrupted database (again) with h2-1.4.199

For what it's worth: using 196 directly on the original corrupted database (so not to run the recovery tool but with our application) shows no signs of the database being corrupted. I know this does not mean anything but nothing I have tried showed any sign of errors or malfunction. Consider me

[h2] Re: Corrupted database (again) with h2-1.4.199

My first inspection of the recovered database reveals that the database seems to be fully intact. I compared it with a backup from the day before the corruption happened. The exact same set of tables is there and the record counts in corresponding tables is the same or the differences are

[h2] Re: Corrupted database (again) with h2-1.4.199

Today I discovered something very interesting. I took a corrupted database that 199 tripped over and tried running the recovery tool. This resulted in a similar exception our application logged at the time the database got corrupted: jambo@jambo8:~$ java -cp h2-1.4.199.jar org.h2.tools.Recover

Re: [h2] Re: Corrupted database (again) with h2-1.4.199

Thanks. I will take that into account. -- You received this message because you are subscribed to the Google Groups "H2 Database" group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to this group, send

Re: [h2] Re: Corrupted database (again) with h2-1.4.199

On 2019/05/22 11:21 AM, Silvio wrote: To determine our options I would like to raise my earlier question again: is there a binary difference in MVStore/PageStore formats between version 199 and 196? In other words: would it be safe to revert to 196 without doing dump/recreate cycles for all

Re: [h2] Re: Corrupted database (again) with h2-1.4.199

Yesterday evening at about 20:00 another database on one of our servers got corrupted (at least that is when errors started appearing in the logging). There was limited server load at the time. In this case it was a database of about 850Mb. To determine our options I would like to raise my

Re: [h2] Re: Corrupted database (again) with h2-1.4.199

Thank you, that is an excellent suggestion. The times are very short now but there is no pressing reason for that other than hoping it would reduce resource consumption. I will change that ASAP. -- You received this message because you are subscribed to the Google Groups "H2 Database" group.

Re: [h2] Re: Corrupted database (again) with h2-1.4.199

Additionally, large write queues are very common in our system. At least one of the most recent corruptions occurred when a user performed an action that caused a new table to be created and ~200K records to be inserted into it (using separate connect/insert/disconnect cycles, pooled of

Re: [h2] Re: Corrupted database (again) with h2-1.4.199

On 2019/05/21 4:21 PM, Silvio wrote: The database URLs currently look like: None of that sounds problematic. You might be able to work around this issue simply by telling your pool to keep connections alive for longer, which will reduce the number of startup/shutdown cycles each database

Re: [h2] Re: Corrupted database (again) with h2-1.4.199

The database URLs currently look like: jdbc:h2:/path/to/file;IGNORECASE=TRUE;MVCC=TRUE;LOCK_TIMEOUT=1 I have experimented with different FILE_LOCK settings but since the databases are accessed from one single process this is unlikely to have any influence. I could probably do without

Re: [h2] Re: Corrupted database (again) with h2-1.4.199

What does one of your database URLs look like? Are you perhaps call Thread.interrupt somewhere? If so, that would close the database abruptly and force it to be re-opened on the next connection attempt. Doing that enough times is likely to cause corruption, because all kinds of stuff could be

[h2] Re: Corrupted database (again) with h2-1.4.199

Sorry for spamming but I am just thinking out loud here: Suppose we do leak connections across threads and multiple writes and/or closes are performed on the same connections. Could that cause database corruption? I know it is a hard call but does it sound likely or even possible that changes

[h2] Re: Corrupted database (again) with h2-1.4.199

Unfortunately we currently have another case of database corruption. The exception stack trace is identical to the one I posted above. What strikes me is that the messages reads "database: flush" and subsequently the stack trace seems to imply the database is being opened. That sounds

[h2] Re: Corrupted database (again) with h2-1.4.199

Yes, databases are shutdown and reopened constantly since the connection pool will only keep unused connections around for a short period of time before closing them. As a result databases are closed by H2 automatically once the last connection is closed. This particular type of issue appears

[h2] Re: Corrupted database (again) with h2-1.4.199

Your stack trace indicates that database is starting, so it is not obvious that it happened "during normal production use of the system". It was shut down (possibly abruptly?) before this. It was more than enough changes, of course, between 1.4.196 and 1.4.199, and many of them are targeted at