Re: [h2] Possible bug in LZF compression
Hi, I confirm the issue is on my side. I made optimization in MapDB which causes the bug (replaced for loop copy with System.arrayCopy) Thanks for help and sorry for false alarm. Jan On Wednesday, May 28, 2014 13:41:24 Thomas Mueller wrote: Hi, Yes, it looks like this doesn't affect H2. Maybe it is related to the changes to the LZF compressor in MapDB? My test case: byte[] data = StringUtils.convertHexToBytes(00ef76fa135e7d216e829a53845a983469ac1e 4edb6120b79667d667e7d4f856010101010022bf4569010023002102 123eeaa90e2f5786ce028e60ec03702706dadecee373a90b09b88a99cc668f46ac33 58c8ea6433279c678846fb6e06eeccd82e2fe888f2ac203476d3918cd4057901 0038ff9e00be438253be4382530100109bf45901002300210 2123eeaa90e2f5786ce028e60ec03702706dadecee373a90b09b88a99cc668f46ac3 8bf80f10129594a7e949cc43c3bd6f8670ba5ab59874305f6839406738a9cf901 0038ff9e0081bd175381bd1753); CompressLZF lzf = new CompressLZF(); byte[] out = new byte[data.length]; int len = lzf.compress(data, data.length, out, 0); byte[] test = new byte[data.length]; lzf.expand(out, 0, len, test, 0, data.length); System.out.println(StringUtils.convertBytesToHex(test)); System.out.println(Arrays.hashCode(data)); System.out.println(Arrays.hashCode(test)); Regards, Thomas On Tue, May 27, 2014 at 1:57 PM, Noel Grandin noelgran...@gmail.com[1] wrote: I massaged your test case into a unit test for H2, and it seems to be working for us. But maybe there is some more transformation that happens to the raw byte array before it hits the LZF compressor. -- You received this message because you are subscribed to the Google Groups H2 Database group.To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com[2].To post to this group, send email to h2-database@googlegroups.com[3].Visit this group at http://groups.google.com/group/h2-database[4].For more options, visit https://groups.google.com/d/optout[5]. -- You received this message because you are subscribed to the Google Groups H2 Database group.To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com[6].To post to this group, send email to h2-database@googlegroups.com[3].Visit this group at http://groups.google.com/group/h2-database[4].For more options, visit https://groups.google.com/d/optout[5]. [1] mailto:noelgran...@gmail.com [2] mailto:h2-database%2bunsubscr...@googlegroups.com [3] mailto:h2-database@googlegroups.com [4] http://groups.google.com/group/h2-database [5] https://groups.google.com/d/optout [6] mailto:h2-database+unsubscr...@googlegroups.com -- You received this message because you are subscribed to the Google Groups H2 Database group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to this group, send email to h2-database@googlegroups.com. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout.
Re: [h2] Corruption in a database created in 1.3.174 when opening and closing it in 1.3.176
2014-06-01 10:16 GMT+02:00 Cecil Westerhof cldwester...@gmail.com: 2014-05-09 15:30 GMT+02:00 Germano Rizzo germano.ri...@gmail.com: I have a reproducible bug with versions 174 and 176 (at least). A db created with 1.3.174 (no particular settings) corrupts when just opening and closing it in 1.3.176. To reproduce: I have also a problem with 176. With at least one (but not all) database I get the following error when opening the database in H2 console: Unique index or primary key violation: PRIMARY KEY ON .PAGE_INDEX; SQL statement: ALTER TABLE PUBLIC.WEIGHTS ADD CONSTRAINT PUBLIC.CONSTRAINT_74 FOREIGN KEY(CATEGORYID) REFERENCES PUBLIC.CATEGORIES(ID) NOCHECK [23505-176] http://127.0.0.2:8082/login.do?jsessionid=906bcb32b9beddff983be35575c2bd82# 23505/23505 When going back to 174 luckily all my databases work like before. weights is defined as: FIELD http://127.0.0.2:8082/query.do?jsessionid=3f87e6740013f7de2ee326159fb984e7# TYPE http://127.0.0.2:8082/query.do?jsessionid=3f87e6740013f7de2ee326159fb984e7# NULL http://127.0.0.2:8082/query.do?jsessionid=3f87e6740013f7de2ee326159fb984e7# KEY http://127.0.0.2:8082/query.do?jsessionid=3f87e6740013f7de2ee326159fb984e7# DEFAULT http://127.0.0.2:8082/query.do?jsessionid=3f87e6740013f7de2ee326159fb984e7# CATEGORYIDUUID(2147483647) NOPRINULLMEASUREDATEDATE(8)NOPRICURRENT_DATE() WEIGHTDOUBLE(17)NONULLFAT DOUBLE(17)YESNULLWATERDOUBLE(17)YESNULLMUSCLE DOUBLE(17)YESNULL When certain information is needed: let me know. I just gone to 175. That works also. So the bug was introduced in 176. -- Cecil Westerhof -- You received this message because you are subscribed to the Google Groups H2 Database group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to this group, send email to h2-database@googlegroups.com. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout.
Re: [h2] Exception when opening DB after forced shutdown (1.3.176)
Hi, I could analyze it now. The problem seems to be that creating a temporary table is not committed, and then the temporary table is deleted in another session. I have a fix / workaround for that, but so far no test case. Please tell me if you have a reproducible test case. Regards, Thomas On Tuesday, May 27, 2014, too tono...@gmail.com wrote: Even though I can reproduce this with our app I fail to create separate test case to reproduce the problem (i.e. to break the DB). What I can do is send you database that H2 fails to open. Perhaps it is not really bug during close but not robust enough recovery during startup. Zipped DB has 10MB and I can share it for you privately if you think that can help thanks Tono On Thursday, May 22, 2014 5:47:11 PM UTC+2, too wrote: I have experimented with this a little and here are few notes - lock file remains in place - this is consistent with shutdown hooks not being executed during JVM abort - size of DB file remains intact during runtime but during process termination the size rises by approximately 50MB - which contradicts previous point and it seems there is something executed during abort - I can replicate problem with 100% success rate using our app but I fail to create test app to replicate the problem (still have a few ideas to try) - I can replicate the problem with older 1.3.x releases as well as with latest 1.4.x release (with MV_STORE=FALSE) - even though I do not think that there are transactions in progress at the time of termination it is possible there are open cursors or something similar - read-related I can avoid this problem by running H2 in server mode but that is not what I want. Tono On Tuesday, May 20, 2014 3:16:03 PM UTC+2, too wrote: Hi, Sometimes when our application is shut down forcibly it is unable to start again with exceptions below. It does not appear that transaction is in progress at the time of shutdown but I can not be sure. I was unable to create test case but our application reproduces this quite consistently (it's very rare that it starts after being terminated). I have attached debug trace file for failed start - not for the shutdown. I tried INFO logging for shutdown (breaking the db) but nothing seemed interesting there, just bunch of selects. Also no error is logged during shutdown or startup. Environment details are - H2 - 1.3.176 - Java Runtime - Java(TM) SE Runtime Environment (1.7.0_17-b02, 32b) on Java HotSpot(TM) Client VM (23.7-b01, mixed mode) - Operating System - Windows 7 (64b, Service Pack 1, version 6.1) - connection URL - jdbc:h2:./dbfile;IFEXISTS=TRUE NullPointerException - this is what usually happens org.h2.jdbc.JdbcSQLException: General error: java.lang.NullPointerException [5-176] at org.h2.message.DbException.getJdbcSQLException(DbException.java:344) ~[h2.jar:1.3.176] at org.h2.message.DbException.get(DbException.java:167) ~[h2.jar:1.3.176] at org.h2.message.DbException.convert(DbException.java:294) ~[h2.jar:1.3.176] at org.h2.engine.Database.openDatabase(Database.java:291) ~[h2.jar:1.3.176] at org.h2.engine.Database.init(Database.java:254) ~[h2.jar:1.3.176] at org.h2.engine.Engine.openSession(Engine.java:57) ~[h2.jar:1.3.176] at org.h2.engine.Engine.openSession(Engine.java:164) ~[h2.jar:1.3.176] at org.h2.engine.Engine.createSessionAndValidate(Engine.java:142) ~[h2.jar:1.3.176] at org.h2.engine.Engine.createSession(Engine.java:125) ~[h2.jar:1.3.176] at org.h2.engine.Engine.createSession(Engine.java:27) ~[h2.jar:1.3.176] -- You received this message because you are subscribed to the Google Groups H2 Database group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com javascript:_e(%7B%7D,'cvml','h2-database%2bunsubscr...@googlegroups.com'); . To post to this group, send email to h2-database@googlegroups.com javascript:_e(%7B%7D,'cvml','h2-database@googlegroups.com');. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups H2 Database group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to this group, send email to h2-database@googlegroups.com. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout.
[h2] Corruption in a database created in 1.3.174 when opening and closing it in 1.3.176
Hi, Yes, this problem was introduced in version 1.3.176, actually by fixing another bug. It happens when upgrading a database with a special kind of foreign key constraint from an older version to 1.3.176. I found the problem now and have a fix for it. I will need to release a new version; a workaround is to stay with the older version, or upgrade to the newer version by first converting to a SQL script and then creating a new database. Regards, Thomas On Mon, Jun 2, 2014 at 2:04 PM, Cecil Westerhof cldwester...@gmail.com javascript:_e(%7B%7D,'cvml','cldwester...@gmail.com'); wrote: 2014-06-01 10:16 GMT+02:00 Cecil Westerhof cldwester...@gmail.com javascript:_e(%7B%7D,'cvml','cldwester...@gmail.com');: 2014-05-09 15:30 GMT+02:00 Germano Rizzo germano.ri...@gmail.com javascript:_e(%7B%7D,'cvml','germano.ri...@gmail.com');: I have a reproducible bug with versions 174 and 176 (at least). A db created with 1.3.174 (no particular settings) corrupts when just opening and closing it in 1.3.176. To reproduce: I have also a problem with 176. With at least one (but not all) database I get the following error when opening the database in H2 console: Unique index or primary key violation: PRIMARY KEY ON .PAGE_INDEX; SQL statement: ALTER TABLE PUBLIC.WEIGHTS ADD CONSTRAINT PUBLIC.CONSTRAINT_74 FOREIGN KEY(CATEGORYID) REFERENCES PUBLIC.CATEGORIES(ID) NOCHECK [23505-176] http://127.0.0.2:8082/login.do?jsessionid=906bcb32b9beddff983be35575c2bd82# 23505/23505 When going back to 174 luckily all my databases work like before. weights is defined as: FIELD http://127.0.0.2:8082/query.do?jsessionid=3f87e6740013f7de2ee326159fb984e7# TYPE http://127.0.0.2:8082/query.do?jsessionid=3f87e6740013f7de2ee326159fb984e7# NULL http://127.0.0.2:8082/query.do?jsessionid=3f87e6740013f7de2ee326159fb984e7# KEY http://127.0.0.2:8082/query.do?jsessionid=3f87e6740013f7de2ee326159fb984e7# DEFAULT http://127.0.0.2:8082/query.do?jsessionid=3f87e6740013f7de2ee326159fb984e7# CATEGORYIDUUID(2147483647) NOPRINULLMEASUREDATEDATE(8)NOPRICURRENT_DATE() WEIGHTDOUBLE(17)NONULLFAT DOUBLE(17)YESNULLWATERDOUBLE(17)YESNULLMUSCLE DOUBLE(17)YESNULL When certain information is needed: let me know. I just gone to 175. That works also. So the bug was introduced in 176. -- Cecil Westerhof -- You received this message because you are subscribed to the Google Groups H2 Database group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com javascript:_e(%7B%7D,'cvml','h2-database%2bunsubscr...@googlegroups.com'); . To post to this group, send email to h2-database@googlegroups.com javascript:_e(%7B%7D,'cvml','h2-database@googlegroups.com');. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups H2 Database group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to this group, send email to h2-database@googlegroups.com. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout.
Re: [h2] Unique index or primary key violation: SYS_ID ON PUBLIC.SYS(ID) [23505-168] (using 1.3.168)
Hi, I finally found and fixed the problem now. This problem will be fixed in the next release. One workaround is to not upgrade to version 1.3.176 if the database was created with an earlier version. Another workaround is to export the database file to a SQL script and re-create it. Regards, Thomas On Wed, May 28, 2014 at 4:42 PM, Nick99 nail.abda...@gmail.com wrote: Same problem. 1. The database was created in CREATE_BUILD 170, then was used with 1.3.174. I've tried using 1.4.178 on that database in Hibernate-based product - got org.h2.jdbc.JdbcSQLException: Unique index or primary key violation: PRIMARY KEY ON .PAGE_INDEX; SQL statement:[...] 1.3.175 works ok on that db. 1.3.176 is not. 2. Opening the db in 1.3.176 Console produces this (table names removed): Unique index or primary key violation: PRIMARY KEY ON .PAGE_INDEX; SQL statement: ALTER TABLE PUBLIC.-- ADD CONSTRAINT PUBLIC.FK28D9808AF98221CA FOREIGN KEY(JESID) REFERENCES PUBLIC.--(ID) NOCHECK [23505-176] 23505/23505 (Help) On Monday, May 26, 2014 10:33:05 AM UTC+3, Germano Rizzo wrote: Provided here https://groups.google.com/forum/?hl=it#!topic/h2-database/lNp80bgbvBY. Thanks! Il giorno giovedì 22 maggio 2014 17:43:18 UTC+2, Thomas Mueller ha scritto: Hi I would need a reproducible test case, or the database file. Regards, Thomas On Thursday, May 22, 2014, jack jin supe...@gmail.com wrote: when I upgrade to 1.3.176,I have the same problem. the old version h2 is works fine with the db file On Tuesday, May 6, 2014 5:00:52 PM UTC+8, Germano Rizzo wrote: Hi Thomas, so your opinion is that all the db's are corrupted? The strange thing is, they fail to open only with the latest stable version. Every other version I tried works fine, and this configuration have worked in 40 installations for 3 years now, across different versions of H2. Do you want me to start another thread? Germano Il giorno martedì 6 maggio 2014 07:59:45 UTC+2, Thomas Mueller ha scritto: Hi, If you are using LOCK_MODE=0;UNDO_LOG=0 then you need to be aware of the risks. See the FAQ and the documentation. Please use different email subjects for different problems. Regards, Thomas On Monday, May 5, 2014, mano german...@gmail.com wrote: mano wrote Opening it with version 1.3.168 will give the error. Sorry, I meant 1.3.176. -- View this message in context: http://h2-database.66688.n3.na bble.com/Unique-index-or-primary-key-violation-SYS-ID-ON- PUBLIC-SYS-ID-23505-168-using-1-3-168-tp4028700p4029078.html Sent from the H2 Database mailing list archive at Nabble.com. -- You received this message because you are subscribed to the Google Groups H2 Database group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to this group, send email to h2-database@googlegroups.com. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups H2 Database group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to this group, send email to h2-database@googlegroups.com. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups H2 Database group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to this group, send email to h2-database@googlegroups.com. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups H2 Database group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to this group, send email to h2-database@googlegroups.com. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout.
Re: [h2] Corruption in a database created in 1.3.174 when opening and closing it in 1.3.176
Great work, Thomas, thank you. Will I be able to open a database created with 174, and never opened in 176, with the new version? Thanks again, Germano Il giorno lunedì 2 giugno 2014 18:04:13 UTC+2, Thomas Mueller ha scritto: Hi, Yes, this problem was introduced in version 1.3.176, actually by fixing another bug. It happens when upgrading a database with a special kind of foreign key constraint from an older version to 1.3.176. I found the problem now and have a fix for it. I will need to release a new version; a workaround is to stay with the older version, or upgrade to the newer version by first converting to a SQL script and then creating a new database. Regards, Thomas On Mon, Jun 2, 2014 at 2:04 PM, Cecil Westerhof cldwester...@gmail.com wrote: 2014-06-01 10:16 GMT+02:00 Cecil Westerhof cldwester...@gmail.com: 2014-05-09 15:30 GMT+02:00 Germano Rizzo germano.ri...@gmail.com: I have a reproducible bug with versions 174 and 176 (at least). A db created with 1.3.174 (no particular settings) corrupts when just opening and closing it in 1.3.176. To reproduce: I have also a problem with 176. With at least one (but not all) database I get the following error when opening the database in H2 console: Unique index or primary key violation: PRIMARY KEY ON .PAGE_INDEX; SQL statement: ALTER TABLE PUBLIC.WEIGHTS ADD CONSTRAINT PUBLIC.CONSTRAINT_74 FOREIGN KEY(CATEGORYID) REFERENCES PUBLIC.CATEGORIES(ID) NOCHECK [23505-176] http://127.0.0.2:8082/login.do?jsessionid=906bcb32b9beddff983be35575c2bd82# 23505/23505 When going back to 174 luckily all my databases work like before. weights is defined as: FIELD http://127.0.0.2:8082/query.do?jsessionid=3f87e6740013f7de2ee326159fb984e7# TYPE http://127.0.0.2:8082/query.do?jsessionid=3f87e6740013f7de2ee326159fb984e7# NULL http://127.0.0.2:8082/query.do?jsessionid=3f87e6740013f7de2ee326159fb984e7# KEY http://127.0.0.2:8082/query.do?jsessionid=3f87e6740013f7de2ee326159fb984e7# DEFAULT http://127.0.0.2:8082/query.do?jsessionid=3f87e6740013f7de2ee326159fb984e7# CATEGORYIDUUID(2147483647) NOPRINULLMEASUREDATEDATE(8)NOPRI CURRENT_DATE()WEIGHTDOUBLE(17)NONULLFAT DOUBLE(17)YESNULLWATERDOUBLE(17) YESNULLMUSCLEDOUBLE(17)YESNULL When certain information is needed: let me know. I just gone to 175. That works also. So the bug was introduced in 176. -- Cecil Westerhof -- You received this message because you are subscribed to the Google Groups H2 Database group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to this group, send email to h2-database@googlegroups.com. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups H2 Database group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to this group, send email to h2-database@googlegroups.com. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout.
[h2] *huge* database file expansion with recovery tool
Running a recovery, the resulting db file is 5x larger, and still growing. I just had a bulk load deadlock. I'm not sure how to interrupt these things, but started with kill -INT, then kill -SEGV, then kill -KILL. After that the db could not be opened, and would dump a bunch of lines like this into the trace: org.h2.jdbc.JdbcSQLException: Table PROBEMAP_PROBES not found; SQL statement: CREATE PRIMARY KEY PUBLIC.PRIMARY_KEY_E9 ON PUBLIC.PROBEMAP_PROBES(ID) [42102-175] 06-02 16:17:52 database: CREATE PRIMARY KEY PUBLIC.PRIMARY_KEY_E9 ON PUBLIC.PROBEMAP_PROBES(ID) I ran the recovery tool with java -cp ~/.m2/repository/com/h2database/h2/1.3.175/h2-1.3.175.jar org.h2.tools.Recover which generated a sql file that was 1.25x the size of the db file. I then loaded the script into a fresh db file with java -cp ~/.m2/repository/com/h2database/h2/1.3.175/h2-1.3.175.jar org.h2.tools.RunScript -url jdbc:h2:recovered -user '' -script cavm-testing.h2.sql which at the moment is more than 5x larger than the original db file. It's fairly remarkable. I may have to kill it. Any idea what would cause this, or if there's a way to prevent it? During the load which deadlocked the url was cavm-testing;CACHE_SIZE=65536;UNDO_LOG=1;LOG=0;MVCC=TRUE -- You received this message because you are subscribed to the Google Groups H2 Database group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to this group, send email to h2-database@googlegroups.com. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout.
Re: [h2] Corruption in a database created in 1.3.174 when opening and closing it in 1.3.176
Hi, Will I be able to open a database created with 174, and never opened in 176, with the new version? Yes. Even if opened with version 1.3.176 multiple times actually. No data is lost, the change just prevents the (old) database from being opened. Regards, Thomas -- You received this message because you are subscribed to the Google Groups H2 Database group. To unsubscribe from this group and stop receiving emails from it, send an email to h2-database+unsubscr...@googlegroups.com. To post to this group, send email to h2-database@googlegroups.com. Visit this group at http://groups.google.com/group/h2-database. For more options, visit https://groups.google.com/d/optout.
