On Dec 14, 5:54 am, "Thomas Mueller" <[email protected]> wrote: > Hi, > > > It it's not really a fair test to use cached tables either which have known > > performance problems in hsqldb > > 4. hsqldb is configured to use cached tables which have known performance > > problems. > > I believe it's quite common that tables don't fit in memory. If HSQLDB > has a problem with that then it should be reflected in the benchmark > result. > > Both H2 and HSQLDB support in-memory operation, the others do not. I > believe this scenario is not all that common. From what I know, most > people use persistent databases / tables.
You missed the point entirely. HSQLDB is primarily an in memory database. You're using a feature that has known performance problem to make it appear that H2 is faster than HSQLDB. In reality HSQLDB beats H2 in almost every single type of operation by a large margin. If you have a large database then HSQLDB isn't the right database to use. You are specifically configuring HSQLDB to reduce it's performance on the benchmark. > > The h2 benchmark isn't fair either: > > 1. derby uses durable operations and none of the others do. > > Derby doesn't use completely durable operations. Maybe it uses 'a bit > more durable operations', but I fail to see any value in that. > Seehttp://h2database.com/html/advanced.html#durability_problemsDerby > doesn't use FileDescriptor.sync() by default, it uses > RandomAccessFile(.., "rwd" or "rws"). Even if it would use > FileDescriptor.sync() (which it doesn't), that would still not always > guarantee durability. I believe we already discussed that? We did discuss this before and you were shown to be wrong before. What derby does is if you are using Java 1.4 or later it uses code in this file: java\engine\org\apache\derby\impl\io\DirRandomAccessFile4 To call the method force() on Channel. http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/FileChannel.html#force(boolean) It only uses "rws" if you're using a version of java prior to 1.4. >Maybe it uses 'a bit more durable operations', but I fail to see any value in >that. Not losing data isn't valuable?! Are you joking? > Derby supports a testing mode (system property > derby.system.durability=test), I tried that. Even if enabled, Derby is > still less than half as fast as H2 in default mode. See > alsohttp://h2database.com/html/performance.html You missed the point entirely again. Changing that one setting makes it closer to being a fair test. The one setting doubles derby's performance on the h2 benchmark. > > 2. derby has a 4m cache by default, h2 has a 16m cache by default. > > The Derby setting and documentation says '4m' but it doesn't mean that > Derby uses 4 MB of heap memory. > Seehttp://h2database.com/html/performance.html#performance_comparison- > the memory usage is listed in the benchmark result. Derby uses 11 MB > for the simple test case, 10 MB for BenchA, 8 MB for BenchB, 9 MB for > BenchC. H2 uses 6 / 9 / 9 / 9 MB. Again you missed the point entirely. You are relying on a significant advantage in the cache size to make h2 look faster than derby. I did a check for System.gc() calls in the h2 benchmark. You don't use any so not all the objects collected. All the numbers mean is derby creates more garbage than h2 for the benchmark. It doesn't mean the cache takes up more ram. > > 3. derby does index logging, h2 doesn't log index changes by default (i can > > be turned on for h2). > > That's true. Unfortunately it can't be turned off for Derby. H2 does > automatically rebuild the indexes if required after a crash. For > medium sized databases (up to 1 GB) this is not a problem as > re-building the indexes doesn't take that long, and crashes are not > that common. You can argue the recovery time for larger databases is > important, but then the benchmarks are made for medium sized > databases. So far there are no benchmarks for large databases. Again, you missed the point entirely. With index logging turned on derby beats h2 on the h2 benchmark. Again you are relying on unfair configuration to make it appear that h2 has a performance advantage. > > Almost all the time for the hsqldb results is in one test (BenchC: > > Transactions). > > Yes, this is described. "HSQLDB is very slow in the last test (BenchC: > Transactions), probably because is has a bad query optimizer.". > Seehttp://h2database.com/html/performance.html#performance_comparison The soon to be released version of HSQLDB doesn't have that problem. HSQLDB 1.9 ties H2 on the benchmark even when you specifically configure it to cripple the performance of HSQLDB. If the configuration isn't crippled HSQLDB 1.9 is twice as faster as H2 on the h2 benchmark. > > With that one change HSQLDB still beats H2 convincingly. > > Sorry I fail to see how you came to this conclusion. As I wrote before > Overall time taken HSQLDB: 155'263ms, H2: 71'275ms You forgot to quote the portion where you said you had to use special configuration for the poleposition test. In the test that I ran without your configuration HSQLDB beats h2. > > The 1.8.x versions of HSQLDB have performance problems with cached You argue in some cases that to make the benchmark fair you have to adjust some settings on other databases. Every setting you configure reduces the performance of the other databases. Every time I mention how to configure the other databases so that it's closer to h2's configuration you ignore it. You're being dishonest when you say you want the benchmark to be fair but refuse to change some very simple settings that would make it a fair test. Brish --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "H2 Database" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/h2-database?hl=en -~----------~----~----~----~------~----~------~--~---
