I have added FileChannel.force(false) flushes after all write operations in the cache store, and now the comparison is also updated with these values.
Radim ----- Original Message ----- | From: "Radim Vansa" <rva...@redhat.com> | To: "infinispan -Dev List" <infinispan-dev@lists.jboss.org> | Sent: Thursday, June 27, 2013 8:54:25 AM | Subject: Re: [infinispan-dev] Cachestores performance | | Yep, write-through. LevelDB JAVA used FileChannelTable implementation | (-Dleveldb.mmap), because Mmaping is not implemented very well and causes | JVM crashes (I believe it's because of calling non-public API via reflection | - I've found post from the Oracle JVM guys discouraging the particular trick | it uses). After writing the record to the log, it calls | FileChannel.force(true), therefore, it should be really on the disc by that | moment. | I have not looked into the JNI implementation but I expect the same. | | By the way, I have updated [1] with numbers when running on more data (2 GB | instead of 100 MB). I won't retype it here, so look there. The performance | is much lower. | I may try also increase JVM heap size and try with a bit more data yet. | | Radim | | [1] https://community.jboss.org/wiki/FileCacheStoreRedesign | | ----- Original Message ----- | | From: "Erik Salter" <an1...@hotmail.com> | | To: "infinispan -Dev List" <infinispan-dev@lists.jboss.org> | | Sent: Wednesday, June 26, 2013 7:40:19 PM | | Subject: Re: [infinispan-dev] Cachestores performance | | | | These were write-through cache stores, right? And with LevelDB, this was | | through to the database file itself? | | | | Erik | | | | -----Original Message----- | | From: infinispan-dev-boun...@lists.jboss.org | | [mailto:infinispan-dev-boun...@lists.jboss.org] On Behalf Of Radim Vansa | | Sent: Wednesday, June 26, 2013 11:24 AM | | To: infinispan -Dev List | | Subject: [infinispan-dev] Cachestores performance | | | | Hi all, | | | | according to [1] I've created the comparison of performance in | | stress-tests. | | | | All setups used local-cache, benchmark was executed via Radargun (actually | | version not merged into master yet [2]). I've used 4 nodes just to get more | | data - each slave was absolutely independent of the others. | | | | First test was preloading performance - the cache started and tried to load | | 1GB of data from harddrive. Without cachestore the startup takes about 2 - | | 4 | | seconds, average numbers for the cachestores are below: | | | | FileCacheStore: 9.8 s | | KarstenFileCacheStore: 14 s | | LevelDB-JAVA impl.: 12.3 s | | LevelDB-JNI impl.: 12.9 s | | | | IMO nothing special, all times seem affordable. We don't benchmark exactly | | storing the data into the cachestore, here FileCacheStore took about 44 | | minutes, while Karsten about 38 seconds, LevelDB-JAVA 4 minutes and | | LevelDB-JNI 96 seconds. The units are right, it's minutes compared to | | seconds. But we all know that FileCacheStore is bloody slow. | | | | Second test is stress test (5 minutes, preceded by 2 minute warmup) where | | each of 10 threads works on 10k entries with 1kB values (~100 MB in total). | | 20 % writes, 80 % reads, as usual. No eviction is configured, therefore the | | cache-store works as a persistent storage only for case of crash. | | | | FileCacheStore: 3.1M reads/s 112 writes/s // on one node the | | performance was only 2.96M reads/s 75 writes/s | | KarstenFileCacheStore: 9.2M reads/s 226k writes/s // yikes! | | LevelDB-JAVA impl.: 3.9M reads/s 5100 writes/s | | LevelDB-JNI impl.: 6.6M reads/s 14k writes/s // on one node the | | performance was 3.9M/8.3k - about half of the others | | Without cache store: 15.5M reads/s 4.4M writes/s | | | | Karsten implementation pretty rules here for two reasons. First of all, it | | does not flush the data (it calls only RandomAccessFile.write()). Other | | cheat is that it stores in-memory the keys and offsets of data values in | | the | | database file. Therefore, it's definitely the best choice for this | | scenario, | | but it does not allow to scale the cache-store, especially in cases where | | the keys are big and values small. However, this performance boost is | | definitely worth checking - I could think of caching the disk offsets in | | memory and querying persistent index only in case of missing record, with | | part of the persistent index flushed asynchronously (the index can be | | always | | rebuilt during the preloading for case of crash). | | | | The third test should have tested the scenario with more data to be stored | | than memory - therefore, the stressors operated on 100k entries (~100 MB of | | data) but eviction was set to 10k entries (9216 entries ended up in memory | | after the test has ended). | | | | FileCacheStore: 750 reads/s 285 writes/s // one node | | had | | only 524 reads and 213 writes per second | | KarstenFileCacheStore: 458k reads/s 137k writes/s | | LevelDB-JAVA impl.: 21k reads/s 9k writes/s // a bit | | varying | | performance | | LevelDB-JNI impl.: 13k-46k reads/s 6.6k-15.2k writes/s // the | | performance varied a lot! | | | | 100 MB of data is not much, but it takes so long to push it into | | FileCacheStore that I won't use more unless we exclude this loser from the | | comparison :) | | | | Radim | | | | [1] https://community.jboss.org/wiki/FileCacheStoreRedesign | | [2] https://github.com/rvansa/radargun/tree/t_keygen | | | | ----------------------------------------------------------- | | Radim Vansa | | Quality Assurance Engineer | | JBoss Datagrid | | tel. +420532294559 ext. 62559 | | | | Red Hat Czech, s.r.o. | | Brno, Purkyňova 99/71, PSČ 612 45 | | Czech Republic | | | | | | _______________________________________________ | | infinispan-dev mailing list | | infinispan-dev@lists.jboss.org | | https://lists.jboss.org/mailman/listinfo/infinispan-dev | | | | | | _______________________________________________ | | infinispan-dev mailing list | | infinispan-dev@lists.jboss.org | | https://lists.jboss.org/mailman/listinfo/infinispan-dev | | _______________________________________________ | infinispan-dev mailing list | infinispan-dev@lists.jboss.org | https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev@lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev
