On Wed, Nov 27, 2013 at 7:39 AM, Sage Weil <[email protected]> wrote: > On Wed, 27 Nov 2013, Sebastien Han wrote: > > Hi Sage, > > If I recall correctly during the summit you mentioned that it was > possible to disable the journal. > > Is it still part of the plan? > > For the kv backend, yeah, since the key/value store will handle making > things transactional. > > Haomai, I still haven't had a chance to look this over yet; it's on my > list! Did you look at the summit notes or watch the session? >
Of course, and I'm willing to implement the next progress. > > sage > > > > > > ???? > > S?bastien Han > > Cloud Engineer > > > > "Always give 100%. Unless you're giving blood.? > > > > Phone: +33 (0)1 49 70 99 72 > > Mail: [email protected] > > Address : 10, rue de la Victoire - 75009 Paris > > Web : www.enovance.com - Twitter : @enovance > > > > On 25 Nov 2013, at 10:00, Sebastien Han <[email protected]> > wrote: > > > > > Nice job Haomai! > > > > > > ???? > > > S?bastien Han > > > Cloud Engineer > > > > > > "Always give 100%. Unless you're giving blood.? > > > > > > Phone: +33 (0)1 49 70 99 72 > > > Mail: [email protected] > > > Address : 10, rue de la Victoire - 75009 Paris > > > Web : www.enovance.com - Twitter : @enovance > > > > > > On 25 Nov 2013, at 02:50, Haomai Wang <[email protected]> wrote: > > > > > >> > > >> > > >> > > >> On Mon, Nov 25, 2013 at 2:17 AM, Mark Nelson <[email protected]> > wrote: > > >> Great Work! This is very exciting! Did you happen to try RADOS bench > at different object sizes and concurrency levels? > > >> > > >> > > >> Maybe can try it later. :-) > > >> > > >> Mark > > >> > > >> > > >> On 11/24/2013 03:01 AM, Haomai Wang wrote: > > >> Hi all, > > >> > > >> For Emperor > > >> blueprint( > http://wiki.ceph.com/01Planning/02Blueprints/Emperor/Add_LevelDB_support_to_ceph_cluster_backend_store > ), > > >> I'm sorry to delay the progress. Now, I have done the most of the > works > > >> for the blueprint's goal. Because of sage's F > > >> blueprint( > http://wiki.ceph.com/index.php?title=01Planning/02Blueprints/Firefly/osd:_new_key%2F%2Fvalue_backend > ), > > >> I need to adjust some codes to match it. The branch is > > >> here(https://github.com/yuyuyu101/ceph/tree/wip/6173). > > >> > > >> I have tested the LevelDB backend on three nodes(eight OSDs) and > compare > > >> it to FileStore(ext4). I just use intern benchmark tool "rados bench" > to > > >> get the comparison. The default ceph configurations is used and > > >> replication size is 2. The filesystem is ext4 and no others changed. > The > > >> results is below: > > >> > > >> *Rados Bench* > > >> > > >> > > >> > > >> *Bandwidth(MB/sec)* > > >> > > >> > > >> > > >> *Average Latency* > > >> > > >> > > >> > > >> *Max Latency* > > >> > > >> > > >> > > >> *Min Latency* > > >> > > >> > > >> > > >> *Stddev Latency* > > >> > > >> > > >> > > >> *Stddev Bandwidth(MB/sec)* > > >> > > >> > > >> > > >> *Max Bandwidth(MB/sec)* > > >> > > >> > > >> > > >> *Min Bandwidth(MB/sec)* > > >> > > >> > > >> > > >> > > >> *KVStore* > > >> > > >> > > >> > > >> *FileStore* > > >> > > >> > > >> > > >> *KVStore* > > >> > > >> > > >> > > >> *FileStore* > > >> > > >> > > >> > > >> *KVStore* > > >> > > >> > > >> > > >> *FileStore* > > >> > > >> > > >> > > >> *KVStore* > > >> > > >> > > >> > > >> *FileStore* > > >> > > >> > > >> > > >> *KVStore* > > >> > > >> > > >> > > >> *FileStore* > > >> > > >> > > >> > > >> *KVStore* > > >> > > >> > > >> > > >> *FileStore* > > >> > > >> > > >> > > >> *KVStore* > > >> > > >> > > >> > > >> *FileStore* > > >> > > >> > > >> > > >> *KVStore* > > >> > > >> > > >> > > >> *FileStore* > > >> > > >> *Write 30* > > >> > > >> > > >> > > >> > > >> 24.590 > > >> > > >> > > >> > > >> 23.495 > > >> > > >> > > >> > > >> 4.87257 > > >> > > >> > > >> > > >> 5.07716 > > >> > > >> > > >> > > >> 14.752 > > >> > > >> > > >> > > >> 13.0885 > > >> > > >> > > >> > > >> 0.580851 > > >> > > >> > > >> > > >> 0.605118 > > >> > > >> > > >> > > >> 2.97708 > > >> > > >> > > >> > > >> 3.30538 > > >> > > >> > > >> > > >> 9.91938 > > >> > > >> > > >> > > >> 10.5986 > > >> > > >> > > >> > > >> 44 > > >> > > >> > > >> > > >> 76 > > >> > > >> > > >> > > >> 0 > > >> > > >> > > >> > > >> 0 > > >> > > >> *Write 20* > > >> > > >> > > >> > > >> > > >> 23.515 > > >> > > >> > > >> > > >> 23.064 > > >> > > >> > > >> > > >> 3.39745 > > >> > > >> > > >> > > >> 3.45711 > > >> > > >> > > >> > > >> 11.6089 > > >> > > >> > > >> > > >> 11.5996 > > >> > > >> > > >> > > >> 0.169507 > > >> > > >> > > >> > > >> 0.138595 > > >> > > >> > > >> > > >> 2.58285 > > >> > > >> > > >> > > >> 2.75962 > > >> > > >> > > >> > > >> 9.14467 > > >> > > >> > > >> > > >> 8.54156 > > >> > > >> > > >> > > >> 44 > > >> > > >> > > >> > > >> 40 > > >> > > >> > > >> > > >> 0 > > >> > > >> > > >> > > >> 0 > > >> > > >> *Write 10* > > >> > > >> > > >> > > >> > > >> 22.927 > > >> > > >> > > >> > > >> 21.980 > > >> > > >> > > >> > > >> 1.73815 > > >> > > >> > > >> > > >> 1.8198 > > >> > > >> > > >> > > >> 5.53792 > > >> > > >> > > >> > > >> 6.46675 > > >> > > >> > > >> > > >> 0.171028 > > >> > > >> > > >> > > >> 0.143392 > > >> > > >> > > >> > > >> 1.05982 > > >> > > >> > > >> > > >> 1.20303 > > >> > > >> > > >> > > >> 9.18403 > > >> > > >> > > >> > > >> 8.74401 > > >> > > >> > > >> > > >> 44 > > >> > > >> > > >> > > >> 40 > > >> > > >> > > >> > > >> 0 > > >> > > >> > > >> > > >> 0 > > >> > > >> *Write 5* > > >> > > >> > > >> > > >> > > >> 19.680 > > >> > > >> > > >> > > >> 20.017 > > >> > > >> > > >> > > >> 1.01492 > > >> > > >> > > >> > > >> 0.997019 > > >> > > >> > > >> > > >> 3.10783 > > >> > > >> > > >> > > >> 3.05008 > > >> > > >> > > >> > > >> 0.143758 > > >> > > >> > > >> > > >> 0.138161 > > >> > > >> > > >> > > >> 0.561548 > > >> > > >> > > >> > > >> 0.571459 > > >> > > >> > > >> > > >> 5.92575 > > >> > > >> > > >> > > >> 6.844 > > >> > > >> > > >> > > >> 36 > > >> > > >> > > >> > > >> 32 > > >> > > >> > > >> > > >> 0 > > >> > > >> > > >> > > >> 0 > > >> > > >> *Read 30* > > >> > > >> > > >> > > >> > > >> 65.852 > > >> > > >> > > >> > > >> 60.688 > > >> > > >> > > >> > > >> 1.80069 > > >> > > >> > > >> > > >> 1.96009 > > >> > > >> > > >> > > >> 9.30039 > > >> > > >> > > >> > > >> 10.1146 > > >> > > >> > > >> > > >> 0.115153 > > >> > > >> > > >> > > >> 0.061657 > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> *Read 20* > > >> > > >> > > >> > > >> > > >> 59.372 > > >> > > >> > > >> > > >> 60.738 > > >> > > >> > > >> > > >> 1.30479 > > >> > > >> > > >> > > >> 1.28383 > > >> > > >> > > >> > > >> 6.28435 > > >> > > >> > > >> > > >> 8.21304 > > >> > > >> > > >> > > >> 0.016843 > > >> > > >> > > >> > > >> 0.012073 > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> *Read 10* > > >> > > >> > > >> > > >> > > >> 65.502 > > >> > > >> > > >> > > >> 55.814 > > >> > > >> > > >> > > >> 0.608805 > > >> > > >> > > >> > > >> 0.7087 > > >> > > >> > > >> > > >> 3.3917 > > >> > > >> > > >> > > >> 4.72626 > > >> > > >> > > >> > > >> 0.016267 > > >> > > >> > > >> > > >> 0.011998 > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> *Read 5* > > >> > > >> > > >> > > >> > > >> 64.176 > > >> > > >> > > >> > > >> 54.928 > > >> > > >> > > >> > > >> 0.307111 > > >> > > >> > > >> > > >> 0.364077 > > >> > > >> > > >> > > >> 1.76391 > > >> > > >> > > >> > > >> 1.90182 > > >> > > >> > > >> > > >> 0.017174 > > >> > > >> > > >> > > >> 0.011999 > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> Charts can be view here(http://img42.com/ziwjP+) and > > >> (http://img42.com/LKhoo+) > > >> > > >> > > >> From above, I'm feeling relieved that the LevelDB backend isn't > > >> useless. Most of metrics are better and if increasing cache size for > > >> LevelDB the results may be more attractive. > > >> Even more, LevelDB backend is used by "KeyValueStore" and much of > > >> optimizations can be done to improve performance such as increase > > >> parallel threads or optimize io path. > > >> > > >> Next, I use "rbd bench-write" to test. The result is pity: > > >> > > >> *RBD Bench-Write* > > >> > > >> > > >> > > >> *OPS/sec* > > >> > > >> > > >> > > >> *Bytes/sec* > > >> > > >> *KVStore* > > >> > > >> > > >> > > >> *FileStore* > > >> > > >> > > >> > > >> *KVStore* > > >> > > >> > > >> > > >> *FileStore* > > >> > > >> *Seq 4096 5* > > >> > > >> > > >> > > >> 27.42 > > >> > > >> > > >> > > >> 716.55 > > >> > > >> > > >> > > >> 111861.51 > > >> > > >> > > >> > > >> 2492149.21 > > >> > > >> *Rand 4096 5* > > >> > > >> > > >> > > >> > > >> 28.27 > > >> > > >> > > >> > > >> 504 > > >> > > >> > > >> > > >> 112331.42 > > >> > > >> > > >> > > >> 1683151.29 > > >> > > >> > > >> Just because kv backend doesn't support read/write operation with > > >> offset/length argument, each read/write operation will call a > additional > > >> read LevelDB api to do. Much of time is consumed by reading entire > large > > >> object in rbd situation. There exists some ways to change such as > split > > >> large object to multi small objects or save metadata to avoid read > > >> arduous operation. > > >> > > >> As sage mentioned in <osd: new key/value > > >> backend>( > http://wiki.ceph.com/index.php?title=01Planning/02Blueprints/Firefly/osd:_new_key%2F%2Fvalue_backend > ), > > >> more kv backends can be added now and I look forward to more people > > >> interested it. I think radosgw situation can fit in kv store in short > ti > > >> > > >> -- > > >> > > >> Best Regards, > > >> > > >> Wheat > > >> > > >> > > >> > > >> _______________________________________________ > > >> ceph-users mailing list > > >> [email protected] > > >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > >> > > >> > > >> _______________________________________________ > > >> ceph-users mailing list > > >> [email protected] > > >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > >> > > >> > > >> > > >> -- > > >> Best Regards, > > >> > > >> Wheat > > >> > > >> _______________________________________________ > > >> ceph-users mailing list > > >> [email protected] > > >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > _______________________________________________ > > > ceph-users mailing list > > > [email protected] > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > _______________________________________________ > ceph-users mailing list > [email protected] > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Best Regards, Wheat
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
