Re: [PATCH] rbd: handle parent_overlap on writes correctly
On Fri, Jun 13, 2014 at 5:26 AM, Josh Durgin josh.dur...@inktank.com wrote: On 06/11/2014 09:40 AM, Ilya Dryomov wrote: The following check in rbd_img_obj_request_submit() rbd_dev-parent_overlap = obj_request-img_offset allows the fall through to the non-layered write case even if both parent_overlap and obj_request-img_offset belong to the same RADOS object. This leads to data corruption, because the area to the left of parent_overlap ends up unconditionally zero-filled instead of being populated with parent data. Suppose we want to write 1M to offset 6M of image bar, which is a clone of foo@snap; object_size is 4M, parent_overlap is 5M: rbd_data.id.0001 -|--| | should be copyup'ed | should be zeroed out | write ... -|--| 4M5M 6M parent_overlapobj_request-img_offset 4..5M should be copyup'ed from foo, yet it is zero-filled, just like 5..6M is. Given that the only striping mode kernel client currently supports is chunking (i.e. stripe_unit == object_size, stripe_count == 1), round parent_overlap up to the next object boundary for the purposes of the overlap check. Signed-off-by: Ilya Dryomov ilya.dryo...@inktank.com --- Good catch! This should be included in any stable kernels 3.10 or later too. Reviewed-by: Josh Durgin josh.dur...@inktank.com This commit in ceph-client already has the 3.10+ stable tag. Thanks, Ilya -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[no subject]
Although you might be nervous about my e-mail as we have not met before. My name is Mrs Teresa Au, I work with HSBC Hong Kong; there is a sum of USD$23,200,000.00 business proposal I want to share with you. It is absolutely risk free; if you are interested send me a reply to my private e-mail below : mrs_t...@126.com Best Regards, Email: mrs_t...@126.com Mrs Teresa Au. Provincia di Treviso - http://www.provincia.treviso.it -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [RFC] add rocksdb support
Hi Xinxin, I tried to compile the wip-rocksdb branch, but the src/rocksdb directory seems to be empty. Do I need toput autoconf/automake in this directory? It doesn't seem to have any other source files and compilation fails: os/RocksDBStore.cc:10:24: fatal error: rocksdb/db.h: No such file or directory compilation terminated. Thanks, Sushma -Original Message- From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel-ow...@vger.kernel.org] On Behalf Of Shu, Xinxin Sent: Monday, June 09, 2014 10:00 PM To: Mark Nelson; Sage Weil Cc: ceph-devel@vger.kernel.org; Zhang, Jian Subject: RE: [RFC] add rocksdb support Hi mark I have finished development of support of rocksdb submodule, a pull request for support of autoconf/automake for rocksdb has been created , you can find https://github.com/ceph/rocksdb/pull/2 , if this patch is ok , I will create a pull request for rocksdb submodule support , currently this patch can be found https://github.com/xinxinsh/ceph/tree/wip-rocksdb . -Original Message- From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel-ow...@vger.kernel.org] On Behalf Of Mark Nelson Sent: Tuesday, June 10, 2014 1:12 AM To: Shu, Xinxin; Sage Weil Cc: ceph-devel@vger.kernel.org; Zhang, Jian Subject: Re: [RFC] add rocksdb support Hi Xinxin, On 05/28/2014 05:05 AM, Shu, Xinxin wrote: Hi sage , I will add two configure options to --with-librocksdb-static and --with-librocksdb , with --with-librocksdb-static option , ceph will compile the code that get from ceph repository , with --with-librocksdb option , in case of distro packages for rocksdb , ceph will not compile the rocksdb code , will use pre-installed library. is that ok for you ? since current rocksdb does not support autoconfautomake , I will add autoconfautomake support for rocksdb , but before that , i think we should fork a stable branch (maybe 3.0) for ceph . I'm looking at testing out the rocksdb support as well, both for the OSD and for the monitor based on some issues we've been seeing lately. Any news on the 3.0 fork and autoconf/automake support in rocksdb? Thanks, Mark -Original Message- From: Mark Nelson [mailto:mark.nel...@inktank.com] Sent: Wednesday, May 21, 2014 9:06 PM To: Shu, Xinxin; Sage Weil Cc: ceph-devel@vger.kernel.org; Zhang, Jian Subject: Re: [RFC] add rocksdb support On 05/21/2014 07:54 AM, Shu, Xinxin wrote: Hi, sage I will add rocksdb submodule into the makefile , currently we want to have fully performance tests on key-value db backend , both leveldb and rocksdb. Then optimize on rocksdb performance. I'm definitely interested in any performance tests you do here. Last winter I started doing some fairly high level tests on raw leveldb/hyperleveldb/raikleveldb. I'm very interested in what you see with rocksdb as a backend. -Original Message- From: Sage Weil [mailto:s...@inktank.com] Sent: Wednesday, May 21, 2014 9:19 AM To: Shu, Xinxin Cc: ceph-devel@vger.kernel.org Subject: Re: [RFC] add rocksdb support Hi Xinxin, I've pushed an updated wip-rocksdb to github/liewegas/ceph.git that includes the latest set of patches with the groundwork and your rocksdb patch. There is also a commit that adds rocksdb as a git submodule. I'm thinking that, since there aren't any distro packages for rocksdb at this point, this is going to be the easiest way to make this usable for people. If you can wire the submodule into the makefile, we can merge this in so that rocksdb support is in the ceph.com packages on ceph.com. I suspect that the distros will prefer to turns this off in favor of separate shared libs, but they can do this at their option if/when they include rocksdb in the distro. I think the key is just to have both --with-librockdb and --with-librocksdb-static (or similar) options so that you can either use the static or dynamically linked one. Has your group done further testing with rocksdb? Anything interesting to share? Thanks! sage -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying
Re: [RFC] add rocksdb support
Don’t forget when a new submodule is added you need to initialize it. From the README: Building Ceph = To prepare the source tree after it has been git cloned, $ git submodule update --init To build the server daemons, and FUSE client, execute the following: $ ./autogen.sh $ ./configure $ make David Zafman Senior Developer http://www.inktank.com http://www.redhat.com On Jun 13, 2014, at 11:51 AM, Sushma Gurram sushma.gur...@sandisk.com wrote: Hi Xinxin, I tried to compile the wip-rocksdb branch, but the src/rocksdb directory seems to be empty. Do I need toput autoconf/automake in this directory? It doesn't seem to have any other source files and compilation fails: os/RocksDBStore.cc:10:24: fatal error: rocksdb/db.h: No such file or directory compilation terminated. Thanks, Sushma -Original Message- From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel-ow...@vger.kernel.org] On Behalf Of Shu, Xinxin Sent: Monday, June 09, 2014 10:00 PM To: Mark Nelson; Sage Weil Cc: ceph-devel@vger.kernel.org; Zhang, Jian Subject: RE: [RFC] add rocksdb support Hi mark I have finished development of support of rocksdb submodule, a pull request for support of autoconf/automake for rocksdb has been created , you can find https://github.com/ceph/rocksdb/pull/2 , if this patch is ok , I will create a pull request for rocksdb submodule support , currently this patch can be found https://github.com/xinxinsh/ceph/tree/wip-rocksdb . -Original Message- From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel-ow...@vger.kernel.org] On Behalf Of Mark Nelson Sent: Tuesday, June 10, 2014 1:12 AM To: Shu, Xinxin; Sage Weil Cc: ceph-devel@vger.kernel.org; Zhang, Jian Subject: Re: [RFC] add rocksdb support Hi Xinxin, On 05/28/2014 05:05 AM, Shu, Xinxin wrote: Hi sage , I will add two configure options to --with-librocksdb-static and --with-librocksdb , with --with-librocksdb-static option , ceph will compile the code that get from ceph repository , with --with-librocksdb option , in case of distro packages for rocksdb , ceph will not compile the rocksdb code , will use pre-installed library. is that ok for you ? since current rocksdb does not support autoconfautomake , I will add autoconfautomake support for rocksdb , but before that , i think we should fork a stable branch (maybe 3.0) for ceph . I'm looking at testing out the rocksdb support as well, both for the OSD and for the monitor based on some issues we've been seeing lately. Any news on the 3.0 fork and autoconf/automake support in rocksdb? Thanks, Mark -Original Message- From: Mark Nelson [mailto:mark.nel...@inktank.com] Sent: Wednesday, May 21, 2014 9:06 PM To: Shu, Xinxin; Sage Weil Cc: ceph-devel@vger.kernel.org; Zhang, Jian Subject: Re: [RFC] add rocksdb support On 05/21/2014 07:54 AM, Shu, Xinxin wrote: Hi, sage I will add rocksdb submodule into the makefile , currently we want to have fully performance tests on key-value db backend , both leveldb and rocksdb. Then optimize on rocksdb performance. I'm definitely interested in any performance tests you do here. Last winter I started doing some fairly high level tests on raw leveldb/hyperleveldb/raikleveldb. I'm very interested in what you see with rocksdb as a backend. -Original Message- From: Sage Weil [mailto:s...@inktank.com] Sent: Wednesday, May 21, 2014 9:19 AM To: Shu, Xinxin Cc: ceph-devel@vger.kernel.org Subject: Re: [RFC] add rocksdb support Hi Xinxin, I've pushed an updated wip-rocksdb to github/liewegas/ceph.git that includes the latest set of patches with the groundwork and your rocksdb patch. There is also a commit that adds rocksdb as a git submodule. I'm thinking that, since there aren't any distro packages for rocksdb at this point, this is going to be the easiest way to make this usable for people. If you can wire the submodule into the makefile, we can merge this in so that rocksdb support is in the ceph.com packages on ceph.com. I suspect that the distros will prefer to turns this off in favor of separate shared libs, but they can do this at their option if/when they include rocksdb in the distro. I think the key is just to have both --with-librockdb and --with-librocksdb-static (or similar) options so that you can either use the static or dynamically linked one. Has your group done further testing with rocksdb? Anything interesting to share? Thanks! sage -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [RFC] add rocksdb support
Currently ceph will get stable rocksdb from branch 3.0.fb of ceph/rocksdb , since PR https://github.com/ceph/rocksdb/pull/2 has not been merged , so if you use 'git submodule update --init' to get rocksdb submodule , It did not support autoconf/automake . -Original Message- From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel-ow...@vger.kernel.org] On Behalf Of Sushma Gurram Sent: Saturday, June 14, 2014 2:52 AM To: Shu, Xinxin; Mark Nelson; Sage Weil Cc: ceph-devel@vger.kernel.org; Zhang, Jian Subject: RE: [RFC] add rocksdb support Hi Xinxin, I tried to compile the wip-rocksdb branch, but the src/rocksdb directory seems to be empty. Do I need toput autoconf/automake in this directory? It doesn't seem to have any other source files and compilation fails: os/RocksDBStore.cc:10:24: fatal error: rocksdb/db.h: No such file or directory compilation terminated. Thanks, Sushma -Original Message- From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel-ow...@vger.kernel.org] On Behalf Of Shu, Xinxin Sent: Monday, June 09, 2014 10:00 PM To: Mark Nelson; Sage Weil Cc: ceph-devel@vger.kernel.org; Zhang, Jian Subject: RE: [RFC] add rocksdb support Hi mark I have finished development of support of rocksdb submodule, a pull request for support of autoconf/automake for rocksdb has been created , you can find https://github.com/ceph/rocksdb/pull/2 , if this patch is ok , I will create a pull request for rocksdb submodule support , currently this patch can be found https://github.com/xinxinsh/ceph/tree/wip-rocksdb . -Original Message- From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel-ow...@vger.kernel.org] On Behalf Of Mark Nelson Sent: Tuesday, June 10, 2014 1:12 AM To: Shu, Xinxin; Sage Weil Cc: ceph-devel@vger.kernel.org; Zhang, Jian Subject: Re: [RFC] add rocksdb support Hi Xinxin, On 05/28/2014 05:05 AM, Shu, Xinxin wrote: Hi sage , I will add two configure options to --with-librocksdb-static and --with-librocksdb , with --with-librocksdb-static option , ceph will compile the code that get from ceph repository , with --with-librocksdb option , in case of distro packages for rocksdb , ceph will not compile the rocksdb code , will use pre-installed library. is that ok for you ? since current rocksdb does not support autoconfautomake , I will add autoconfautomake support for rocksdb , but before that , i think we should fork a stable branch (maybe 3.0) for ceph . I'm looking at testing out the rocksdb support as well, both for the OSD and for the monitor based on some issues we've been seeing lately. Any news on the 3.0 fork and autoconf/automake support in rocksdb? Thanks, Mark -Original Message- From: Mark Nelson [mailto:mark.nel...@inktank.com] Sent: Wednesday, May 21, 2014 9:06 PM To: Shu, Xinxin; Sage Weil Cc: ceph-devel@vger.kernel.org; Zhang, Jian Subject: Re: [RFC] add rocksdb support On 05/21/2014 07:54 AM, Shu, Xinxin wrote: Hi, sage I will add rocksdb submodule into the makefile , currently we want to have fully performance tests on key-value db backend , both leveldb and rocksdb. Then optimize on rocksdb performance. I'm definitely interested in any performance tests you do here. Last winter I started doing some fairly high level tests on raw leveldb/hyperleveldb/raikleveldb. I'm very interested in what you see with rocksdb as a backend. -Original Message- From: Sage Weil [mailto:s...@inktank.com] Sent: Wednesday, May 21, 2014 9:19 AM To: Shu, Xinxin Cc: ceph-devel@vger.kernel.org Subject: Re: [RFC] add rocksdb support Hi Xinxin, I've pushed an updated wip-rocksdb to github/liewegas/ceph.git that includes the latest set of patches with the groundwork and your rocksdb patch. There is also a commit that adds rocksdb as a git submodule. I'm thinking that, since there aren't any distro packages for rocksdb at this point, this is going to be the easiest way to make this usable for people. If you can wire the submodule into the makefile, we can merge this in so that rocksdb support is in the ceph.com packages on ceph.com. I suspect that the distros will prefer to turns this off in favor of separate shared libs, but they can do this at their option if/when they include rocksdb in the distro. I think the key is just to have both --with-librockdb and --with-librocksdb-static (or similar) options so that you can either use the static or dynamically linked one. Has your group done further testing with rocksdb? Anything interesting to share? Thanks! sage -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html --
