On Wed, Dec 10, 2014 at 9:10 AM, Wang, Zhiqiang <[email protected]> wrote: > For multiple clients accessing the same volume, same object, I guess the > clients need to do some synchronization between them to guarantee what it > reads is what it wrote. That is to say, if two clients, say A and B, A is > doing write, B is doing read. If B wants to read what A writes, it needs to > know A has completed the write before issuing the read. If A hasn't completed > the write yet, it could be possible that the write fails, so you can't expect > B reads what A writes. I think this makes sense from the client/application > perspective.
I think each write for a object is a barrier. If receiving a write op, previous read ops all are allowed to out of order and read ops after write op also can out of order. I think it's still SERIALIZABLE level for rados client. > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Sage Weil > Sent: Wednesday, December 10, 2014 8:43 AM > To: Cook, Nigel > Cc: Yehuda Sadeh; Josh Durgin; Samuel Just; Wang, Zhiqiang; Jason Dillaman; > ceph-devel > Subject: Re: rados read ordering > > On Wed, 10 Dec 2014, Cook, Nigel wrote: >> Folks >> >> I'm wondering if this is related to the question I posed a few days >> ago.. >> >> Can CEPH support 2 clients simultaneously accessing a single volume - >> for example a database cluster - and honor read and write order of >> blocks across the multiple clients? >> >> Can you comment? > > I don't think so. This (non)change is about a single client submitting two > reads to a single object (say, different blocks in the same disk) and whether > the OSD is allowed to respond out of order (say, because some blocks are in > cache and some aren't). > > In the shared volume case, it is generally not important what happens with > requests that are submitted in parallel.. they can take different amounts of > time on the wire and which happens first (i.e., which arrives at the OSD > first) depends on happenstance. What does matter is that any read that > happens after a complete write reflects that read, which is why the concern > is around caching. > > sage > > >> >> Regards, >> Nigel Cook +1 720 319 7508 >> >> Sent from a mobile device. >> Please excuse both my brevity and sp3lling >> >> On Dec 8, 2014 10:12 AM, Yehuda Sadeh <[email protected]> wrote: >> On Mon, Dec 8, 2014 at 9:03 AM, Sage Weil <[email protected]> wrote: >> > The current RADOS behavior is that reads (on any given object) are always >> > processed in the order they are submitted by the client. This causes a >> > few headaches for the cache tiering that it would be nice to avoid. It >> > also occurs to me that there are likely cases where we could go a lot >> > faster by not strictly ordering things. For example, a stat can respond >> > more quickly than a large read, and some reads may hit cache while others >> > go to disk. This doesn't happen currently because of the (lame) way we do >> > reads synchronously, but hope that can change too. >> > >> > I propose we drop this semantic. If a client wants reads to have a strict >> > ordering, they can set the existing RWORDERED flag (which also orders them >> > with respect to writes). That's not the most general thing ever, but I'm >> > not sure we care about callers who want reads ordered with respect to each >> > other but not writes. >> > >> > The real question is whether there are any users that want/need this >> > currently. I can't think of any offhand. In several places we submit >> > multiple *writes* and expect them to be strictly ordered (e.g., we >> > set a completion on teh last write only). I don't think we do this >> > anywhere for reads though... >> > >> > Josh, Yehuda, Jason--can you think of any in RBD or RGW that would depend >> > on this? >> > >> >> None that I can think of. For objects data, we already stripe it >> across multiple objects, and the underlying assumption is that we're >> going to get responses out of order so we make sure we commit >> in-order. Guards are used on the head object, and the read is >> synchronous there anyway. I can't think of any other place where we'd >> have an issue. >> >> Yehuda >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to [email protected] >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to [email protected] >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to [email protected] > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to [email protected] > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Best Regards, Wheat -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
