Re: Use of OpOrder in memtable
Right. Although IIRC there is another OpOrder that coordinates the migration of readers from memtables to the sstables that replace them (and the reclamation of any off-heap memory associated with the memtable) On 13 February 2018 at 22:59, Tyagi, Preetika wrote: > Ah I see. That makes sense. > And it doesn't have to anything with the read requests going on in > parallel with write requests, right? > I mean we read the data from memtable depending on whatever has been > written into memtable so far and return it to the client (of course > including SSTable read and timestamp comparison etc.) > > -Original Message- > From: Benedict Elliott Smith [mailto:bened...@apache.org] > Sent: Tuesday, February 13, 2018 2:25 PM > To: dev@cassandra.apache.org > Subject: Re: Use of OpOrder in memtable > > If you look closely, there can be multiple memtables extant at once. > While all "new" writes are routed to the latest memtable, there may still > be writes that have begun but not yet completed. The memtable cannot be > flushed until any stragglers have completed, and some stragglers *may* > still need to be routed to their designated memtable (if they had only just > begun when the flush triggered). It helps avoid these race conditions on > either side of the equation. > > On 13 February 2018 at 22:09, Tyagi, Preetika > wrote: > > > Hi all, > > > > I'm trying to understand the behavior of memtable when writes/flush > > operations are going on in parallel. > > > > In my understanding, once a memtable is full it is queued for flushing > > and a new memtable is created for ongoing write operations. > > However, I was looking at the code and it looks like the OpOrder class > > is used (don't understand all details) to ensure the synchronization > > between producers (writes) and consumers (batch flushes). > > So I am a bit confused about when exactly it is needed. There will > > always be only one latest memtable for write operations and all old > > memtables are flushed so where this producer/consumer interaction on > > the same memtable is needed? > > > > Thanks, > > Preetika > > > > >
RE: Use of OpOrder in memtable
Ah I see. That makes sense. And it doesn't have to anything with the read requests going on in parallel with write requests, right? I mean we read the data from memtable depending on whatever has been written into memtable so far and return it to the client (of course including SSTable read and timestamp comparison etc.) -Original Message- From: Benedict Elliott Smith [mailto:bened...@apache.org] Sent: Tuesday, February 13, 2018 2:25 PM To: dev@cassandra.apache.org Subject: Re: Use of OpOrder in memtable If you look closely, there can be multiple memtables extant at once. While all "new" writes are routed to the latest memtable, there may still be writes that have begun but not yet completed. The memtable cannot be flushed until any stragglers have completed, and some stragglers *may* still need to be routed to their designated memtable (if they had only just begun when the flush triggered). It helps avoid these race conditions on either side of the equation. On 13 February 2018 at 22:09, Tyagi, Preetika wrote: > Hi all, > > I'm trying to understand the behavior of memtable when writes/flush > operations are going on in parallel. > > In my understanding, once a memtable is full it is queued for flushing > and a new memtable is created for ongoing write operations. > However, I was looking at the code and it looks like the OpOrder class > is used (don't understand all details) to ensure the synchronization > between producers (writes) and consumers (batch flushes). > So I am a bit confused about when exactly it is needed. There will > always be only one latest memtable for write operations and all old > memtables are flushed so where this producer/consumer interaction on > the same memtable is needed? > > Thanks, > Preetika > >
Re: Use of OpOrder in memtable
If you look closely, there can be multiple memtables extant at once. While all "new" writes are routed to the latest memtable, there may still be writes that have begun but not yet completed. The memtable cannot be flushed until any stragglers have completed, and some stragglers *may* still need to be routed to their designated memtable (if they had only just begun when the flush triggered). It helps avoid these race conditions on either side of the equation. On 13 February 2018 at 22:09, Tyagi, Preetika wrote: > Hi all, > > I'm trying to understand the behavior of memtable when writes/flush > operations are going on in parallel. > > In my understanding, once a memtable is full it is queued for flushing and > a new memtable is created for ongoing write operations. > However, I was looking at the code and it looks like the OpOrder class is > used (don't understand all details) to ensure the synchronization between > producers (writes) and consumers (batch flushes). > So I am a bit confused about when exactly it is needed. There will always > be only one latest memtable for write operations and all old memtables are > flushed so where this producer/consumer interaction on the same memtable is > needed? > > Thanks, > Preetika > >
Use of OpOrder in memtable
Hi all, I'm trying to understand the behavior of memtable when writes/flush operations are going on in parallel. In my understanding, once a memtable is full it is queued for flushing and a new memtable is created for ongoing write operations. However, I was looking at the code and it looks like the OpOrder class is used (don't understand all details) to ensure the synchronization between producers (writes) and consumers (batch flushes). So I am a bit confused about when exactly it is needed. There will always be only one latest memtable for write operations and all old memtables are flushed so where this producer/consumer interaction on the same memtable is needed? Thanks, Preetika
