It looks like there is ordering within one client (ordering based on timestamp) and looks like this *order is preserved across all replicas* however the benefits of async given the ordering restriction are slightly blur for me.
On Tue, Feb 21, 2017 at 2:35 AM, Kant Kodali <k...@peernova.com> wrote: > Agreed with multiple clients one cannot guarantee the order however with > multiple clients the client side timestamps will overlap as well. so even > in the case of LWT's and multiple clients the order is not guaranteed > right. By multiple clients I mean multiple C* sessions on the driver side. > if multiple LWT's have same time stamps from different clients I would > assume one of the LWT's from one client can be overwritten by the other LWT > from another client with same timestamp. > > > > > > On Tue, Feb 21, 2017 at 1:52 AM, Benjamin Roth <benjamin.r...@jaumo.com> > wrote: > >> For eventual consistency, it does not matter if it is sync or async. LWW >> always works as long as clocks are synchronized. >> Thats a design pattern of CS or EC databases in general. Every write has >> a timestamp and no matter at what time it arrives, the last write will win >> even if a "sooner" write arrives late due to network latency oder a >> unavailable server that receives a hint after 1 hour. >> Doing replication sync will kill all the benefits you have from CS's >> design: >> - low latency >> - partition tolerance >> - high availability >> >> Doing sync replication would also not guarantee a state as another client >> could "interfer" with your write. So you still have no "linearizability". >> Only LWT does this. >> You cannot rely on orders in CS. No matter how replication works. You >> only can rely "eventually" on it but there is never a point in time you can >> tell 100% your system is completely consistent. >> >> Maybe what you could do if you are talking of "orders" and that pointer >> thing you mentioned earlier: Try sth similar like MVs do. >> Create a trigger, operate on your local dataset, read the order based on >> PK (locally) and update "the pointer" on every write (also locally). If you >> then store your pointer with the last known timestamp of your base data, >> you also have a LWW on your pointer so also the last pointer wins when >> reading with > CL_ONE. >> But that will probably harm your write performance. >> >> 2017-02-21 10:36 GMT+01:00 Kant Kodali <k...@peernova.com>: >> >>> @Benjamin I am more looking for how C* replication works underneath. >>> There are few things here that I would need some clarification. >>> >>> 1. Does C* uses sync replication or async replication? If it is async >>> replication how can one get performance especially when there is an >>> ordering constraint among requests to comply with LWW. Also below is a >>> statement from C* website so how can one choose between sync or async >>> replication? any configuration parameter that needs to be passed in? >>> >>> "Choose between synchronous or asynchronous replication for each update. >>> " >>> >>> http://cassandra.apache.org/ >>> >>> 2. Is it Guaranteed that C* coordinator writes data in the same order to >>> all the replicas (either sync or async)? >>> >>> Thanks, >>> kant >>> >>> On Tue, Feb 21, 2017 at 1:23 AM, Benjamin Roth <benjamin.r...@jaumo.com> >>> wrote: >>> >>>> To me that sounds like a completely different design pattern and a >>>> different use case. >>>> CS was not designed to guarantee order. It was build to be linear >>>> scalable, highly concurrent and eventual consistent. >>>> To me it sounds like a ACID DB better serves what you are asking for. >>>> >>>> 2017-02-21 10:17 GMT+01:00 Kant Kodali <k...@peernova.com>: >>>> >>>>> Agreed that async performs better than sync in general but the catch >>>>> here to me is the "order". >>>>> >>>>> The whole point of async is to do out of order processing by which I >>>>> mean say if a request 1 comes in at time t1 and a request 2 comes in at >>>>> time t2 where t1 < t2 and say now that t1 is taking longer to process than >>>>> t2 in which case request 2 should get a response first and subsequently a >>>>> response for request 1. This is where I would imagine all the benefits of >>>>> async come in but the moment you introduce order by saying for Last Write >>>>> Wins all the async requests should be processed in order I would imagine >>>>> all the benefits of async are lost. >>>>> >>>>> Let's see if anyone can comment about how it works inside C*. >>>>> >>>>> Thanks! >>>>> >>>>> >>>>> >>>>> On Mon, Feb 20, 2017 at 10:54 PM, Dor Laor <d...@scylladb.com> wrote: >>>>> >>>>>> Could be. Let's stay tuned to see if someone else pick it up. >>>>>> Anyway, if it's synchronous, you'll have a large penalty for latency. >>>>>> >>>>>> On Mon, Feb 20, 2017 at 10:11 PM, Kant Kodali <k...@peernova.com> >>>>>> wrote: >>>>>> >>>>>>> Thanks again for the response! if they mean it between client and >>>>>>> server I am not sure why they would use the word "replication" in the >>>>>>> statement below since there is no replication between client and server( >>>>>>> coordinator). >>>>>>> >>>>>>> "Choose between synchronous or asynchronous replication for each >>>>>>>> update." >>>>>>>> >>>>>>> >>>>>>> Sent from my iPhone >>>>>>> >>>>>>> On Feb 20, 2017, at 5:30 PM, Dor Laor <d...@scylladb.com> wrote: >>>>>>> >>>>>>> I think they mean the client to server and not among the servers >>>>>>> >>>>>>> On Mon, Feb 20, 2017 at 5:28 PM, Kant Kodali <k...@peernova.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Also here is a statement from C* website >>>>>>>> >>>>>>>> "Choose between synchronous or asynchronous replication for each >>>>>>>> update." >>>>>>>> >>>>>>>> http://cassandra.apache.org/ >>>>>>>> >>>>>>>> Looks like we can choose then either sync or async then? >>>>>>>> >>>>>>>> On Mon, Feb 20, 2017 at 5:25 PM, Kant Kodali <k...@peernova.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Dor, >>>>>>>>> >>>>>>>>> Great response! My comments are inline. >>>>>>>>> >>>>>>>>> Thanks a lot, >>>>>>>>> kant >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Feb 20, 2017 at 4:41 PM, Dor Laor <d...@scylladb.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> I sent this answer but it bounced off the user@apache. >>>>>>>>>> Here is the email anyway: >>>>>>>>>> >>>>>>>>>> ---------- Forwarded message ---------- >>>>>>>>>> From: Dor Laor <d...@scylladb.com> >>>>>>>>>> Date: Mon, Feb 20, 2017 at 4:37 PM >>>>>>>>>> Subject: Re: Does C* coordinator writes to replicas in same order >>>>>>>>>> or different order? >>>>>>>>>> To: d...@cassandra.apache.org >>>>>>>>>> Cc: user@cassandra.apache.org >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> + The C* coordinator send async write requests to the replicas. >>>>>>>>>> This is very important since it allows it to return a low >>>>>>>>>> latency >>>>>>>>>> reply to the client once the CL is reached. You wouldn't want >>>>>>>>>> to serialize the replicas one after the other. >>>>>>>>>> >>>>>>>>> >>>>>>>>> *so coordinator wont wait until a CL is reached before it >>>>>>>>> process another request? * >>>>>>>>> >>>>>>>>>> >>>>>>>>>> + The client <-> server sync/async isn't related to the >>>>>>>>>> coordinator >>>>>>>>>> in this case. >>>>>>>>>> >>>>>>>>>> + In the case of concurrent writes (always the case...), the >>>>>>>>>> time stamp >>>>>>>>>> sets the order. Note that it's possible to work with client >>>>>>>>>> timestamps or >>>>>>>>>> server timestamps. The client ones are usually the best >>>>>>>>>> choice. >>>>>>>>>> >>>>>>>>> >>>>>>>>> *In theory, Why we say concurrent writes they should have the >>>>>>>>> same timestamp right? What I am really looking for is that if I send >>>>>>>>> write >>>>>>>>> request concurrently for record 1 and record 2 are they guaranteed to >>>>>>>>> be >>>>>>>>> inserted in the same order across replicas? (Whatever order >>>>>>>>> coordinator may >>>>>>>>> choose is fine but I want the same order across all replicas and with >>>>>>>>> async >>>>>>>>> replication I am not sure how that is possible ? for example, if a >>>>>>>>> request >>>>>>>>> arrives with timestamp t1 and another request arrives with a >>>>>>>>> timestamp t2 >>>>>>>>> where t1 < t2...with async replication what if one replica chooses to >>>>>>>>> execute t2 first and then t1 simply because t1 is slow while another >>>>>>>>> replica choose to execute t1 first and then t2..how would that work? >>>>>>>>> )* >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Note that C* each node can be a coordinator (one per request) and >>>>>>>>>> its >>>>>>>>>> the desired case in order to load balance the incoming requests. >>>>>>>>>> Once again, >>>>>>>>>> timestamps determine the order among the requests. >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Dor >>>>>>>>>> >>>>>>>>>> On Mon, Feb 20, 2017 at 4:12 PM, Kant Kodali <k...@peernova.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> when C* coordinator writes to replicas does it write it in same >>>>>>>>>>> order or >>>>>>>>>>> different order? other words, Does the replication happen >>>>>>>>>>> synchronously or >>>>>>>>>>> asynchrnoulsy ? Also does this depend sync or async client? What >>>>>>>>>>> happens in >>>>>>>>>>> the case of concurrent writes to a coordinator ? >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> kant >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Benjamin Roth >>>> Prokurist >>>> >>>> Jaumo GmbH · www.jaumo.com >>>> Wehrstraße 46 · 73035 Göppingen · Germany >>>> Phone +49 7161 304880-6 <+49%207161%203048806> · Fax +49 7161 304880-1 >>>> <+49%207161%203048801> >>>> AG Ulm · HRB 731058 · Managing Director: Jens Kammerer >>>> >>> >>> >> >> >> -- >> Benjamin Roth >> Prokurist >> >> Jaumo GmbH · www.jaumo.com >> Wehrstraße 46 · 73035 Göppingen · Germany >> Phone +49 7161 304880-6 <+49%207161%203048806> · Fax +49 7161 304880-1 >> <+49%207161%203048801> >> AG Ulm · HRB 731058 · Managing Director: Jens Kammerer >> > >
