Re: rainbird question (why is the 1minute buffer needed?)

2011-05-23 Thread Ryan King
On Sun, May 22, 2011 at 11:00 AM, Yang tedd...@gmail.com wrote: Thanks, I did read through that pdf doc, and went through the counters code in 0.8-rc2, I think I understand the logic in that code. in my hypothetical implementation, I am not suggesting to overstep the complicated logic

Re: rainbird question (why is the 1minute buffer needed?)

2011-05-23 Thread Yang
Thanks Ryan, could you please share more details: according to what you observed in testing, why was performance worse if you do not do extra buffering? I was thinking (could be wrong) that without extra buffering, the counter update goes to Memtable.putIfPresent() and

Re: rainbird question (why is the 1minute buffer needed?)

2011-05-23 Thread Ryan King
On Mon, May 23, 2011 at 12:06 PM, Yang tedd...@gmail.com wrote: Thanks Ryan, could you please share more details: according to what you observed in testing,  why was performance  worse if you do not do extra buffering? I was thinking (could be wrong)  that without extra buffering, the

Re: rainbird question (why is the 1minute buffer needed?)

2011-05-23 Thread Jeremy Hanna
On May 23, 2011, at 2:23 PM, Ryan King wrote: On Mon, May 23, 2011 at 12:06 PM, Yang tedd...@gmail.com wrote: Thanks Ryan, could you please share more details: according to what you observed in testing, why was performance worse if you do not do extra buffering? I was thinking

Re: rainbird question (why is the 1minute buffer needed?)

2011-05-23 Thread Ryan King
Maybe. We haven't really tested it without buffering and probably won't anytime soon. 1 minute latency is good enough for what we're doing. On Mon, May 23, 2011 at 1:58 PM, Jeremy Hanna jeremy.hanna1...@gmail.com wrote: On May 23, 2011, at 2:23 PM, Ryan King wrote: On Mon, May 23, 2011 at

Re: rainbird question (why is the 1minute buffer needed?)

2011-05-22 Thread aaron morton
The implementation of distributed counters is more complicated than your example, there is a design doc attached to the ticket here https://issues.apache.org/jira/browse/CASSANDRA-1072 By collapsing some of those +1 increments together at the application level there is less work for the

Re: rainbird question (why is the 1minute buffer needed?)

2011-05-22 Thread Milind Parikh
I believe that the key reason is souped up performance for most recent data. And yes, an intelligent flush leaves you vulnerable to some data loss. /*** sent from my android...please pardon occasional typos as I respond @ the speed of thought / On May