On Aug 18, 2011, at 5:25 PM, dormando wrote: >> >> Although there are already 30+ companies and open-source projects with >> sFlow collectors I fully expect most memcached users will write their >> own collection-and-analysis tools once they can get this data! Don't >> you agree? So it's not about any one collector, it's about >> defining a useful, scalable measurement that everyone can feel >> comfortable using, even in production, even on the largest clusters. >> >> On a positive note, it does seem like there is some consensus on the >> value of random-transaction-sampling here. But do we have agreement >> that this feed should be made available for external consumption (i.e. >> the whole cluster sends to one place that is not itself a memcached >> node), and that UDP should be used as the transport? I'd like to >> understand if we are on the same page when it comes to these broader >> architectural questions. > > Don't forget the original thread as well. I'm trying to solve two issues: > > 1) Sampling useful data out of a cluster. > > 2) Providing something useful for application developers > > The second case is an OS X user who fires up memcached locally, writes > some rails code, then wonders what's going on under the hood. 1-in-1000 > sampling there is counterproductive. Headers only is often useless. > > stats cachedump is most often used for the latter, and everyone needs to > remember that users never get to 1) if they can't figure out 2). Maybe I > should flip those priorities around? >
I certainly agree that you want both of these features. However they are wildly different. (1) is for monitoring in production, and (2) is for testing and troubleshooting. The requirements are so divergent that there may not be any overlap at all in the implementation of each. In fact the more separate they are the better because there is a lot of pressure on (1) to be ultra-stable and never change, while you are likely to think of new ideas for (2) all the time. So there's no need to hesitate if you can already do (1) today. Let's face it, you have been very successful and there are rather a lot of users who have already gotten past (2) :) Neil > -Dormando
