On Aug 8, 5:52 pm, Dustin <[email protected]> wrote: > On Aug 8, 11:47 am, Neil Mckee <[email protected]> wrote: > > > I looked pretty hard at the shim idea back in May, but the engine protocol > > is really a different protocol. There was not a 1:1 correspondence with > > the standard memcached operations. > > Well, all the memcached operations are built on top of it... do you > mean specifically multiget might call into the engine multiple times > for a single "request"?
Yes. That's one example. I think there were others where the memcache operation resulted in more than one engine transaction. > > > If we define a standard sFlow-MEMCACHE measurement then it should be > > something that any memcache daemon can export, so it's really tied to the > > external spec, and shouldn't reflect anything about the internal structure > > of the implementation. > > > Instead I went back and tried to minimize the number of source code lines > > that would have to be added to memcached.c and thread.c. It's just a > > handful now. Pretty minimal. > > I can understand what you're getting at here, but if source > modification is required to integrate with your product, that sounds > like a failing from the memcached perspective. It seems like we can > reach a compromise such that your products can do all the cool stuff > they do with rich support from memcached, but without memcached having > to specifically support your products (unless I'm misunderstanding). Although there are already 30+ companies and open-source projects with sFlow collectors I fully expect most memcached users will write their own collection-and-analysis tools once they can get this data! Don't you agree? So it's not about any one collector, it's about defining a useful, scalable measurement that everyone can feel comfortable using, even in production, even on the largest clusters. On a positive note, it does seem like there is some consensus on the value of random-transaction-sampling here. But do we have agreement that this feed should be made available for external consumption (i.e. the whole cluster sends to one place that is not itself a memcached node), and that UDP should be used as the transport? I'd like to understand if we are on the same page when it comes to these broader architectural questions. Neil
