Re: Strange behaviour on get-multi over binary protocol

2013-02-19 Thread Brian Aker
Hi, On Feb 18, 2013, at 7:11 PM, dormando dorma...@rydia.net wrote: 1) libmemcached requests 2 keys 2) memcached responds with 1 key 3) libmemcached sends no-op packet 4) memcached responds with 2nd key, no-op packet Assuming one server and binary protocol... a single write to the socket

Re: Strange behaviour on get-multi over binary protocol

2013-02-19 Thread Brian Aker
Hi, On Feb 19, 2013, at 12:14 AM, dormando dorma...@rydia.net wrote: Both keys go out okay, but the no-op at the end seems to go out in a separate packet. I've noticed this on several installs using libmemcached, verified with tcpdump/etc. I didn't write this part of the binary code, Trond

Re: Strange behaviour on get-multi over binary protocol

2013-02-19 Thread Trond Norbye
Its been a while since I looked at that code but if my memory is correct we're using the quiet' mode of the get requests so that it won't send not found results. The noop is then used as an internal marker so that you know on the receiving side that you've received all of the responses from the

Re: Strange behaviour on get-multi over binary protocol

2013-02-19 Thread dormando
This is correct. You use the no-op packet to be sure you're not waiting for any more responses, since you're not going to get miss packets for missing keys. No reason for it to be a separate write/packet though. On Tue, 19 Feb 2013, Trond Norbye wrote: Its been a while since I looked at that

Re: Strange behaviour on get-multi over binary protocol

2013-02-19 Thread Brian Aker
Agreed, I'll take a look and see why that is happening. From looking at the code I can see where it is happening, I just need to find out if there was a reason for it. The default value for io_key_prefetch is zero, which is what is causing the flush to happen:

Re: Strange behaviour on get-multi over binary protocol

2013-02-19 Thread Diogo Baeder
Hi guys, Good news: everything's working now, here, with your help I figured out how to deal with this situation; However, I opted by forcing the ordering of the requests and responses in my proxy, because of the specific needs for the proxy. (I could explain in details, but I thought it would be

Strange behaviour on get-multi over binary protocol

2013-02-18 Thread Diogo Baeder
Hi guys, I'm getting a rather strange behaviour when I try to issue a get-multi on memcached, so I'm not sure this is a problem with my understanding of how the protocol was specified or if it's an issue with either pylibmc or libmemcached. What I expect: As I understand it, for multi-get,

Re: Strange behaviour on get-multi over binary protocol

2013-02-18 Thread dormando
However, after trying to follow this behaviour in a proxy I'm building, this order of interactions is not being respected; So, what I did afterwards, to assert that something strange was going on, was to fire up Wireshark and listen for memcached requests and responses. Here's a sample of

Re: Strange behaviour on get-multi over binary protocol

2013-02-18 Thread Diogo Baeder
Hi Dormando, So, I've tested Memcached by sending direct byte blocks to it in the order I expected it to behave, and indeed it works, as you said it would: https://gist.github.com/diogobaeder/4982425 I have no clue why libmemcached does that switch in the middle, but I understood what you

Re: Strange behaviour on get-multi over binary protocol

2013-02-18 Thread Brian Aker
Hi, On Feb 18, 2013, at 6:03 PM, Diogo Baeder diogobae...@gmail.com wrote: I have no clue why libmemcached does that switch in the middle, but I understood what you said about not expecting things to happen in an exact order Are you sure the data is on the same server? Libmemcached

Re: Strange behaviour on get-multi over binary protocol

2013-02-18 Thread dormando
On Feb 18, 2013, at 6:03 PM, Diogo Baeder diogobae...@gmail.com wrote: I have no clue why libmemcached does that switch in the middle, but I understood what you said about not expecting things to happen in an exact order Are you sure the data is on the same server? Libmemcached

Re: Strange behaviour on get-multi over binary protocol

2013-02-18 Thread Diogo Baeder
Yep, agreed, Dormando, not a problem, just different from my initial expectations. I'll just have to figure out how to use Tornado in my favor, to build this part, and deal correctly with the asynchronicity. :-) Cheers! __ Diogo Baeder http://diogobaeder.com.br On Tue,

Re: Strange behaviour on get-multi over binary protocol

2013-02-18 Thread Diogo Baeder
Guess what: I just built a fake Memcached server, to answer hardcoded values (same as before) to a get_multi op for pylibmc, with normally ordered batches (2 reqs, 1 noop, 2 resp, 1 noop), and it worked. So, in the end, it seems like forcing the ordering is not what is causing me troubles, it's