Hi,
Eric Lambert fixed up some of the wording in the binary protocol
specification (check out http://github.com/elambert/memcached/tree/binprot
)
Please make sure we get this fix merged in as well :)
Cheers,
Trond
On Oct 19, 2008, at 3:21 PM, Toru Maesaka wrote:
Hi guys,
I fixed the stats opaque issue pointed out by Dustin's test case at
the hackathon and just pushed it to github:
http://github.com/tmaesaka/memcached/commit/86159eb7b5c611d25a33eb1cb6c750c806350912
I also refactored 'server_stats()' which is a function that returns
server specific stats. In the previous implementation, a string in the
usual stats format was created and returned if server_stats() was
working for the ascii protocol. Otherwise, server_stats() would parse
the string and create binary protocol packets based on that data.
The above approach is not nice at all and I didn't like it from the
beginning. It was implemented like that because it was written before
the callback idea was adopted. Now there are no conditional statements
to determine whether server_stats() is working for the binary protocol
or not (because the appropriate callback is now passed to
server_stats).
Oh, and I noticed that Dustin had merged my binary tree and there is
something I feel really guilty about... Some pushes I made to my tree
were done manually to bring over 1.2.6/1.2.7 stable patches into the
binary tree. This is probably going to hit us badly when merging other
trees even with the same ancestor (e.g. brad's
http://github.com/bradfitz/memcached/commits/incr_fix).
Theres probably a way to fix this but I'm not quite a git ninja yet...
Does anyone have any thoughts/solution for this?
Cheers,
Toru
On Fri, Oct 17, 2008 at 5:08 AM, Ben Manes <[EMAIL PROTECTED]>
wrote:
"Trond also mentioned interesting lock reduction ideas, like moving
from a
single cache lock to lock per hash region (perhaps like java
concurrent hash
map). I think Marc, in a different conversation, suggested the
idea of a
lock-free hash table (ala Cliff Click)."
You may also want to consider the hopscotch algorithm:
http://groups.google.com/group/hopscotch-hashing?pli=1
----- Original Message ----
From: steve.yen <[EMAIL PROTECTED]>
To: memcached <[email protected]>
Sent: Thursday, October 16, 2008 9:08:26 AM
Subject: Re: Memcached Hackathon Report(s)
A fun hackathon, folks were there until 2am. Other conversations
that
I recall...
- Paul Saab & Marc Kwiatkowski described a bunch of the Facebook
changes that helped them get towards 300K reqs/sec on 8 core server.
I think they worked a bit towards putting up an svn/git repo so their
changes
could be more easily folded back by the community into the mainline
(putting
out something with change history rather than just a huge hard-to-
digest patch).
Appreciate that!
- In particular, they described one performance blocker which they
fixed
of the single stats lock and how they moved towards a stats-per-
thread
model that tracks info in thread local data. These TLD stats are
aggregated lazily, on-demand in response to a stats request.
- Trond and I talked a little about making the upcoming storage
engine
interface
async rather than synchronous flavored. Trond also mentioned
interesting lock
reduction ideas, like moving from a single cache lock to lock per
hash
region
(perhaps like java concurrent hash map). I think Marc, in a
different
conversation,
suggested the idea of a lock-free hash table (ala Cliff Click).
- For those of you who thought a hash table per thread might be
interesting (like me),
Trond says it's been tried and is not an improvement. The contention
just moves
towards the request queueing in front of each thread.
- There was talk and progress about adding replication into
libmemcached client lib.
Senthil (apologies if I misremembered your name) from Yahoo did the
work, and we
talked with Brian Aker about replication issues, such as handling
flapping nodes.
steve
On Oct 15, 1:48 am, Dustin <[EMAIL PROTECTED]> wrote:
The hackathon went well again. Admittedly, I didn't talk to
everyone
there, but from the corner with the food, there was some additional
protocol work and tree merging going on. Please fill in interesting
things that happened that I didn't know about or forgot.
In particular, Toru implemented stats for the binary protocol. It's
pretty good stuff. I wrote java client support for it, and helped
iron out some minor issues that arose.
I think the only change that came out of this was that the opaque
behavior was kind of undefined. All opaque values coming out of
stats
were set to 0 (and the requested opaque was ignored). Stats values
will have the magic opaque value of 0x746f7275 and the terminating
stat will have the requested opaque value.
*INCOMPATIBILITY NOTES*
We have one change coming up that has been affecting me a lot re:
compatibility and that's the removal of hold values from delete. If
you rely on this behavior, please let someone know -- it's going
away.
Brad did some work on making the long parser more strict in incr/
decr. If you rely on arbitrary objects being treated as 0, you
have a
silent bug that will turn into a loud bug in the future.
--
Trond Norbye
Database Technology Group E-mail: [EMAIL PROTECTED]
SUN Microsystems Phone: +47 73842100
Haakon VII's gt. 7B Fax: +47 73842101
7485 Trondheim, Norway
