Issue 3 in memcached: Textual delete command w/ expire time noreply in binary protocol version

2009-10-29 Thread memcached



Comment #4 on issue 3 by dsallings: Textual delete command w/ expire time   
noreply in binary protocol version

http://code.google.com/p/memcached/issues/detail?id=3

I've pushed a branch for this (finally... this is an old one).

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


Issue 54 in memcached: init script missing LSB section

2009-10-29 Thread memcached


Updates:
Status: Started

Comment #2 on issue 54 by dsallings: init script missing LSB section
http://code.google.com/p/memcached/issues/detail?id=54

I pushed a branch with Monty's change.  The other change requires patching  
files that
are not in the distribution (and sounds like a different bug if we wanted  
to take

action on it).

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


Issue 56 in memcached: Minimum slab size is particular for growth factor of 2

2009-10-29 Thread memcached


Updates:
Status: Started

Comment #1 on issue 56 by dsallings: Minimum slab size is particular for  
growth factor of 2

http://code.google.com/p/memcached/issues/detail?id=56

I agree.  This is entirely unexpected behavior and the history doesn't tell  
us why
someone made it a special case and if you want this you can easily get the  
same

effect by using -n 90 (but if you don't want the effect, you can't undo it).

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


Issue 62 in memcached: Patch: connection queue simplification

2009-10-29 Thread memcached


Updates:
Status: Invalid

Comment #9 on issue 62 by dsallings: Patch: connection queue simplification
http://code.google.com/p/memcached/issues/detail?id=62

I don't think there's a bug here.  This looks like something interesting  
could come
out of it, but there are a lot of things to consider in reducing the  
complexity of a

working system.

I wouldn't discourage work towards it, but consider the feedback given here.
Probably better to have this kind of discussion on the mailing list.

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


Issue 67 in memcached: TCP/UDP ports need to follow unless explicitly told not to

2009-10-29 Thread memcached



Comment #4 on issue 67 by dsallings: TCP/UDP ports need to follow unless  
explicitly told not to

http://code.google.com/p/memcached/issues/detail?id=67

mdounin -- I do like this idea.  We had this discussion in an email thread  
and it
turned into a bikeshed.  No two participants so far agree on the  
behavior.  :)


http://groups.google.com/group/memcached/browse_thread/thread/1f8dcbc5efb1e9eb/fa1dadb60c934d13

Personally, I'd like *something* here.

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


Issue 68 in memcached: incr/decr resize corruption

2009-10-29 Thread memcached


Updates:
Status: Started

Comment #2 on issue 68 by dsallings: incr/decr resize corruption
http://code.google.com/p/memcached/issues/detail?id=68

I wrote a test that confirms that resizing itself does not cause corruption.
Inspection of the current code doesn't show how this would be possible, so  
I'm
considering this an issue whose fix sprang miraculously from the many  
refinements
that have occurred in this area since the version this discussion was  
referencing

(which was, I believe an early 1.2).

Anyone who disagrees, please update the test.  :)

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


dormando's awesome memcached top v0.1

2009-10-29 Thread dormando

Yo,

I couldn't sleep, so:
http://github.com/dormando/damemtop
(or: http://consoleninja.net/code/memcached/damemtop-0.1.tar.gz)

Early release of a utility I've been working on in the last few days. Yes,
sorry, I'm aware this makes /four/ memcached top programs. So, I had to
make mine awesome.

In order to be truly awesome, I need to spend another day working on it to
add a few things, but it's in a state now where it can be useful to
people. So, up it goes, and I'll take feedback/ideas/patches.

In short, it's a top utility which lets you take any stat memcached spits
out from 'stats', 'stats items', or 'stats slabs', and display it in a
'top' like interface. With totals, averages, etc. It also supports
computed columns, hit_rate, fill_rate, soon to be many more. Finally,
you can choose an arbitrary column to sort the output. I have more
memcached's than will fit on a stretched out terminal, so it's nice to be
able to sort :)

In order to change the display around you'll need to edit the
damemtop.yaml file (example included). Also in order to run it at all
you'll need to install AnyEvent and YAML CPAN modules. I'm brutally aware
of adversity for installing simple modules, but these are in very common
use, and AnyEvent allows the utility to scale to hundreds of instances. It
takes 0.2 seconds to poll every single stat and display against TypePad's
entire cluster.

Upcoming ideas/features:
- a '--helpme' mode that makes a big YAML dump folks can share with the
  mailing list to expediate assistance.
- many more computed columns.
- a drill down mode for exploring a single or custom set of instances.
- a slabs mode for easy analysis and aggregation of the individual slab
  stats.
- online config editing.
- more formatting. shorteners for large numbers. bytes - K - M -
  G/etc.
- better docs, more fleshed out config loader.
- scrolling output modes.
- multi-cluster support (switch views between groups of servers)
- rolling averages for some views.
- latency monitor (testing a bunch of commands)
- YAML output/input modes for logging, output into monitoring/graphing
  systems, input into multiple 'damemtop' listeners.
- pretty colors.
- reorganize code a little. It got messier than I like :/

Dunno... stuff? Maybe a quickie mode that can give you warnings or notes
about your configuration based on current stats? I'll work on this for a
few hours each week and kick out a new version for a month or so. I don't
expect (nor want) it to reach the complexity of something like innotop.

The intent for this module to replace the 'scripts/memcached-tool'
program, and be distributed with memcached itself.

have fun,
-Dormando


Re: noreply error

2009-10-29 Thread kroki

On Oct 23, 9:36 pm, dormando dorma...@rydia.net wrote:
 Try it without the '0' in there. The delete expiration was (the only?)
 incompatible change from 1.2.8 to 1.4.0. It's gone now.

What was the reason to *break* the protocol then?  Couldn't memcached
just ignore the expiration time here?  What happens now is: a client
sends delete key 0 noreply.  Server answers ERROR, but the client
doesn't expect the reply (because of noreply).  The client then
sends some other command, say set bla-bla, now reads ERROR, then
sends get bla-bla, reads STORED, etc.  I.e. all replies are
shifted because of that unexpected extra ERROR.  What was the
point of that?

The commit in question is 5da8dbab2a815c00617ab60b641391ada8d96f40.
Can it be fixed?


Re: noreply error

2009-10-29 Thread Trond Norbye


kroki wrote:

On Oct 29, 1:10 pm, dormando dorma...@rydia.net wrote:
  

The reason was that nobody was using that extra parameter and the
developers wanted to remove it. Also, despite having *a year's*
worth of releases and prodding on the mailing list for client authors to
test, this is the first time it's come up.

We already have issue #3 filed to adjust this :P It's ancient but we were
already planning on adjusting it for the next release. I'd suggest you
look at the issue at code.google.com/p/memcached and see if the ideas are
on track? That'd be a big help!



I don't see how fix for issue#3 helps.  The test case even expects
that bogus ERROR (see
http://github.com/dustin/memcached/blob/f1c6f07044056f09c1f960d226588a5dd5521e2e/t/issue_3.t#L19).

I'm not telling that expiration should work.  I only say that delete
key NUM and delete key NUM noreply should be a valid syntax.  This
is because:

 - there exist more clients than memcached developers are aware of
(many sites implement their own lightweight versions).

 - not every client author constantly monitors the list and is ready
to invest the time to update the client every time memcached developer
comes with the idea of what will be better for everyone.

 - not every installation uses latest version of memcached, so
existing clients have to be compatible with both old and new server
versions.

There was no technical reasons to break backward compatibility, and it
should be easy to restore it ;).
  
There is, because delayed delete was a bit special.. it would pin the 
object in memory until the expiry time ran out making add / replace / 
incr / delete fail.. By just ignoring the number we would break the 
logic the clients expected.. I think it is a lot better that the server 
sends back an error message and let the client handle that instead.


I would also find it extremely hard to believe that people will go from 
memcached 1.2.x to 1.4.x without testing that the clients they use still 
work with the server (and deferred deletes is still available in 1.2.8).


Cheers,

Trond



Re: noreply error

2009-10-29 Thread kroki

On Oct 29, 3:47 pm, Trond Norbye trond.nor...@sun.com wrote:
 I think it is a lot better that the server
 sends back an error message and let the client handle that instead.

Handle... how?  What are your suggestions on how to support this in
clients?


Issue 54 in memcached: init script missing LSB section

2009-10-29 Thread memcached


Updates:
Status: Fixed

Comment #3 on issue 54 by trond.norbye: init script missing LSB section
http://code.google.com/p/memcached/issues/detail?id=54

Pushed

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


Re: noreply error

2009-10-29 Thread kroki

On Oct 29, 4:00 pm, kroki tomash.brec...@gmail.com wrote:
 Handle... how?  What are your suggestions on how to support this in
 clients?

Note that even Brian's libmemcached fails to support this protocol
change.  From libmemcached-0.34:

if (expiration)
  send_length= (size_t) snprintf(buffer,
MEMCACHED_DEFAULT_COMMAND_SIZE,
 delete %s%.*s %u%s\r\n,
 ptr-prefix_key,
 (int) key_length, key,
 (uint32_t)expiration, no_reply ?
 noreply : );
else
   send_length= (size_t) snprintf(buffer,
MEMCACHED_DEFAULT_COMMAND_SIZE,
  delete %s%.*s%s\r\n,
  ptr-prefix_key,
  (int)key_length, key, no_reply ?
 noreply :);

(expiration is a time_t).  The code says that if expiration is non-
zero (old behavior) the command will be delete key EXP [noreply].
And it expiration is zero the command will be delete key [noreply].
Clearly, there's no way to send zero expiration time to memcached 
1.4.0, which is simply broken because normally zero is passed by the
user.  So delete doesn't work for libmemcached+memcached  1.4.0.

Please reconcider ;).


Re: noreply error

2009-10-29 Thread Trond Norbye


kroki wrote:

On Oct 29, 3:47 pm, Trond Norbye trond.nor...@sun.com wrote:
  

I think it is a lot better that the server
sends back an error message and let the client handle that instead.



Handle... how?  What are your suggestions on how to support this in
clients?
  
Clients should always be able to receive error messages. This specific 
problem is a bit tough, because you send an invalid command to the 
server and believes that it is a legal command you shouldn't receive any 
output from, so you will most likely not see the error before you 
execute the next command you expect a response to. With the ASCII 
protocoll you cannot map the error back to the command causing the 
error, but that problem is addressed in the binary protocol (the error 
packet contains the command opcode and the opaque field).


This specific problem is something you should discover when you run your 
test suite towards the new server to before deploying it.


Cheers,

Trond




Issue 68 in memcached: incr/decr resize corruption

2009-10-29 Thread memcached


Updates:
Status: Fixed

Comment #3 on issue 68 by trond.norbye: incr/decr resize corruption
http://code.google.com/p/memcached/issues/detail?id=68

I'll close the bug for now, but feel free to reopen the bug if you can  
reproduce the

bug with 1.4.2 or newer.

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


Issue 57 in memcached: Second biggest slabclass is always less than half the biggest

2009-10-29 Thread memcached



Comment #2 on issue 57 by trond.norbye: Second biggest slabclass is always  
less than half the biggest

http://code.google.com/p/memcached/issues/detail?id=57

The lru test fails with your patch

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


Issue 56 in memcached: Minimum slab size is particular for growth factor of 2

2009-10-29 Thread memcached



Comment #2 on issue 56 by trond.norbye: Minimum slab size is particular for  
growth factor of 2

http://code.google.com/p/memcached/issues/detail?id=56

LRU test fails with this patch

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


Re: dormando's awesome memcached top v0.1

2009-10-29 Thread Jay Paroline

Just got this running on my box pointing at all our servers, so far
it's looking good! The only hiccup for me was that it expected the
yaml file to be in /etc and I was just editing it in place. Paying
attention to the error message made it pretty obvious what I did wrong
though.

Jay

dormando wrote:
 Yo,

 I couldn't sleep, so:
 http://github.com/dormando/damemtop
 (or: http://consoleninja.net/code/memcached/damemtop-0.1.tar.gz)

 Early release of a utility I've been working on in the last few days. Yes,
 sorry, I'm aware this makes /four/ memcached top programs. So, I had to
 make mine awesome.

 In order to be truly awesome, I need to spend another day working on it to
 add a few things, but it's in a state now where it can be useful to
 people. So, up it goes, and I'll take feedback/ideas/patches.

 In short, it's a top utility which lets you take any stat memcached spits
 out from 'stats', 'stats items', or 'stats slabs', and display it in a
 'top' like interface. With totals, averages, etc. It also supports
 computed columns, hit_rate, fill_rate, soon to be many more. Finally,
 you can choose an arbitrary column to sort the output. I have more
 memcached's than will fit on a stretched out terminal, so it's nice to be
 able to sort :)

 In order to change the display around you'll need to edit the
 damemtop.yaml file (example included). Also in order to run it at all
 you'll need to install AnyEvent and YAML CPAN modules. I'm brutally aware
 of adversity for installing simple modules, but these are in very common
 use, and AnyEvent allows the utility to scale to hundreds of instances. It
 takes 0.2 seconds to poll every single stat and display against TypePad's
 entire cluster.

 Upcoming ideas/features:
 - a '--helpme' mode that makes a big YAML dump folks can share with the
   mailing list to expediate assistance.
 - many more computed columns.
 - a drill down mode for exploring a single or custom set of instances.
 - a slabs mode for easy analysis and aggregation of the individual slab
   stats.
 - online config editing.
 - more formatting. shorteners for large numbers. bytes - K - M -
   G/etc.
 - better docs, more fleshed out config loader.
 - scrolling output modes.
 - multi-cluster support (switch views between groups of servers)
 - rolling averages for some views.
 - latency monitor (testing a bunch of commands)
 - YAML output/input modes for logging, output into monitoring/graphing
   systems, input into multiple 'damemtop' listeners.
 - pretty colors.
 - reorganize code a little. It got messier than I like :/

 Dunno... stuff? Maybe a quickie mode that can give you warnings or notes
 about your configuration based on current stats? I'll work on this for a
 few hours each week and kick out a new version for a month or so. I don't
 expect (nor want) it to reach the complexity of something like innotop.

 The intent for this module to replace the 'scripts/memcached-tool'
 program, and be distributed with memcached itself.

 have fun,
 -Dormando


Issue 104 in memcached: stats bug for cmd_get

2009-10-29 Thread memcached



Comment #1 on issue 104 by nerdynick: stats bug for cmd_get
http://code.google.com/p/memcached/issues/detail?id=104

I can confirm the reproduction of this with 1.4.2 and Ubuntu 8.04. I tried  
this also

with 1 thread and 4 threads. Producing the same results.

Sample data was 99 inserts 149 gets.
Producing 99 hits and 50 misses.

STAT pid 1773
STAT uptime 21
STAT time 1256828076
STAT version 1.4.2
STAT pointer_size 32
STAT rusage_user 0.004000
STAT rusage_system 0.012000
STAT curr_connections 4
STAT total_connections 7
STAT connection_structures 5
STAT cmd_get 99
STAT cmd_set 99
STAT cmd_flush 0
STAT get_hits 99
STAT get_misses 50
STAT delete_misses 0
STAT delete_hits 0
STAT incr_misses 0
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT bytes_read 172744
STAT bytes_written 171700
STAT limit_maxbytes 16777216
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 1
STAT conn_yields 0
STAT bytes 173628
STAT curr_items 99
STAT total_items 99
STAT evictions 0

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


Issue 57 in memcached: Second biggest slabclass is always less than half the biggest

2009-10-29 Thread memcached



Comment #3 on issue 57 by dsallings: Second biggest slabclass is always  
less than half the biggest

http://code.google.com/p/memcached/issues/detail?id=57

I noticed that last night.  Probably should've updated the bug to avoid  
having you

waste time on it.  :/

I think it's a bug in the test, though.  We'll get it fixed before the  
weekend.


--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


Issue 56 in memcached: Minimum slab size is particular for growth factor of 2

2009-10-29 Thread memcached


Updates:
Status: Fixed

Comment #3 on issue 56 by dsallings: Minimum slab size is particular for  
growth factor of 2

http://code.google.com/p/memcached/issues/detail?id=56

I think you may have been looking at something including 57.  No tests fail  
with this
(just ran it locally and through buildbot).  I'm going to go ahead and push  
it.


--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


Issue 69 in memcached: DTrace probes only fire on deletes which delete things, not those that don't

2009-10-29 Thread memcached


Updates:
Owner: trond.norbye

Comment #1 on issue 69 by dsallings: DTrace probes only fire on deletes  
which delete things, not those that don't

http://code.google.com/p/memcached/issues/detail?id=69

(No comment was entered for this change.)

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


Issue 3 in memcached: Textual delete command w/ expire time noreply in binary protocol version

2009-10-29 Thread memcached


Updates:
Status: Fixed

Comment #7 on issue 3 by trond.norbye: Textual delete command w/ expire  
time  noreply in binary protocol version

http://code.google.com/p/memcached/issues/detail?id=3

(No comment was entered for this change.)

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


Issue 69 in memcached: DTrace probes only fire on deletes which delete things, not those that don't

2009-10-29 Thread memcached


Updates:
Status: Invalid

Comment #2 on issue 69 by trond.norbye: DTrace probes only fire on deletes  
which delete things, not those that don't

http://code.google.com/p/memcached/issues/detail?id=69

The probe is only defined to trigger upon success... if you need to figure  
out if it

is a cache hit / miss you need to use the process-command-start  and end..

We could create a new set of probes to be more specific, but that should be  
filed as

an RFE...

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


Issue 104 in memcached: stats bug for cmd_get

2009-10-29 Thread memcached


Updates:
Status: Started
Owner: trond.norbye

Comment #2 on issue 104 by trond.norbye: stats bug for cmd_get
http://code.google.com/p/memcached/issues/detail?id=104

Patch at: http://github.com/trondn/memcached/tree/issue_104

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


Re: Issue 104 in memcached: stats bug for cmd_get

2009-10-29 Thread memcached


Updates:
Status: Fixed

Comment #3 on issue 104 by dsallings: stats bug for cmd_get
http://code.google.com/p/memcached/issues/detail?id=104

Thanks, Trond.

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


Re: Issue 90 in memcached: RFE: generic config parser

2009-10-29 Thread memcached


Updates:
Status: Fixed

Comment #3 on issue 90 by trond.norbye: RFE: generic config parser
http://code.google.com/p/memcached/issues/detail?id=90

Pushed to the engine branch.

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings


Last call for 1.4.3 (and apologies for the spam)

2009-10-29 Thread dormando

Hey,

We're doing a bunch of bug fixin' and issue killin' in advance of an
early release of 1.4.3-rc1, which is scheduled for this saturday, october
31st (ooh spooky!). 1.4.3-final will be out a week after that, possibly
earlier.

Apologies for the issue spam, please bear with us for two more days while
we empty the queue :)

If you have any bugs, requests, etc, now's the time to speak up!

-Dormando



Re: Issue 57 in memcached: Second biggest slabclass is always less than half the biggest

2009-10-29 Thread memcached


Updates:
Status: Fixed

Comment #4 on issue 57 by dorma...@rydia.net: Second biggest slabclass is  
always less than half the biggest

http://code.google.com/p/memcached/issues/detail?id=57

Fixed the test and pushed this + test fix into my for_143 branch.

--
You received this message because you are listed in the owner
or CC fields of this issue, or because you starred this issue.
You may adjust your issue notification preferences at:
http://code.google.com/hosting/settings