Hey, 'memslap' in the libmemcached library can be used to help load test. There're no standard set of test though.
-Dormando On Fri, 26 Dec 2008, Mike Lambert wrote: > > Thanks for the answer, the argument about trying to cleanly separate > slabs and items makes sense. (Though yes, I would have brought up > exactly those asserts you mentioned, thanks for clarifying. :P) > > In fact, if you want to treat the items as raw chunks of memory (with > no introspection), you can just assume they are larger than the size > of a pointer, you then cast them to void**'s and use them as > forwarding pointers. Just need to be sure that you're the only one > messing with their memory as long as they're on the free list. > > But no, I have idea the pain/cost associated with the dynamic > allocation. I have no evidence to support it might be a problem with > anything (fragmentation, memory bloat, cost of growing it, etc). It > was more something I stumbled on while trying to grok the codebase, > and wanted to make sure I wasn't missing anything. Does the codebase > have any loadtest frameworks that would make such a change easier to > evaluate? > > Thanks, > Mike > > On Fri, Dec 26, 2008 at 06:30, Anatoly Vorobey <[email protected]> wrote: > > On Fri, Dec 26, 2008 at 11:02 AM, Mike Lambert <[email protected]> wrote: > >> > >> Hey all, > >> > >> We use memcached in a big way here, so I started digging into the > >> memcached codebase to poke around and see how things work. One > >> question I had, is why free items are stored in an array of pointers > >> instead of using the linked list items inherent in the items > >> themselves. > >> > >> When items are alloc'd, the next and prev pointers are cleared. When > >> they are free'd, only the ->slabs_clsid is cleared as an "indicator" > >> before passing it to slabs_free. slabs_free then stores it in p- > >> >slots, an array of pointers to free items. Why aren't these instead > >> using the ->next pointers of the items to create a singly linked list. > >> This would save on the need for dynamic memory allocation in the p- > >> >slots array. > >> > >> The only downside I can see is that computing slab stats (used_chunks > >> and free_chunks) suddenly becomes an expensive operation. Is the stats > >> computation efficiency the sole reason for this use of a p-slots > >> array, or am I missing something? I understand the value of these > >> useful memcache stats, and can appreciate making this tradeoff, I'm > >> more curious if these stats were the *sole* reason for using an array, > >> or if there's something more subtle (or obvious) that I'm missing. > > > > Hey Mike, > > I don't think anyone cared much about the efficiency of slab stats at the > > time. > > The original reason was probably that we didn't want the slabs allocator to > > require > > anything particular of the bits it allocates. Even though we ended up using > > it only for > > item structs, it was planned to at least try using it for malloc'd things > > like connection > > structs and other bits and ends. > > You'll say, but slabs.c checks slabs_clsid==0 in the item struct all the > > item in asserts! > > And the poorly known slabs_reassign modifies it, too! And you'll be > > completely right, > > but neither of these were in the first few versions, and neither is crucial > > to how > > the allocator works. > > It seems quite possible to change the allocator to reuse the pointers inside > > the item struct. > > It might be a good idea to estimate the headache due to the currently > > dynamic allocation > > of p->slots first, however. Do you know that it ends up costing a lot of > > memory in your usage? > > > > -- > > Anatoly Vorobey, [email protected] > > http://avva.livejournal.com (Russian) > > http://www.lovestwell.org (English) > > > > >
