Thanks for the answer, the argument about trying to cleanly separate slabs and items makes sense. (Though yes, I would have brought up exactly those asserts you mentioned, thanks for clarifying. :P)
In fact, if you want to treat the items as raw chunks of memory (with no introspection), you can just assume they are larger than the size of a pointer, you then cast them to void**'s and use them as forwarding pointers. Just need to be sure that you're the only one messing with their memory as long as they're on the free list. But no, I have idea the pain/cost associated with the dynamic allocation. I have no evidence to support it might be a problem with anything (fragmentation, memory bloat, cost of growing it, etc). It was more something I stumbled on while trying to grok the codebase, and wanted to make sure I wasn't missing anything. Does the codebase have any loadtest frameworks that would make such a change easier to evaluate? Thanks, Mike On Fri, Dec 26, 2008 at 06:30, Anatoly Vorobey <[email protected]> wrote: > On Fri, Dec 26, 2008 at 11:02 AM, Mike Lambert <[email protected]> wrote: >> >> Hey all, >> >> We use memcached in a big way here, so I started digging into the >> memcached codebase to poke around and see how things work. One >> question I had, is why free items are stored in an array of pointers >> instead of using the linked list items inherent in the items >> themselves. >> >> When items are alloc'd, the next and prev pointers are cleared. When >> they are free'd, only the ->slabs_clsid is cleared as an "indicator" >> before passing it to slabs_free. slabs_free then stores it in p- >> >slots, an array of pointers to free items. Why aren't these instead >> using the ->next pointers of the items to create a singly linked list. >> This would save on the need for dynamic memory allocation in the p- >> >slots array. >> >> The only downside I can see is that computing slab stats (used_chunks >> and free_chunks) suddenly becomes an expensive operation. Is the stats >> computation efficiency the sole reason for this use of a p-slots >> array, or am I missing something? I understand the value of these >> useful memcache stats, and can appreciate making this tradeoff, I'm >> more curious if these stats were the *sole* reason for using an array, >> or if there's something more subtle (or obvious) that I'm missing. > > Hey Mike, > I don't think anyone cared much about the efficiency of slab stats at the > time. > The original reason was probably that we didn't want the slabs allocator to > require > anything particular of the bits it allocates. Even though we ended up using > it only for > item structs, it was planned to at least try using it for malloc'd things > like connection > structs and other bits and ends. > You'll say, but slabs.c checks slabs_clsid==0 in the item struct all the > item in asserts! > And the poorly known slabs_reassign modifies it, too! And you'll be > completely right, > but neither of these were in the first few versions, and neither is crucial > to how > the allocator works. > It seems quite possible to change the allocator to reuse the pointers inside > the item struct. > It might be a good idea to estimate the headache due to the currently > dynamic allocation > of p->slots first, however. Do you know that it ends up costing a lot of > memory in your usage? > > -- > Anatoly Vorobey, [email protected] > http://avva.livejournal.com (Russian) > http://www.lovestwell.org (English) > >
