On Sat, 25 Aug 2001, Ryan Bloom wrote: > Woah! Those patches were large, and I don't think I agree with them > at all.
The fact that it's a big patch is why I've put it off this long. I had that first phase written months ago and never committed it because it was large. Note that much of that first patch was stuff that would have come right back out again in phase 2... some temporary hacks were necessary as while I made the basic changes and before changing the API so I could have something I could test for breakage at several points along the way. This is the approach we've been talking about doing for months and nobody objected before as far as I can remember. I posted this three-phase-plan thingy on the list on July 8 and nobody voiced any concerns. It just hasn't got done because no one ever got around to it. Phase 1 seemed harmless enough (big though it was)... but now that I'm working on Phase 2, the more I see how greatly it affects the API, the less I like it, so I'm willing to change. > Plus, now we sometimes have both pools and sms's in the > same structure. How are we synch'ing those two? We're not. It's annoying, I know, but we're just counting on the code not to clean up the data before while we're still using the buckets. But we already do that, so that's not a change in the requirements. Anyhow, the idea is that the SMS used will live as long as the thread (which is certainly at least as long as the data), and the caller just keeps giving you back a pointer to that SMS so you don't have to look up your thread ID all the time. > Why can't we just create a free list for buckets? That way, we take > the hit for allocating buckets only when necessary, and at some point, > we will reach a steady state. The free list, could be per thread, and > I believe I could have it coded tomorrow night. That's what we were going to do originally when you and Greg and I and the others started talking about this, what, 7 or 8 months ago? But then when the SMS's came around they seemed ideal because we could just write ourselves up a blocks SMS that gave us our free list tailored the way we wanted it and pop it in. The SMS is intended to be per-thread and last as long as the thread though extra allocated memory might be cleaned up now and then to avoid leaks, but because we get to choose when that cleanup occurs, we can still reach our steady state. As a fringe benefit, it has been indicated on-list that some custom bucket writers might want to use custom memory allocation systems, and this takes care of that without us having to do anything special at all. (Assume for the purposes of this paragraph that we ARE going to use SMS.) It was debated whether the SMS to be used should be looked up by the buckets code itself or whether it should be passed in by the caller. Passing it in by the caller is what eventually came out seeming the best because otherwise the buckets code (which has no way to know what its thread ID is) has to go ask the OS for its thread ID and then look up the SMS in a hash table or something based on that ID. If the caller gives it to us, the caller (which DOES know the thread ID at some point) can just stash a pointer to that thread's SMS somewhere and pass it in to the bucket functions and it's constant time instead of pseudo-constant or logarithmic or linear or whatever. On the other hand, now that I'm actually implementing that as phase 2, I sort of dislike how cluttered it makes the API seem. So we could take a fork in the road from here and leave it up to the buckets to find their thread ID. Less performant, but not terribly bad. Probably still better than malloc, though the win is less clear. Or we could keep going. (Assume for the purposes of this paragraph that we're NOT going to use SMS, either because the group consensus is "SMS were a nice try, but they couldn't beat pools, so it's time to get rid of them altogether" or because we'd just rather do something custom here.) The same thread ID problem exists for a buckets-specific-non-SMS free list, so you'd end up doing the call to find out what your thread ID is and then do your hash lookup because the caller can't help you anymore. So that was the logic behind it. Thoughts? --Cliff -------------------------------------------------------------- Cliff Woolley [EMAIL PROTECTED] Charlottesville, VA
