Hi, all. On Wed, Aug 4, 2010 at 7:10 AM, Robert Collins <[email protected]> wrote: > On Wed, Aug 4, 2010 at 2:46 PM, Robert Collins > <[email protected]> wrote: >> So, we've got a partial deployment of memcache in place, and while it >> may be contentious, I think it has distracted us from solving root >> cause problems. > ..> So, I propose to turn memcached off on Friday, unless I'm lynched >> before then, which I kindof expect :) > > For clarity, I'm proposing removing all the uses of it that users are > noticing (and thus complaining about) - not removing the facility from > the datacentre: memcached is a good tool and its good that we have it > available. >
Rob and I chatted on IRC about this thread and my concerns. I'll just record them here for the sake of people watching this thread but not IRC. In the end, I'm happy with Rob's final proposal above -- that we should remove unnecessary or bad uses of memcached and that were not focusing on wide spread use until we fix root causes of timeout issues. The facility to use memcached would remain in place. I originally read this as "rip out memcached completely" which had me concerned. I understand Rob is not pushing for that now. I do, however, hate to see us move away from memcached completely. It's a great tool when used properly, and like feature flags, we need to be using it to gain experience, facility, and expertise. I took some issue with the hot bugs list being held up as the poster child for bad memcached usage and the assumption that the use of caching there was to fix timeout issues. Go look at bug 590992. Adding in caching was done only after the initial timeout issue was fixed, and this was done during the great DB fire post-10.06 rollout, and only as a means to buy some overhead when high concurrency or high load raised it's ugly head again. I was never happy that the data was stale as bugs in the list were worked on, but my assumption, based on bug comments I think, was that we were still refining the memcached mechanism and that some form of cache invalidation was being worked on, which is why I didn't back out the caching there yet. Certainly, I'm fine removing the caching of this list if there's nothing being worked on to make invalidating cached fragments possible. IMHO, we can refine our use of memcached while also developing a shared understanding of how best to fix timeouts and performance issues without discounting or removing all the work that has gone on before. Also, I really appreciate Foundations providing memcached and know it will be a great benefit when we get its use correct. Cheers, deryck -- Deryck Hodge https://launchpad.net/~deryck http://www.devurandom.org/ _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
