Re: Design challenges in chunkd self-checking

2010-01-05 Thread Pete Zaitcev
On Tue, 22 Dec 2009 22:36:16 -0500 Jeff Garzik j...@garzik.org wrote: Seems like a mutex-wrapped GLib hash table would work... I dunno about this... See, I think it's like kernel timers: there's a lot of premium on having add and remove quick, and the rest is whatever. The important part is not

Re: Design challenges in chunkd self-checking

2010-01-05 Thread Pete Zaitcev
On Tue, 05 Jan 2010 16:02:58 -0500 Jeff Garzik j...@garzik.org wrote: On 01/05/2010 03:47 PM, Pete Zaitcev wrote: On Tue, 22 Dec 2009 22:36:16 -0500 Jeff Garzikj...@garzik.org wrote: Seems like a mutex-wrapped GLib hash table would work... I dunno about this... See, I think it's like

Re: Design challenges in chunkd self-checking

2010-01-05 Thread Pete Zaitcev
On Tue, 05 Jan 2010 16:53:55 -0500 Jeff Garzik j...@garzik.org wrote: If you have a constant pointer value [for the lifetime of the hash table entry], use g_direct_hash. If you have a nul-terminated string, GLib also has g_str_hash. Of course I considered these, but thanks to our keys

Design challenges in chunkd self-checking

2009-12-22 Thread Pete Zaitcev
I'm looking into adding self-checking to chunkd. This involves basically a process that re-reads everything stored in the chunkserver and verifies that it's still ok. Nothing can be simpler, right? So, current problems for which I'd like input are: - Scheduling and deconflicting with normal

Re: Design challenges in chunkd self-checking

2009-12-22 Thread Jeff Garzik
On 12/22/2009 04:41 PM, Pete Zaitcev wrote: I'm looking into adding self-checking to chunkd. This involves basically a process that re-reads everything stored in the chunkserver and verifies that it's still ok. Nothing can be simpler, right? So, current problems for which I'd like input are:

Re: Design challenges in chunkd self-checking

2009-12-22 Thread Pete Zaitcev
On Tue, 22 Dec 2009 17:43:58 -0500 Jeff Garzik j...@garzik.org wrote: It is normal and reasonable to maintain global information about all in-progress operations. Caching systems do that, for example, to ensure multiple cache requests for object A do not initiate multiple simultaneous

Re: Design challenges in chunkd self-checking

2009-12-22 Thread Jeff Garzik
On 12/22/2009 08:40 PM, Pete Zaitcev wrote: On Tue, 22 Dec 2009 17:43:58 -0500 Jeff Garzikj...@garzik.org wrote: It is normal and reasonable to maintain global information about all in-progress operations. Caching systems do that, for example, to ensure multiple cache requests for object A