Philip Martin wrote on Thu, Jul 07, 2011 at 08:36:06 +0100: > Daniel Shahaf <danie...@elego.de> writes: > > > The process to edit a revprop that had been packed would be: > > > > * grab write lock > > * prepare a new pack-file-with-inline-manifest > > * move-into-place, atomically replacing the previous pack file > > * ungrab write lock > > > > That is what guarantees cp(1) consistency that Hyrum mentions. > > Atomic replace is going to involve a retry loop on Windows, and so could > potentially take a long time. Virus scanners on repository disks? >
Yes, this is going to be a problem. > Holding the write lock will block readers so for better concurrency we > want to minimise the time that a write lock is held, or have more locks. > > Is there one lock per pack file or one lock per repository? Having > multiple locks means that a write only blocks a proportion of the reads, > but also means an operation like log has to acquire multiple locks (in > series not in parallel). > It will block writers, not readers. But having more fine-grained locks seems a good idea. > We can move the prepare/write outside the write lock by using a sequence > number in the pack file. Then the algorithm could be: > > * prepare a new pack file with incremented sequence number > * grab write lock > * check old sequence number > * if changed: drop lock and start again > * otherwise: atomic replace pack file > * drop write lock > > That doesn't help with the retry problem on Windows, but may reduce the > time the write lock is held, particularly if the pack file is large. It > shoud work well if writes are much less common than reads. If writes > are common then the simple write lock serialisation may be better. > Good idea. > The two algorithms should interoperate so, even if we use a simple write > lock today, implementing the sequence number would give us options in > the future.
