On Sat, Jun 8, 2013 at 7:20 PM, Jeff Janes <jeff.ja...@gmail.com> wrote: > If archiving is on and failure is due to no space, could we just keep trying > XLogFileInit again for a couple minutes to give archiving a chance to do its > things? Doing that while holding onto locks and a critical section would be > unfortunate, but if the alternative is a PANIC, it might be acceptable.
Blech. I think that's setting our standards pretty low. It would neither be possible to use the system nor to shut it down cleanly; I think the effect would be to turn an immediate PANIC into a slightly-delayed PANIC, possibly accompanied by some DBA panic. It seems to me that there are two general ways of approaching this problem. 1. Discover sooner that we're out of space. Once we've modified the buffer and entered the critical section, it's too late to have second thoughts about completing the operation. If we could guarantee prior to modifying the buffers that enough WAL space was present to store the record we're about to write, then we'd be certain not to fail for this reason. In theory, this is simple: keep track of how much uncommitted WAL space we have. Increment the value when we create new WAL segments and decrement it by the size of the WAL record we plan to write. In practice, it's not so simple. We don't know whether we're going to emit FPIs until after we enter the critical section, so the size of the record can't be known precisely early enough. We could think about estimating the space needed conservatively and truing it up occasionally. However, there's a second problem: the available-WAL-space counter would surely become a contention point. Here's a sketch of a possible solution. Suppose we know that an individual WAL record can't be larger than, uh, 64kB. I'm not sure there is a limit on the size of a WAL record, but let's say there is, or we can install one, at around that size limit. Before we enter a critical section that's going to write a WAL record, we verify that the amount of WAL space remaining is at least 64kB * MaxBackends. If it's not, we embark on a series of short sleeps, rechecking after each one; if we hit some time limit, we ERROR out. As long as every backend checks this before every WAL record, we can always be sure there will be at least 64kB left for us. With this approach, the shared variable that stores the amount of WAL space remaining only needs to be updated under WALInsertLock; the "reservation" step only involves a read. That might be cheap enough not to matter. 2. Recover from the fact that we ran out of space by backing out the changes to shared buffers. Initially, I thought this might be a promising approach: if we've modified any shared buffers and discover that we can't log the changes, just invalidate the buffers! Of course, it doesn't work, because the buffer might have have already been dirty when we locked it. So we'd actually need a way to reverse out all the changes we were about to log. That's probably too expensive to contemplate, in general; and the code would likely get so little testing as to invite bugs. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
