Tom Lane wrote:
"Matthew T. O'Connor" <matthew@zeut.net> writes:
It's not clear to me why a worker cares that there is a new worker, since the new worker is going to ignore all the tables that are already claimed by all worker todo lists.


That seems wrong to me, since it means that new workers will ignore
tables that are scheduled for processing by an existing worker, no
matter how far in the future that schedule extends.  As an example,
suppose you have half a dozen large tables in need of vacuuming.
The first worker in will queue them all up, and subsequent workers
will do nothing useful, at least not till the first worker is done
with the first table.  Having the first worker update its todo
list file after each table allows the earlier tables to be exposed
for reconsideration, but that's expensive and it does nothing for
later tables.

Well the big problem that we have is not that large tables are being starved, so this doesn't bother me too much, plus there is only so much IO, so one worker working sequentially through the big tables seems OK to me.

I suggest that maybe we don't need exposed TODO lists at all.  Rather
the workers could have internal TODO lists that are priority-sorted
in some way, and expose only their current table OID in shared memory.
Then the algorithm for processing each table in your list is

        1. Grab the AutovacSchedule LWLock exclusively.
        2. Check to see if another worker is currently processing
           that table; if so drop LWLock and go to next list entry.
        3. Recompute whether table needs vacuuming; if not,
           drop LWLock and go to next entry.  (This test covers the
           case where someone vacuumed the table since you made your
           list.)
        4. Put table OID into shared memory, drop LWLock, then
           vacuum table.
        5. Clear current-table OID from shared memory, then
           repeat for next list entry.

This creates a behavior of "whoever gets to it first" rather than
allowing workers to claim tables that they actually won't be able
to service any time soon.

Right, but you could wind up with as many workers working concurrently as you have tables in a database which doesn't seem like a good idea either. One thing I like about the todo list setup Alvaro had is that new workers will be assigned fewer tables to work on and hence exit sooner. We are going to fire off a new worker every autovac_naptime so availability of new workers isn't going to be a problem.


---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
      subscribe-nomail command to [EMAIL PROTECTED] so that your
      message can get through to the mailing list cleanly

Reply via email to