On 04/21/2015 09:53 AM, Michael Paquier wrote:
On Thu, Apr 16, 2015 at 8:57 PM, Heikki Linnakangas wrote:
Oh, hang on, that's not necessarily true. On promotion, the standby
archives
the last, partial WAL segment from the old timeline. That's just wrong
(http://www.postgresql.org/message-id/52fcd37c.3070...@vmware.com), and in
fact I somehow thought I changed that already, but apparently not. So
let's
stop doing that.
Er. Are you planning to prevent the standby from archiving the last partial
segment from the old timeline at promotion?
Yes.
I thought from previous discussions that we should do it as master
(be it crashed, burned, burried or dead) may not have the occasion to
do it. By preventing its archiving you close the door to the case
where master did not have the occasion to archive it.
The current situation is a mess:
1. Even though we archive the last segment in the standby, there is no
guarantee that the master had archived all the previous segments already.
2. If the master is not totally dead, it might try to archive the same
file with more WAL in it, at the same time or just afterwards, or even
just before the standby has completed promotion. Which copy do you keep
in the archive? Having to deal with that makes the archive_command more
complicated.
Note that even though we don't archive the partial last segment on the
previous timeline, the same WAL is copied to the first segment on the
new timeline. So the WAL isn't lost.
People may be surprised that a base backup taken from a node that has
archive_mode = on set (that's the case in a very large number of cases)
will not be able to work as-is as node startup will fail as follows:
FATAL: archive_mode='on' cannot be used in archive recovery
HINT: Use 'shared' or 'always' mode instead.
Hmm, good point.
One idea would be to simply ignore the fact that archive_mode = on on nodes
in recovery instead of dropping an error. Note that I like the fact that it
drops an error as that's clear, I just point the fact that people may be
surprised that base backups are not working anymore now in this case.
By "ignore", what behaviour do you mean? Would "on" be equivalent to
"shared", "always", or something else?
Or we could keep the current behaviour with archive_mode=on (except for
the last segment thing, which is just wrong), where the standby only
archives the new timeline, and nothing from the previous timelines. Are
the use cases where you'd want that, rather than the new "shared" mode?
I wanted to keep the 'on' mode for backwards-compatibility, but if that
causes more problems, it might be better to just remove it and force the
admin to choose what kind of a setup he has, with "shared" or "always".
Creating a dependency between the pgstat machinery and the WAL sender looks
weak to me. For example with this patch a master cannot stop, as it waits
indefinitely:
LOG: using stale statistics instead of current ones because stats
collector is not responding
LOG: sending archival report:
Hmm, yeah, having walsender to wait for the stats file to appear is not
good.
You could scan archive_status/ but that would be costly if there are many
entries to scan and I think that walsender should be highly responsive. Or
you could directly store the name of the lastly archived WAL segment marked
as .done in let's say archive_status/last_archived. An entry for that in
the control file does not seem the right place as a node may not have
archive_mode enabled that's why I am not mentioning it.
The ways that the archiver process can communicate with the rest of the
system are limited, for the sake of robustness. Writing to the control
file is definitely not OK. I think using the stats collector is OK for
this, but we'll have to arrange it so that the walsender doesn't block
on it, and should probably not force new stat file so often. A 5-10
seconds old stats file would be perfectly fine for this purpose.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
