On Fri, Oct 17, 2014 at 2:23 PM, Fujii Masao <masao.fu...@gmail.com> wrote:
> I found one problem in the 0002 patch. The patch changes the recovery so
> that
> it creates .done files for every WAL files which exist in pg_xlog
> directory at
> the end of recovery. But even WAL files which will have to be archived
> later
> can exist in pg_xlog at that moment. For example, the latest, recycled and
> fully-written-but-not-archived-yet (i.e., maybe having .ready files) WAL
> files.
> The patch wrongly prevents them from being archived at all.
>
Re-looking at patch 2, yes you are right. Even if it was mentioned that we
should do that for a node that had promotion triggered it was not done this
way as a check on CheckForStandbyTrigger() is actually missing. Attached is
an updated patch.
> ISTM that the 0001 patch has the similar problem. Please imagine the
> following
> scenario.
> In this case, the patch seems to make the restartpoint recycle even WAL
> files
> which have .ready files and will have to be archived later. Thought?
>
Right, that's really backward. This was the approach taken before c9cc7e0,
and this commit actually prevents removal of unarchived WAL files during
recovery.
--
Michael
From fdf1f4a46a2f870ee2aeda11f909c465ed12450a Mon Sep 17 00:00:00 2001
From: Michael Paquier <mich...@otacoo.com>
Date: Wed, 22 Oct 2014 15:18:33 +0200
Subject: [PATCH] Enforce all WAL segment files to be marked as .done at node
promotion
This is a safety mechanism to ensure that there are no files that are not
considered as .done as some segments may have been missed particularly in
the case of partially written files or files being written when a disconnection
occurred between a streaming standby and its root node. This makes the node
reaching promotion having a state consistent with what is expected using
the assumption that all the WAL segment files that are done being streamed
should be always considered as archived by the node.
---
src/backend/access/transam/xlog.c | 10 ++++++++
src/backend/access/transam/xlogarchive.c | 39 +++++++++++++++++++++++++++++++-
src/include/access/xlog_internal.h | 1 +
3 files changed, 49 insertions(+), 1 deletion(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 235b442..7179003 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6862,6 +6862,16 @@ StartupXLOG(void)
}
/*
+ * Create a .done entry for each WAL file present in pg_xlog that has
+ * not been yet marked as such for a node that has been promoted. In some
+ * cases where for example a streaming replica has had a connection to a
+ * remote node cut abruptly, such WAL files may have been only partially
+ * written or even not flagged correctly with .done.
+ */
+ if (InRecovery && CheckForStandbyTrigger())
+ XLogArchiveForceDoneAll();
+
+ /*
* Kill WAL receiver, if it's still running, before we continue to write
* the startup checkpoint record. It will trump over the checkpoint and
* subsequent records if it's still alive when we start writing WAL.
diff --git a/src/backend/access/transam/xlogarchive.c b/src/backend/access/transam/xlogarchive.c
index 047efa2..931106f 100644
--- a/src/backend/access/transam/xlogarchive.c
+++ b/src/backend/access/transam/xlogarchive.c
@@ -554,6 +554,40 @@ XLogArchiveNotifySeg(XLogSegNo segno)
}
/*
+ * XLogArchiveForceDoneAll
+ *
+ * Wrapper of XLogArchiveForceDone scanning all the XLOG segments files in
+ * XLOGDIR, switching them forcibly to <XLOG>.done.
+ */
+void
+XLogArchiveForceDoneAll(void)
+{
+ DIR *xldir;
+ struct dirent *xlde;
+
+
+ xldir = AllocateDir(XLOGDIR);
+ if (xldir == NULL)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not open transaction log directory \"%s\": %m",
+ XLOGDIR)));
+
+ /*
+ * Scan all the WAL segments present in the archives and switch them to
+ * .done.
+ */
+ while ((xlde = ReadDir(xldir, XLOGDIR)) != NULL)
+ {
+ if (strlen(xlde->d_name) == 24 &&
+ strspn(xlde->d_name, "0123456789ABCDEF") == 24)
+ XLogArchiveForceDone(xlde->d_name);
+ }
+
+ FreeDir(xldir);
+}
+
+/*
* XLogArchiveForceDone
*
* Emit notification forcibly that an XLOG segment file has been successfully
@@ -582,7 +616,6 @@ XLogArchiveForceDone(const char *xlog)
(errcode_for_file_access(),
errmsg("could not rename file \"%s\" to \"%s\": %m",
archiveReady, archiveDone)));
-
return;
}
@@ -604,6 +637,10 @@ XLogArchiveForceDone(const char *xlog)
archiveDone)));
return;
}
+
+ ereport(DEBUG2,
+ (errmsg("archive status file of \"%s\" has been forcibly switched from \"%s\" to \"%s\"",
+ xlog, archiveReady, archiveDone)));
}
/*
diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h
index 27b9899..04a0de3 100644
--- a/src/include/access/xlog_internal.h
+++ b/src/include/access/xlog_internal.h
@@ -286,6 +286,7 @@ extern void ExecuteRecoveryCommand(char *command, char *commandName,
extern void KeepFileRestoredFromArchive(char *path, char *xlogfname);
extern void XLogArchiveNotify(const char *xlog);
extern void XLogArchiveNotifySeg(XLogSegNo segno);
+extern void XLogArchiveForceDoneAll(void);
extern void XLogArchiveForceDone(const char *xlog);
extern bool XLogArchiveCheckDone(const char *xlog);
extern bool XLogArchiveIsBusy(const char *xlog);
--
2.1.2
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
