Re: [PATCHES] Archiver restart timer

2004-12-02 Thread Bruce Momjian

Patch applied.  Thanks.

---


Simon Riggs wrote:
> On Sun, 2004-11-21 at 23:07, Simon Riggs wrote:
> > I enclose a short patch to reduce the PGARCH_RESTART_INTERVAL from 60
> > seconds to 10 seconds. The original number was plucked from thin air
> > some months ago, and I'd like to review that now based upon further
> > thought, observation and experience.
> > 
> > This change has little or no effect on performance, since the interval
> > is there mainly to avoid repeated respawn attempts if archiver fails at
> > startup. Archiver start-up time is very quick, so there is little danger
> > of exceeding 10 seconds.
> > 
> > On a busy system, if the archiver does die, then many files can build up
> > in the 60 seconds before respawning. That xlog file backlog could take
> > some time to clear. This then leaves a larger than normal window of data
> > loss for a possibly long period.
> > 
> > It's a minor change only, with no other effect on function.
> 
> ...and now for the diff -c version ;-)
> 
> -- 
> Best Regards, Simon Riggs

[ Attachment, skipping... ]

> 
> ---(end of broadcast)---
> TIP 9: the planner will ignore your desire to choose an index scan if your
>   joining column's datatypes do not match

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  [EMAIL PROTECTED]   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 5: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faqs/FAQ.html


Re: [PATCHES] Archiver restart timer

2004-11-21 Thread Simon Riggs
On Sun, 2004-11-21 at 23:07, Simon Riggs wrote:
> I enclose a short patch to reduce the PGARCH_RESTART_INTERVAL from 60
> seconds to 10 seconds. The original number was plucked from thin air
> some months ago, and I'd like to review that now based upon further
> thought, observation and experience.
> 
> This change has little or no effect on performance, since the interval
> is there mainly to avoid repeated respawn attempts if archiver fails at
> startup. Archiver start-up time is very quick, so there is little danger
> of exceeding 10 seconds.
> 
> On a busy system, if the archiver does die, then many files can build up
> in the 60 seconds before respawning. That xlog file backlog could take
> some time to clear. This then leaves a larger than normal window of data
> loss for a possibly long period.
> 
> It's a minor change only, with no other effect on function.

...and now for the diff -c version ;-)

-- 
Best Regards, Simon Riggs
Index: pgarch.c
===
RCS file: /projects/cvsroot/pgsql/src/backend/postmaster/pgarch.c,v
retrieving revision 1.10
diff -d -c -r1.10 pgarch.c
*** pgarch.c	9 Nov 2004 13:01:26 -	1.10
--- pgarch.c	21 Nov 2004 22:59:03 -
***
*** 51,57 
  #define PGARCH_AUTOWAKE_INTERVAL 60		/* How often to force a poll of
  		 * the archive status directory;
  		 * in seconds. */
! #define PGARCH_RESTART_INTERVAL 60		/* How often to attempt to restart
  		 * a failed archiver; in seconds. */
  
  /* --
--- 51,57 
  #define PGARCH_AUTOWAKE_INTERVAL 60		/* How often to force a poll of
  		 * the archive status directory;
  		 * in seconds. */
! #define PGARCH_RESTART_INTERVAL 10		/* How often to attempt to restart
  		 * a failed archiver; in seconds. */
  
  /* --

---(end of broadcast)---
TIP 9: the planner will ignore your desire to choose an index scan if your
  joining column's datatypes do not match


[PATCHES] Archiver restart timer

2004-11-21 Thread Simon Riggs
I enclose a short patch to reduce the PGARCH_RESTART_INTERVAL from 60
seconds to 10 seconds. The original number was plucked from thin air
some months ago, and I'd like to review that now based upon further
thought, observation and experience.

This change has little or no effect on performance, since the interval
is there mainly to avoid repeated respawn attempts if archiver fails at
startup. Archiver start-up time is very quick, so there is little danger
of exceeding 10 seconds.

On a busy system, if the archiver does die, then many files can build up
in the 60 seconds before respawning. That xlog file backlog could take
some time to clear. This then leaves a larger than normal window of data
loss for a possibly long period.

It's a minor change only, with no other effect on function.

-- 
Best Regards, Simon Riggs
/*-
 *
 * pgarch.c
 *
 *	PostgreSQL WAL archiver
 *
 *	All functions relating to archiver are included here
 *
 *	- All functions executed by archiver process
 *
 *	- archiver is forked from postmaster, and the two
 *	processes then communicate using signals. All functions
 *	executed by postmaster are included in this file.
 *
 *	Initial author: Simon Riggs		[EMAIL PROTECTED]
 *
 * Portions Copyright (c) 1996-2004, PostgreSQL Global Development Group
 * Portions Copyright (c) 1994, Regents of the University of California
 *
 *
 * IDENTIFICATION
 *	  $PostgreSQL: pgsql/src/backend/postmaster/pgarch.c,v 1.10 2004/11/09 13:01:26 petere Exp $
 *
 *-
 */
#include "postgres.h"

#include 
#include 
#include 
#include 
#include 

#include "access/xlog_internal.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "postmaster/pgarch.h"
#include "postmaster/postmaster.h"
#include "storage/fd.h"
#include "storage/ipc.h"
#include "storage/pg_shmem.h"
#include "storage/pmsignal.h"
#include "utils/guc.h"
#include "utils/ps_status.h"


/* --
 * Timer definitions.
 * --
 */
#define PGARCH_AUTOWAKE_INTERVAL 60		/* How often to force a poll of
		 * the archive status directory;
		 * in seconds. */
#define PGARCH_RESTART_INTERVAL 10		/* How often to attempt to restart
		 * a failed archiver; in seconds. */

/* --
 * Archiver control info.
 *
 * We expect that archivable files within pg_xlog will have names between
 * MIN_XFN_CHARS and MAX_XFN_CHARS in length, consisting only of characters
 * appearing in VALID_XFN_CHARS.  The status files in archive_status have
 * corresponding names with ".ready" or ".done" appended.
 * --
 */
#define MIN_XFN_CHARS	16
#define MAX_XFN_CHARS	40
#define VALID_XFN_CHARS "0123456789ABCDEF.history.backup"

#define NUM_ARCHIVE_RETRIES 3


/* --
 * Local data
 * --
 */
static time_t last_pgarch_start_time;

/*
 * Flags set by interrupt handlers for later service in the main loop.
 */
static volatile sig_atomic_t got_SIGHUP = false;
static volatile sig_atomic_t wakened = false;

/* --
 * Local function forward declarations
 * --
 */
#ifdef EXEC_BACKEND
static pid_t pgarch_forkexec(void);
#endif

NON_EXEC_STATIC void PgArchiverMain(int argc, char *argv[]);
static void pgarch_exit(SIGNAL_ARGS);
static void ArchSigHupHandler(SIGNAL_ARGS);
static void pgarch_waken(SIGNAL_ARGS);
static void pgarch_MainLoop(void);
static void pgarch_ArchiverCopyLoop(void);
static bool pgarch_archiveXlog(char *xlog);
static bool pgarch_readyXlog(char *xlog);
static void pgarch_archiveDone(char *xlog);


/* 
 * Public functions called from postmaster follow
 * 
 */

/*
 * pgarch_start
 *
 *	Called from postmaster at startup or after an existing archiver
 *	died.  Attempt to fire up a fresh archiver process.
 *
 *	Returns PID of child process, or 0 if fail.
 *
 *	Note: if fail, we will be called again from the postmaster main loop.
 */
int
pgarch_start(void)
{
	time_t		curtime;
	pid_t		pgArchPid;

	/*
	 * Do nothing if no archiver needed
	 */
	if (!XLogArchivingActive())
		return 0;

	/*
	 * Do nothing if too soon since last archiver start.  This is a safety
	 * valve to protect against continuous respawn attempts if the
	 * archiver is dying immediately at launch. Note that since we will be
	 * re-called from the postmaster main loop, we will get another chance
	 * later.
	 */
	curtime = time(NULL);
	if ((unsigned int) (curtime - last_pgarch_start_time) <
		(unsigned int) PGARCH_RESTART_INTERVAL)
		return 0;
	last_pgarch_start_time = curtime;

	fflush(stdout);
	fflush(stderr);

#ifdef __BEOS__
	/* Specific beos actions before backend startup */
	beos_before_backend_startup();
#endif

#ifdef EXEC_BACKEND
	switch ((pgArchPid = pgarch_forkexec()))
#else
	switch ((pgArchPid = fork()))
#endif
	{
		case -1:
#ifdef __BEOS__
			/* Specific beos actions */
			beos_