On Thu, Jul 14, 2005 at 10:52:56AM -0400, Tom Lane wrote:
> I've applied Alvaro's latest integrated-autovacuum patch. There are
> still a number of loose ends to be dealt with before beta, though:
Ok, here's a patch that deals with some of this:
- The stat collector is modified so as to keep shared relations separate
from regular ones. Also, backends sends messages separately.
Autovacuum takes advantage of this, so it correctly identifies the
appropiate time to operate on a shared relation, irrespective of the
database where they were modified. Note however that it uses each
database's pg_autovacuum settings. This means it could be vacuumed
sooner in one database than another, but I don't think it's a problem.
- Temp tables are completely ignored.
- pg_statistic is completely ignored.
- databases with no stat entry are still ignored, except that they are
checked for Xid wraparound like any other. The "oldest" one is chosen
for vacuum in a particular autovacuum run.
- A database-wide vacuum forces a pg_database flat-file update, so that
the wraparound check actually works.
- The postmaster's main loop sleeps Min(60, autovacuum_naptime), in
order to be able to pick naptimes smaller than 60 seconds. In order
not to make the loop a busy-wait, I forced a minimum of 1 to that GUC
var.
Some comments:
- Now that we have a real Xid wraparound check, we could go back to
having any table with no stat entry ignored, which was the original
coding. There's no danger of wraparound, and there'd be no work done
to a table that doesn't have any activity. We have to consider what
happens at stat reset -- AFAICS there's no problem, because as soon as
the table sees some activity, it will be picked up by pgstat.
However, it would be bad if stats are reset right after some heavy
activity on a table. Maybe the only thing we need is documentation.
- datallowcon is still ignored. Now it's safe to do so, because we have
a real Xid wraparound check. Changing it requires extending the
pg_database flat-file (should be fairly easy).
- There are stat messages emitted for a database-wide vacuum, just like
any other. This means that all tables in the database would end up in
pgstat; and also all databases, including those with datallowconn = false.
This may not be good. I'm not sure what exactly to do about it. Do
we want to disallow such stats? Disable message sending (or
collecting) in some circumstances?
- I haven't done anything yet w.r.t. the custom vacuum_delay nor sleep
scale factor.
- There are still no docs.
--
Alvaro Herrera (<alvherre[a]alvh.no-ip.org>)
"Porque Kim no hacia nada, pero, eso sí,
con extraordinario éxito" ("Kim", Kipling)
Index: src/backend/access/transam/xlog.c
===================================================================
RCS file: /home/alvherre/cvs/pgsql/src/backend/access/transam/xlog.c,v
retrieving revision 1.210
diff -c -r1.210 xlog.c
*** src/backend/access/transam/xlog.c 23 Jul 2005 15:31:16 -0000 1.210
--- src/backend/access/transam/xlog.c 24 Jul 2005 17:55:51 -0000
***************
*** 465,471 ****
TimeLineID endTLI,
uint32 endLogId, uint32 endLogSeg);
static void WriteControlFile(void);
- static void ReadControlFile(void);
static char *str_time(time_t tnow);
static void issue_xlog_fsync(void);
--- 465,470 ----
***************
*** 3383,3390 ****
errmsg("could not close control file: %m")));
}
! static void
! ReadControlFile(void)
{
pg_crc32 crc;
int fd;
--- 3382,3394 ----
errmsg("could not close control file: %m")));
}
! /*
! * Read and verify the control file, filling the ControlFile struct.
! *
! * If nextXid is not NULL, the latest Checkpoint's nextXid is returned.
! */
! void
! ReadControlFile(TransactionId *nextXid)
{
pg_crc32 crc;
int fd;
***************
*** 3525,3530 ****
--- 3529,3537 ----
ControlFile->lc_ctype),
errhint("It looks like you need to initdb or install locale
support.")));
+ if (PointerIsValid(nextXid))
+ *nextXid = ControlFile->checkPointCopy.nextXid;
+
/* Make the fixed locale settings visible as GUC variables, too */
SetConfigOption("lc_collate", ControlFile->lc_collate,
PGC_INTERNAL, PGC_S_OVERRIDE);
***************
*** 3650,3656 ****
* for the reasons why).
*/
if (!IsBootstrapProcessingMode())
! ReadControlFile();
}
/*
--- 3657,3663 ----
* for the reasons why).
*/
if (!IsBootstrapProcessingMode())
! ReadControlFile(NULL);
}
/*
***************
*** 4232,4238 ****
* Note: in most control paths, *ControlFile is already valid and we
need
* not do ReadControlFile() here, but might as well do it to be sure.
*/
! ReadControlFile();
if (ControlFile->logSeg == 0 ||
ControlFile->state < DB_SHUTDOWNED ||
--- 4239,4245 ----
* Note: in most control paths, *ControlFile is already valid and we
need
* not do ReadControlFile() here, but might as well do it to be sure.
*/
! ReadControlFile(NULL);
if (ControlFile->logSeg == 0 ||
ControlFile->state < DB_SHUTDOWNED ||
Index: src/backend/commands/analyze.c
===================================================================
RCS file: /home/alvherre/cvs/pgsql/src/backend/commands/analyze.c,v
retrieving revision 1.87
diff -c -r1.87 analyze.c
*** src/backend/commands/analyze.c 14 Jul 2005 05:13:39 -0000 1.87
--- src/backend/commands/analyze.c 22 Jul 2005 01:08:51 -0000
***************
*** 317,323 ****
* a zero-column table.
*/
if (!vacstmt->vacuum)
! pgstat_report_analyze(RelationGetRelid(onerel), 0, 0);
vac_close_indexes(nindexes, Irel, AccessShareLock);
relation_close(onerel, AccessShareLock);
--- 317,324 ----
* a zero-column table.
*/
if (!vacstmt->vacuum)
! pgstat_report_analyze(RelationGetRelid(onerel),
onerel->rd_rel->relisshared,
! 0, 0);
vac_close_indexes(nindexes, Irel, AccessShareLock);
relation_close(onerel, AccessShareLock);
***************
*** 436,443 ****
}
/* report results to the stats collector, too */
! pgstat_report_analyze(RelationGetRelid(onerel), totalrows,
! totaldeadrows);
}
/* Done with indexes */
--- 437,444 ----
}
/* report results to the stats collector, too */
! pgstat_report_analyze(RelationGetRelid(onerel),
onerel->rd_rel->relisshared,
! totalrows,
totaldeadrows);
}
/* Done with indexes */
Index: src/backend/commands/vacuum.c
===================================================================
RCS file: /home/alvherre/cvs/pgsql/src/backend/commands/vacuum.c,v
retrieving revision 1.311
diff -c -r1.311 vacuum.c
*** src/backend/commands/vacuum.c 14 Jul 2005 05:13:39 -0000 1.311
--- src/backend/commands/vacuum.c 24 Jul 2005 15:34:23 -0000
***************
*** 41,46 ****
--- 41,47 ----
#include "tcop/pquery.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+ #include "utils/flatfiles.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
***************
*** 712,718 ****
* vac_update_dbstats() -- update statistics for one database
*
* Update the whole-database statistics that are kept in its
pg_database
! * row.
*
* We violate no-overwrite semantics here by storing new values
for the
* statistics columns directly into the tuple that's already on
the page.
--- 713,719 ----
* vac_update_dbstats() -- update statistics for one database
*
* Update the whole-database statistics that are kept in its
pg_database
! * row, and the flat-file copy of pg_database.
*
* We violate no-overwrite semantics here by storing new values
for the
* statistics columns directly into the tuple that's already on
the page.
***************
*** 721,728 ****
*
* This routine is shared by full and lazy VACUUM. Note that it
is only
* applied after a database-wide VACUUM operation.
- *
- * Note that we don't bother to update the flat-file copy of
pg_database.
*/
static void
vac_update_dbstats(Oid dbid,
--- 722,727 ----
***************
*** 768,773 ****
--- 767,775 ----
heap_endscan(scan);
heap_close(relation, RowExclusiveLock);
+
+ /* Mark the flat-file for update at commit */
+ database_file_update_needed();
}
***************
*** 1165,1172 ****
vacrelstats->rel_tuples,
vacrelstats->hasindex);
/* report results to the stats collector, too */
! pgstat_report_vacuum(RelationGetRelid(onerel), vacstmt->analyze,
! vacrelstats->rel_tuples);
}
--- 1167,1174 ----
vacrelstats->rel_tuples,
vacrelstats->hasindex);
/* report results to the stats collector, too */
! pgstat_report_vacuum(RelationGetRelid(onerel),
onerel->rd_rel->relisshared,
! vacstmt->analyze,
vacrelstats->rel_tuples);
}
Index: src/backend/commands/vacuumlazy.c
===================================================================
RCS file: /home/alvherre/cvs/pgsql/src/backend/commands/vacuumlazy.c,v
retrieving revision 1.55
diff -c -r1.55 vacuumlazy.c
*** src/backend/commands/vacuumlazy.c 14 Jul 2005 05:13:40 -0000 1.55
--- src/backend/commands/vacuumlazy.c 22 Jul 2005 01:08:51 -0000
***************
*** 182,189 ****
hasindex);
/* report results to the stats collector, too */
! pgstat_report_vacuum(RelationGetRelid(onerel), vacstmt->analyze,
! vacrelstats->rel_tuples);
}
--- 182,189 ----
hasindex);
/* report results to the stats collector, too */
! pgstat_report_vacuum(RelationGetRelid(onerel),
onerel->rd_rel->relisshared,
! vacstmt->analyze,
vacrelstats->rel_tuples);
}
Index: src/backend/libpq/hba.c
===================================================================
RCS file: /home/alvherre/cvs/pgsql/src/backend/libpq/hba.c,v
retrieving revision 1.144
diff -c -r1.144 hba.c
*** src/backend/libpq/hba.c 28 Jun 2005 22:16:45 -0000 1.144
--- src/backend/libpq/hba.c 22 Jul 2005 01:08:51 -0000
***************
*** 39,44 ****
--- 39,45 ----
#define atooid(x) ((Oid) strtoul((x), NULL, 10))
+ #define atoxid(x) ((TransactionId) strtoul((x), NULL, 10))
/* Max size of username ident server can return */
#define IDENT_USERNAME_MAX 512
***************
*** 1002,1014 ****
* dbname: gets database name (must be of size NAMEDATALEN bytes)
* dboid: gets database OID
* dbtablespace: gets database's default tablespace's OID
*
* This is not much related to the other functions in hba.c, but we put it
* here because it uses the next_token() infrastructure.
*/
bool
read_pg_database_line(FILE *fp, char *dbname,
! Oid *dboid, Oid *dbtablespace)
{
char buf[MAX_TOKEN];
--- 1003,1017 ----
* dbname: gets database name (must be of size NAMEDATALEN bytes)
* dboid: gets database OID
* dbtablespace: gets database's default tablespace's OID
+ * dbfrozenxid: get database's frozen Xid
*
* This is not much related to the other functions in hba.c, but we put it
* here because it uses the next_token() infrastructure.
*/
bool
read_pg_database_line(FILE *fp, char *dbname,
! Oid *dboid, Oid *dbtablespace,
! TransactionId *dbfrozenxid)
{
char buf[MAX_TOKEN];
***************
*** 1027,1036 ****
if (!isdigit((unsigned char) buf[0]))
elog(FATAL, "bad data in flat pg_database file");
*dbtablespace = atooid(buf);
- /* discard datfrozenxid */
next_token(fp, buf, sizeof(buf));
if (!isdigit((unsigned char) buf[0]))
elog(FATAL, "bad data in flat pg_database file");
/* expect EOL next */
if (next_token(fp, buf, sizeof(buf)))
elog(FATAL, "bad data in flat pg_database file");
--- 1030,1039 ----
if (!isdigit((unsigned char) buf[0]))
elog(FATAL, "bad data in flat pg_database file");
*dbtablespace = atooid(buf);
next_token(fp, buf, sizeof(buf));
if (!isdigit((unsigned char) buf[0]))
elog(FATAL, "bad data in flat pg_database file");
+ *dbfrozenxid = atoxid(buf);
/* expect EOL next */
if (next_token(fp, buf, sizeof(buf)))
elog(FATAL, "bad data in flat pg_database file");
Index: src/backend/postmaster/autovacuum.c
===================================================================
RCS file: /home/alvherre/cvs/pgsql/src/backend/postmaster/autovacuum.c,v
retrieving revision 1.1
diff -c -r1.1 autovacuum.c
*** src/backend/postmaster/autovacuum.c 14 Jul 2005 05:13:40 -0000 1.1
--- src/backend/postmaster/autovacuum.c 24 Jul 2005 17:31:32 -0000
***************
*** 23,29 ****
--- 23,31 ----
#include "access/genam.h"
#include "access/heapam.h"
+ #include "access/xlog.h"
#include "catalog/indexing.h"
+ #include "catalog/namespace.h"
#include "catalog/pg_autovacuum.h"
#include "catalog/pg_database.h"
#include "commands/vacuum.h"
***************
*** 41,46 ****
--- 43,49 ----
#include "tcop/tcopprot.h"
#include "utils/flatfiles.h"
#include "utils/fmgroids.h"
+ #include "utils/lsyscache.h"
#include "utils/memutils.h"
#include "utils/ps_status.h"
#include "utils/relcache.h"
***************
*** 68,74 ****
--- 71,79 ----
{
Oid oid;
char *name;
+ TransactionId frozenxid;
PgStat_StatDBEntry *entry;
+ int32 age;
} autovac_dbase;
***************
*** 76,83 ****
static pid_t autovac_forkexec(void);
#endif
NON_EXEC_STATIC void AutoVacMain(int argc, char *argv[]);
! static void autovac_check_wraparound(void);
! static void do_autovacuum(PgStat_StatDBEntry *dbentry);
static List *autovac_get_database_list(void);
static void test_rel_for_autovac(Oid relid, PgStat_StatTabEntry *tabentry,
Form_pg_class classForm, Form_pg_autovacuum avForm,
--- 81,87 ----
static pid_t autovac_forkexec(void);
#endif
NON_EXEC_STATIC void AutoVacMain(int argc, char *argv[]);
! static void do_autovacuum(PgStat_StatDBEntry *dbentry, bool whole_db);
static List *autovac_get_database_list(void);
static void test_rel_for_autovac(Oid relid, PgStat_StatTabEntry *tabentry,
Form_pg_class classForm, Form_pg_autovacuum avForm,
***************
*** 194,200 ****
--- 198,206 ----
ListCell *cell;
List *dblist;
autovac_dbase *db;
+ bool whole_db = false;
sigjmp_buf local_sigjmp_buf;
+ TransactionId nextXid;
/* we are a postmaster subprocess now */
IsUnderPostmaster = true;
***************
*** 269,289 ****
/*
* Choose a database to connect to. We pick the database that was least
! * recently auto-vacuumed.
*
* XXX This could be improved if we had more info about whether it needs
* vacuuming before connecting to it. Perhaps look through the pgstats
* data for the database's tables?
- *
- * XXX it is NOT good that we totally ignore databases that have no
- * pgstats entry ...
*/
db = NULL;
foreach(cell, dblist)
{
! autovac_dbase *tmp = lfirst(cell);
tmp->entry = pgstat_fetch_stat_dbentry(tmp->oid);
if (!tmp->entry)
continue;
--- 275,327 ----
/*
* Choose a database to connect to. We pick the database that was least
! * recently auto-vacuumed, or one that needs database-wide vacuum (to
! * prevent Xid wraparound-related data loss.)
*
* XXX This could be improved if we had more info about whether it needs
* vacuuming before connecting to it. Perhaps look through the pgstats
* data for the database's tables?
*/
db = NULL;
+ /*
+ * Get the next Xid that was current as of the last checkpoint.
+ * We will use it to determine whether databases are about to need
+ * database-wide vacuums.
+ */
+ ReadControlFile(&nextXid);
+
foreach(cell, dblist)
{
! autovac_dbase *tmp = lfirst(cell);
! bool this_whole_db;
+ /*
+ * We decide to vacuum a little earlier than when vacuum.c's
+ * vac_truncate_clog() would decide start giving warnings. We
+ * look for the database that most urgently needs a
database-wide
+ * vacuum. As soon as one is found, any other database is
+ * ignored.
+ */
+ tmp->age = (int32) (nextXid - tmp->frozenxid);
+ this_whole_db = (tmp->age > (int32) ((MaxTransactionId >> 3) *
3));
+
+ if (whole_db || this_whole_db)
+ {
+ if (!this_whole_db)
+ continue;
+ if (!db || tmp->age > db->age)
+ {
+ db = tmp;
+ whole_db = true;
+ }
+ continue;
+ }
+
+ /*
+ * Skip databases with no pgstat entry; it means it hasn't seen
+ * any activity.
+ */
tmp->entry = pgstat_fetch_stat_dbentry(tmp->oid);
if (!tmp->entry)
continue;
***************
*** 316,322 ****
/*
* And do an appropriate amount of work on it
*/
! do_autovacuum(db->entry);
}
/* One iteration done, go away */
--- 354,360 ----
/*
* And do an appropriate amount of work on it
*/
! do_autovacuum(db->entry, whole_db);
}
/* One iteration done, go away */
***************
*** 338,343 ****
--- 376,382 ----
FILE *db_file;
Oid db_id;
Oid db_tablespace;
+ TransactionId db_frozenxid;
filename = database_getflatfilename();
db_file = AllocateFile(filename, "r");
***************
*** 346,352 ****
(errcode_for_file_access(),
errmsg("could not open file \"%s\": %m",
filename)));
! while (read_pg_database_line(db_file, thisname, &db_id, &db_tablespace))
{
autovac_dbase *db;
--- 385,392 ----
(errcode_for_file_access(),
errmsg("could not open file \"%s\": %m",
filename)));
! while (read_pg_database_line(db_file, thisname, &db_id, &db_tablespace,
! &db_frozenxid))
{
autovac_dbase *db;
***************
*** 354,359 ****
--- 394,400 ----
db->oid = db_id;
db->name = pstrdup(thisname);
+ db->frozenxid = db_frozenxid;
/* this gets set later */
db->entry = NULL;
***************
*** 369,374 ****
--- 410,421 ----
/*
* Process a database.
*
+ * If whole_db is true, the database is processed as a whole, and the
+ * dbentry parameter is ignored. If it's false, dbentry must hold a valid
+ * pointer to the database entry in the stats databases' hash table, and
+ * it will be used to determine whether vacuum or analyze is needed on a
+ * table per table basis.
+ *
* Note that test_rel_for_autovac generates two separate lists, one for
* vacuum and other for analyze. This is to facilitate processing all
* analyzes first, and then all vacuums.
***************
*** 377,383 ****
* order not to ignore shutdown commands for too long.
*/
static void
! do_autovacuum(PgStat_StatDBEntry *dbentry)
{
Relation classRel,
avRel;
--- 424,430 ----
* order not to ignore shutdown commands for too long.
*/
static void
! do_autovacuum(PgStat_StatDBEntry *dbentry, bool whole_db)
{
Relation classRel,
avRel;
***************
*** 387,392 ****
--- 434,441 ----
*analyze_tables = NIL;
MemoryContext AutovacMemCxt;
+ Assert(dbentry != NULL || whole_db);
+
/* Memory context where cross-transaction state is stored */
AutovacMemCxt = AllocSetContextCreate(TopMemoryContext,
"Autovacuum context",
***************
*** 405,485 ****
*/
MemoryContextSwitchTo(AutovacMemCxt);
! /*
! * If this database is old enough to need a whole-database VACUUM,
! * don't bother checking each table. If that happens, this function
! * will issue the VACUUM command and won't return.
! */
! autovac_check_wraparound();
!
! CHECK_FOR_INTERRUPTS();
!
! classRel = heap_open(RelationRelationId, AccessShareLock);
! avRel = heap_open(AutovacuumRelationId, AccessShareLock);
!
! relScan = heap_beginscan(classRel, SnapshotNow, 0, NULL);
!
! /* Scan pg_class looking for tables to vacuum */
! while ((tuple = heap_getnext(relScan, ForwardScanDirection)) != NULL)
{
! Form_pg_class classForm = (Form_pg_class) GETSTRUCT(tuple);
! Form_pg_autovacuum avForm = NULL;
! PgStat_StatTabEntry *tabentry;
! SysScanDesc avScan;
! HeapTuple avTup;
! ScanKeyData entry[1];
! Oid relid;
!
! /* Skip non-table entries. */
! /* XXX possibly allow RELKIND_TOASTVALUE entries here too? */
! if (classForm->relkind != RELKIND_RELATION)
! continue;
!
! relid = HeapTupleGetOid(tuple);
!
! /* See if we have a pg_autovacuum entry for this relation. */
! ScanKeyInit(&entry[0],
! Anum_pg_autovacuum_vacrelid,
! BTEqualStrategyNumber, F_OIDEQ,
! ObjectIdGetDatum(relid));
!
! avScan = systable_beginscan(avRel, AutovacuumRelidIndexId, true,
!
SnapshotNow, 1, entry);
!
! avTup = systable_getnext(avScan);
!
! if (HeapTupleIsValid(avTup))
! avForm = (Form_pg_autovacuum) GETSTRUCT(avTup);
!
! tabentry = hash_search(dbentry->tables, &relid,
! HASH_FIND, NULL);
! test_rel_for_autovac(relid, tabentry, classForm, avForm,
! &vacuum_tables,
&analyze_tables);
! systable_endscan(avScan);
! }
! heap_endscan(relScan);
! heap_close(avRel, AccessShareLock);
! heap_close(classRel, AccessShareLock);
! CHECK_FOR_INTERRUPTS();
! /*
! * Perform operations on collected tables.
! */
! if (analyze_tables)
! autovacuum_do_vac_analyze(analyze_tables, false);
! CHECK_FOR_INTERRUPTS();
! /* get back to proper context */
! MemoryContextSwitchTo(AutovacMemCxt);
! if (vacuum_tables)
! autovacuum_do_vac_analyze(vacuum_tables, true);
/* Finally close out the last transaction. */
CommitTransactionCommand();
--- 454,548 ----
*/
MemoryContextSwitchTo(AutovacMemCxt);
! if (whole_db)
{
! elog(DEBUG2, "autovacuum: VACUUM ANALYZE whole database");
! autovacuum_do_vac_analyze(NIL, true);
! }
! else
! {
! /* Get the stat database entry where pgstat stores shared
relations */
! PgStat_StatDBEntry *shared =
pgstat_fetch_stat_dbentry(InvalidOid);
! CHECK_FOR_INTERRUPTS();
! classRel = heap_open(RelationRelationId, AccessShareLock);
! avRel = heap_open(AutovacuumRelationId, AccessShareLock);
! relScan = heap_beginscan(classRel, SnapshotNow, 0, NULL);
!
! /* Scan pg_class looking for tables to vacuum */
! while ((tuple = heap_getnext(relScan, ForwardScanDirection)) !=
NULL)
! {
! Form_pg_class classForm = (Form_pg_class)
GETSTRUCT(tuple);
! Form_pg_autovacuum avForm = NULL;
! PgStat_StatTabEntry *tabentry;
! SysScanDesc avScan;
! HeapTuple avTup;
! ScanKeyData entry[1];
! Oid relid;
! Oid nspOid;
!
! /* Skip non-table entries. */
! /* XXX possibly allow RELKIND_TOASTVALUE entries here
too? */
! if (classForm->relkind != RELKIND_RELATION)
! continue;
!
! relid = HeapTupleGetOid(tuple);
!
! /* Skip temp tables (i.e. those in temp namespaces) */
! nspOid = get_rel_namespace(relid);
! if (isTempNamespace(nspOid))
! continue;
!
! /* See if we have a pg_autovacuum entry for this
relation. */
! ScanKeyInit(&entry[0],
! Anum_pg_autovacuum_vacrelid,
! BTEqualStrategyNumber, F_OIDEQ,
! ObjectIdGetDatum(relid));
!
! avScan = systable_beginscan(avRel,
AutovacuumRelidIndexId, true,
!
SnapshotNow, 1, entry);
!
! avTup = systable_getnext(avScan);
!
! if (HeapTupleIsValid(avTup))
! avForm = (Form_pg_autovacuum) GETSTRUCT(avTup);
!
! if (classForm->relisshared)
! tabentry = hash_search(shared->tables, &relid,
!
HASH_FIND, NULL);
! else
! tabentry = hash_search(dbentry->tables, &relid,
!
HASH_FIND, NULL);
!
! test_rel_for_autovac(relid, tabentry, classForm, avForm,
!
&vacuum_tables, &analyze_tables);
!
! systable_endscan(avScan);
! }
!
! heap_endscan(relScan);
! heap_close(avRel, AccessShareLock);
! heap_close(classRel, AccessShareLock);
! CHECK_FOR_INTERRUPTS();
! /*
! * Perform operations on collected tables.
! */
! if (analyze_tables)
! autovacuum_do_vac_analyze(analyze_tables, false);
! CHECK_FOR_INTERRUPTS();
! /* get back to proper context */
! MemoryContextSwitchTo(AutovacMemCxt);
! if (vacuum_tables)
! autovacuum_do_vac_analyze(vacuum_tables, true);
! }
/* Finally close out the last transaction. */
CommitTransactionCommand();
***************
*** 607,615 ****
}
else if (anltuples > anlthresh)
{
! elog(DEBUG2, "will ANALYZE %s",
! RelationGetRelationName(rel));
! *analyze_tables = lappend_oid(*analyze_tables, relid);
}
RelationClose(rel);
--- 670,682 ----
}
else if (anltuples > anlthresh)
{
! /* ANALYZE refuses to work with pg_statistics */
! if (relid != StatisticRelationId)
! {
! elog(DEBUG2, "will ANALYZE %s",
! RelationGetRelationName(rel));
! *analyze_tables = lappend_oid(*analyze_tables, relid);
! }
}
RelationClose(rel);
***************
*** 646,706 ****
}
/*
- * autovac_check_wraparound
- * Check database Xid wraparound
- *
- * Check pg_database to see if the last database-wide VACUUM was too long ago,
- * and issue one now if so. If this comes to pass, we do not return, as there
- * is no point in checking individual tables -- they will all get vacuumed
- * anyway.
- */
- static void
- autovac_check_wraparound(void)
- {
- Relation relation;
- ScanKeyData entry[1];
- HeapScanDesc scan;
- HeapTuple tuple;
- Form_pg_database dbform;
- int32 age;
- bool whole_db;
-
- relation = heap_open(DatabaseRelationId, AccessShareLock);
-
- /* Must use a heap scan, since there's no syscache for pg_database */
- ScanKeyInit(&entry[0],
- ObjectIdAttributeNumber,
- BTEqualStrategyNumber, F_OIDEQ,
- ObjectIdGetDatum(MyDatabaseId));
-
- scan = heap_beginscan(relation, SnapshotNow, 1, entry);
-
- tuple = heap_getnext(scan, ForwardScanDirection);
-
- if (!HeapTupleIsValid(tuple))
- elog(ERROR, "could not find tuple for database %u",
MyDatabaseId);
-
- dbform = (Form_pg_database) GETSTRUCT(tuple);
-
- /*
- * We decide to vacuum at the same point where vacuum.c's
- * vac_truncate_clog() would decide to start giving warnings.
- */
- age = (int32) (GetTopTransactionId() - dbform->datfrozenxid);
- whole_db = (age > (int32) ((MaxTransactionId >> 3) * 3));
-
- heap_endscan(scan);
- heap_close(relation, AccessShareLock);
-
- if (whole_db)
- {
- elog(LOG, "autovacuum: VACUUM ANALYZE whole database");
- autovacuum_do_vac_analyze(NIL, true);
- proc_exit(0);
- }
- }
-
- /*
* AutoVacuumingActive
* Check GUC vars and report whether the autovacuum process should
be
* running.
--- 713,718 ----
Index: src/backend/postmaster/pgstat.c
===================================================================
RCS file: /home/alvherre/cvs/pgsql/src/backend/postmaster/pgstat.c,v
retrieving revision 1.101
diff -c -r1.101 pgstat.c
*** src/backend/postmaster/pgstat.c 24 Jul 2005 00:33:28 -0000 1.101
--- src/backend/postmaster/pgstat.c 24 Jul 2005 17:55:54 -0000
***************
*** 119,130 ****
static bool pgStatRunningInCollector = FALSE;
! static int pgStatTabstatAlloc = 0;
! static int pgStatTabstatUsed = 0;
! static PgStat_MsgTabstat **pgStatTabstatMessages = NULL;
#define TABSTAT_QUANTUM 4 /* we alloc this many at a time
*/
static int pgStatXactCommit = 0;
static int pgStatXactRollback = 0;
--- 119,141 ----
static bool pgStatRunningInCollector = FALSE;
! /*
! * Place where backends store per-table info to be sent to the collector.
! * We store shared relations separately from non-shared ones, to be able to
! * send them in separate messages.
! */
! typedef struct TabStatArray
! {
! int tsa_alloc; /* num
allocated */
! int tsa_used; /* num
actually used */
! PgStat_MsgTabstat **tsa_messages; /* the array itself */
! } TabStatArray;
#define TABSTAT_QUANTUM 4 /* we alloc this many at a time
*/
+ static TabStatArray RegularTabStat;
+ static TabStatArray SharedTabStat;
+
static int pgStatXactCommit = 0;
static int pgStatXactRollback = 0;
***************
*** 182,187 ****
--- 193,199 ----
static void pgstat_recv_vacuum(PgStat_MsgVacuum *msg, int len);
static void pgstat_recv_analyze(PgStat_MsgAnalyze *msg, int len);
+ extern void display_statfile_contents(void);
/* ------------------------------------------------------------
* Public functions called from postmaster follow
***************
*** 614,619 ****
--- 626,632 ----
if (pgStatSock < 0)
return;
+ /* can't use pgstat_setheader() because it's not called in a backend */
MemSet(&(msg.m_hdr), 0, sizeof(msg.m_hdr));
msg.m_hdr.m_type = PGSTAT_MTYPE_BETERM;
msg.m_hdr.m_procpid = pid;
***************
*** 684,690 ****
* ---------
*/
void
! pgstat_report_vacuum(Oid tableoid, bool analyze, PgStat_Counter tuples)
{
PgStat_MsgVacuum msg;
--- 697,704 ----
* ---------
*/
void
! pgstat_report_vacuum(Oid tableoid, bool shared, bool analyze,
! PgStat_Counter tuples)
{
PgStat_MsgVacuum msg;
***************
*** 692,698 ****
return;
pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_VACUUM);
! msg.m_databaseid = MyDatabaseId;
msg.m_tableoid = tableoid;
msg.m_analyze = analyze;
msg.m_tuples = tuples;
--- 706,715 ----
return;
pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_VACUUM);
! if (shared)
! msg.m_databaseid = InvalidOid;
! else
! msg.m_databaseid = MyDatabaseId;
msg.m_tableoid = tableoid;
msg.m_analyze = analyze;
msg.m_tuples = tuples;
***************
*** 706,712 ****
* --------
*/
void
! pgstat_report_analyze(Oid tableoid, PgStat_Counter livetuples,
PgStat_Counter deadtuples)
{
PgStat_MsgAnalyze msg;
--- 723,729 ----
* --------
*/
void
! pgstat_report_analyze(Oid tableoid, bool shared, PgStat_Counter livetuples,
PgStat_Counter deadtuples)
{
PgStat_MsgAnalyze msg;
***************
*** 715,721 ****
return;
pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ANALYZE);
! msg.m_databaseid = MyDatabaseId;
msg.m_tableoid = tableoid;
msg.m_live_tuples = livetuples;
msg.m_dead_tuples = deadtuples;
--- 732,741 ----
return;
pgstat_setheader(&msg.m_hdr, PGSTAT_MTYPE_ANALYZE);
! if (shared)
! msg.m_databaseid = InvalidOid;
! else
! msg.m_databaseid = MyDatabaseId;
msg.m_tableoid = tableoid;
msg.m_live_tuples = livetuples;
msg.m_dead_tuples = deadtuples;
***************
*** 784,790 ****
pgstat_collect_blocklevel))
{
/* Not reporting stats, so just flush whatever we have */
! pgStatTabstatUsed = 0;
return;
}
--- 804,811 ----
pgstat_collect_blocklevel))
{
/* Not reporting stats, so just flush whatever we have */
! RegularTabStat.tsa_used = 0;
! SharedTabStat.tsa_used = 0;
return;
}
***************
*** 792,800 ****
* For each message buffer used during the last query set the header
* fields and send it out.
*/
! for (i = 0; i < pgStatTabstatUsed; i++)
{
! PgStat_MsgTabstat *tsmsg = pgStatTabstatMessages[i];
int n;
int len;
--- 813,821 ----
* For each message buffer used during the last query set the header
* fields and send it out.
*/
! for (i = 0; i < RegularTabStat.tsa_used; i++)
{
! PgStat_MsgTabstat *tsmsg = RegularTabStat.tsa_messages[i];
int n;
int len;
***************
*** 811,818 ****
tsmsg->m_databaseid = MyDatabaseId;
pgstat_send(tsmsg, len);
}
! pgStatTabstatUsed = 0;
}
--- 832,857 ----
tsmsg->m_databaseid = MyDatabaseId;
pgstat_send(tsmsg, len);
}
+ RegularTabStat.tsa_used = 0;
+
+ /* Ditto, for shared relations */
+ for (i = 0; i < SharedTabStat.tsa_used; i++)
+ {
+ PgStat_MsgTabstat *tsmsg = SharedTabStat.tsa_messages[i];
+ int n;
+ int len;
! n = tsmsg->m_nentries;
! len = offsetof(PgStat_MsgTabstat, m_entry[0]) +
! n * sizeof(PgStat_TableEntry);
!
! /* We don't report transaction commit/abort here */
!
! pgstat_setheader(&tsmsg->m_hdr, PGSTAT_MTYPE_TABSTAT);
! tsmsg->m_databaseid = InvalidOid;
! pgstat_send(tsmsg, len);
! }
! SharedTabStat.tsa_used = 0;
}
***************
*** 1045,1081 ****
}
/*
! * Create or enlarge the pgStatTabstatMessages array
*/
static void
! more_tabstat_space(void)
{
PgStat_MsgTabstat *newMessages;
PgStat_MsgTabstat **msgArray;
! int newAlloc = pgStatTabstatAlloc + TABSTAT_QUANTUM;
int i;
/* Create (another) quantum of message buffers */
newMessages = (PgStat_MsgTabstat *)
MemoryContextAllocZero(TopMemoryContext,
sizeof(PgStat_MsgTabstat) * TABSTAT_QUANTUM);
/* Create or enlarge the pointer array */
! if (pgStatTabstatMessages == NULL)
msgArray = (PgStat_MsgTabstat **)
MemoryContextAlloc(TopMemoryContext,
sizeof(PgStat_MsgTabstat *) * newAlloc);
else
msgArray = (PgStat_MsgTabstat **)
! repalloc(pgStatTabstatMessages,
sizeof(PgStat_MsgTabstat *) *
newAlloc);
for (i = 0; i < TABSTAT_QUANTUM; i++)
! msgArray[pgStatTabstatAlloc + i] = newMessages++;
! pgStatTabstatMessages = msgArray;
! pgStatTabstatAlloc = newAlloc;
! Assert(pgStatTabstatUsed < pgStatTabstatAlloc);
}
/* ----------
--- 1084,1124 ----
}
/*
! * Enlarge a TabStatArray
*/
static void
! more_tabstat_space(TabStatArray *tsarr)
{
PgStat_MsgTabstat *newMessages;
PgStat_MsgTabstat **msgArray;
! int newAlloc;
int i;
+ AssertArg(PointerIsValid(tsarr));
+
+ newAlloc = tsarr->tsa_alloc + TABSTAT_QUANTUM;
+
/* Create (another) quantum of message buffers */
newMessages = (PgStat_MsgTabstat *)
MemoryContextAllocZero(TopMemoryContext,
sizeof(PgStat_MsgTabstat) * TABSTAT_QUANTUM);
/* Create or enlarge the pointer array */
! if (tsarr->tsa_messages == NULL)
msgArray = (PgStat_MsgTabstat **)
MemoryContextAlloc(TopMemoryContext,
sizeof(PgStat_MsgTabstat *) * newAlloc);
else
msgArray = (PgStat_MsgTabstat **)
! repalloc(tsarr->tsa_messages,
sizeof(PgStat_MsgTabstat *) *
newAlloc);
for (i = 0; i < TABSTAT_QUANTUM; i++)
! msgArray[tsarr->tsa_alloc + i] = newMessages++;
! tsarr->tsa_messages = msgArray;
! tsarr->tsa_alloc = newAlloc;
! Assert(tsarr->tsa_used < tsarr->tsa_alloc);
}
/* ----------
***************
*** 1092,1097 ****
--- 1135,1141 ----
{
Oid rel_id = rel->rd_id;
PgStat_TableEntry *useent;
+ TabStatArray *tsarr;
PgStat_MsgTabstat *tsmsg;
int mb;
int i;
***************
*** 1112,1123 ****
return;
}
/*
* Search the already-used message slots for this relation.
*/
! for (mb = 0; mb < pgStatTabstatUsed; mb++)
{
! tsmsg = pgStatTabstatMessages[mb];
for (i = tsmsg->m_nentries; --i >= 0;)
{
--- 1156,1169 ----
return;
}
+ tsarr = rel->rd_rel->relisshared ? &SharedTabStat : &RegularTabStat;
+
/*
* Search the already-used message slots for this relation.
*/
! for (mb = 0; mb < tsarr->tsa_used; mb++)
{
! tsmsg = tsarr->tsa_messages[mb];
for (i = tsmsg->m_nentries; --i >= 0;)
{
***************
*** 1146,1159 ****
/*
* If we ran out of message buffers, we just allocate more.
*/
! if (pgStatTabstatUsed >= pgStatTabstatAlloc)
! more_tabstat_space();
/*
* Use the first entry of the next message buffer.
*/
! mb = pgStatTabstatUsed++;
! tsmsg = pgStatTabstatMessages[mb];
tsmsg->m_nentries = 1;
useent = &tsmsg->m_entry[0];
MemSet(useent, 0, sizeof(PgStat_TableEntry));
--- 1192,1205 ----
/*
* If we ran out of message buffers, we just allocate more.
*/
! if (tsarr->tsa_used >= tsarr->tsa_alloc)
! more_tabstat_space(tsarr);
/*
* Use the first entry of the next message buffer.
*/
! mb = tsarr->tsa_used++;
! tsmsg = tsarr->tsa_messages[mb];
tsmsg->m_nentries = 1;
useent = &tsmsg->m_entry[0];
MemSet(useent, 0, sizeof(PgStat_TableEntry));
***************
*** 1183,1195 ****
* message buffer used without slots, causing the next report to tell
* new xact-counters.
*/
! if (pgStatTabstatAlloc == 0)
! more_tabstat_space();
! if (pgStatTabstatUsed == 0)
{
! pgStatTabstatUsed++;
! pgStatTabstatMessages[0]->m_nentries = 0;
}
}
--- 1229,1241 ----
* message buffer used without slots, causing the next report to tell
* new xact-counters.
*/
! if (RegularTabStat.tsa_alloc == 0)
! more_tabstat_space(&RegularTabStat);
! if (RegularTabStat.tsa_used == 0)
{
! RegularTabStat.tsa_used++;
! RegularTabStat.tsa_messages[0]->m_nentries = 0;
}
}
***************
*** 1215,1227 ****
* message buffer used without slots, causing the next report to tell
* new xact-counters.
*/
! if (pgStatTabstatAlloc == 0)
! more_tabstat_space();
! if (pgStatTabstatUsed == 0)
{
! pgStatTabstatUsed++;
! pgStatTabstatMessages[0]->m_nentries = 0;
}
}
--- 1261,1273 ----
* message buffer used without slots, causing the next report to tell
* new xact-counters.
*/
! if (RegularTabStat.tsa_alloc == 0)
! more_tabstat_space(&RegularTabStat);
! if (RegularTabStat.tsa_used == 0)
{
! RegularTabStat.tsa_used++;
! RegularTabStat.tsa_messages[0]->m_nentries = 0;
}
}
***************
*** 2387,2393 ****
* pgstat_read_statsfile() -
*
* Reads in an existing statistics collector and initializes the
! * databases hash table (who's entries point to the tables hash tables)
* and the current backend table.
* ----------
*/
--- 2433,2439 ----
* pgstat_read_statsfile() -
*
* Reads in an existing statistics collector and initializes the
! * databases' hash table (whose entries point to the tables' hash tables)
* and the current backend table.
* ----------
*/
Index: src/backend/postmaster/postmaster.c
===================================================================
RCS file: /home/alvherre/cvs/pgsql/src/backend/postmaster/postmaster.c,v
retrieving revision 1.460
diff -c -r1.460 postmaster.c
*** src/backend/postmaster/postmaster.c 21 Jul 2005 03:56:11 -0000 1.460
--- src/backend/postmaster/postmaster.c 24 Jul 2005 17:54:31 -0000
***************
*** 1164,1176 ****
/*
* Wait for something to happen.
*
! * We wait at most one minute, to ensure that the other
background
! * tasks handled below get done even when no requests are
! * arriving.
*/
memcpy((char *) &rmask, (char *) &readmask, sizeof(fd_set));
! timeout.tv_sec = 60;
timeout.tv_usec = 0;
PG_SETMASK(&UnBlockSig);
--- 1164,1176 ----
/*
* Wait for something to happen.
*
! * We wait at most one minute, or the minimum autovacuum delay,
to
! * ensure that the other background tasks handled below get done
! * even when no requests are arriving.
*/
memcpy((char *) &rmask, (char *) &readmask, sizeof(fd_set));
! timeout.tv_sec = Min(60, autovacuum_naptime);
timeout.tv_usec = 0;
PG_SETMASK(&UnBlockSig);
Index: src/backend/utils/init/postinit.c
===================================================================
RCS file: /home/alvherre/cvs/pgsql/src/backend/utils/init/postinit.c,v
retrieving revision 1.153
diff -c -r1.153 postinit.c
*** src/backend/utils/init/postinit.c 14 Jul 2005 05:13:41 -0000 1.153
--- src/backend/utils/init/postinit.c 24 Jul 2005 15:50:28 -0000
***************
*** 78,83 ****
--- 78,84 ----
char *filename;
FILE *db_file;
char thisname[NAMEDATALEN];
+ TransactionId frozenxid;
filename = database_getflatfilename();
db_file = AllocateFile(filename, "r");
***************
*** 86,92 ****
(errcode_for_file_access(),
errmsg("could not open file \"%s\": %m",
filename)));
! while (read_pg_database_line(db_file, thisname, db_id, db_tablespace))
{
if (strcmp(thisname, name) == 0)
{
--- 87,94 ----
(errcode_for_file_access(),
errmsg("could not open file \"%s\": %m",
filename)));
! while (read_pg_database_line(db_file, thisname, db_id, db_tablespace,
! &frozenxid))
{
if (strcmp(thisname, name) == 0)
{
***************
*** 170,179 ****
/*
* Also check that the database is currently allowing connections.
* (We do not enforce this in standalone mode, however, so that there is
! * a way to recover from "UPDATE pg_database SET datallowconn = false;")
*/
dbform = (Form_pg_database) GETSTRUCT(tup);
! if (IsUnderPostmaster && !dbform->datallowconn)
ereport(FATAL,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("database \"%s\" is not currently accepting
connections",
--- 172,182 ----
/*
* Also check that the database is currently allowing connections.
* (We do not enforce this in standalone mode, however, so that there is
! * a way to recover from "UPDATE pg_database SET datallowconn = false;".
! * We do not enforce it for the autovacuum process either.)
*/
dbform = (Form_pg_database) GETSTRUCT(tup);
! if (IsUnderPostmaster && !IsAutoVacuumProcess() &&
!dbform->datallowconn)
ereport(FATAL,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("database \"%s\" is not currently accepting
connections",
Index: src/backend/utils/misc/guc.c
===================================================================
RCS file: /home/alvherre/cvs/pgsql/src/backend/utils/misc/guc.c,v
retrieving revision 1.277
diff -c -r1.277 guc.c
*** src/backend/utils/misc/guc.c 23 Jul 2005 21:05:47 -0000 1.277
--- src/backend/utils/misc/guc.c 24 Jul 2005 17:55:56 -0000
***************
*** 1412,1423 ****
},
{
{"autovacuum_naptime", PGC_SIGHUP, AUTOVACUUM,
gettext_noop("Time to sleep between autovacuum runs, in
seconds."),
NULL
},
&autovacuum_naptime,
! 60, 0, INT_MAX, NULL, NULL
},
{
{"autovacuum_vacuum_threshold", PGC_SIGHUP, AUTOVACUUM,
--- 1412,1427 ----
},
{
+ /*
+ * Note we set an artificial minimum of 1 sec, so that the
+ * postmaster's main loop does not turn into busy-wait.
+ */
{"autovacuum_naptime", PGC_SIGHUP, AUTOVACUUM,
gettext_noop("Time to sleep between autovacuum runs, in
seconds."),
NULL
},
&autovacuum_naptime,
! 60, 1, INT_MAX, NULL, NULL
},
{
{"autovacuum_vacuum_threshold", PGC_SIGHUP, AUTOVACUUM,
Index: src/include/pgstat.h
===================================================================
RCS file: /home/alvherre/cvs/pgsql/src/include/pgstat.h,v
retrieving revision 1.33
diff -c -r1.33 pgstat.h
*** src/include/pgstat.h 14 Jul 2005 05:13:43 -0000 1.33
--- src/include/pgstat.h 22 Jul 2005 01:08:51 -0000
***************
*** 384,393 ****
extern void pgstat_report_activity(const char *what);
extern void pgstat_report_tabstat(void);
extern void pgstat_report_autovac(void);
! extern void pgstat_report_vacuum(Oid tableoid, bool analyze,
PgStat_Counter tuples);
! extern void pgstat_report_analyze(Oid tableoid, PgStat_Counter livetuples,
! PgStat_Counter deadtuples);
extern int pgstat_vacuum_tabstat(void);
extern void pgstat_reset_counters(void);
--- 384,393 ----
extern void pgstat_report_activity(const char *what);
extern void pgstat_report_tabstat(void);
extern void pgstat_report_autovac(void);
! extern void pgstat_report_vacuum(Oid tableoid, bool shared, bool analyze,
PgStat_Counter tuples);
! extern void pgstat_report_analyze(Oid tableoid, bool shared,
! PgStat_Counter livetuples,
PgStat_Counter deadtuples);
extern int pgstat_vacuum_tabstat(void);
extern void pgstat_reset_counters(void);
Index: src/include/access/xlog.h
===================================================================
RCS file: /home/alvherre/cvs/pgsql/src/include/access/xlog.h,v
retrieving revision 1.66
diff -c -r1.66 xlog.h
*** src/include/access/xlog.h 4 Jul 2005 04:51:52 -0000 1.66
--- src/include/access/xlog.h 23 Jul 2005 02:04:33 -0000
***************
*** 155,160 ****
--- 155,161 ----
extern void xlog_redo(XLogRecPtr lsn, XLogRecord *record);
extern void xlog_desc(char *buf, uint8 xl_info, char *rec);
+ extern void ReadControlFile(TransactionId *nextXid);
extern void UpdateControlFile(void);
extern int XLOGShmemSize(void);
extern void XLOGShmemInit(void);
Index: src/include/libpq/hba.h
===================================================================
RCS file: /home/alvherre/cvs/pgsql/src/include/libpq/hba.h,v
retrieving revision 1.38
diff -c -r1.38 hba.h
*** src/include/libpq/hba.h 28 Jun 2005 05:09:13 -0000 1.38
--- src/include/libpq/hba.h 22 Jul 2005 01:08:51 -0000
***************
*** 37,42 ****
extern int hba_getauthmethod(hbaPort *port);
extern int authident(hbaPort *port);
extern bool read_pg_database_line(FILE *fp, char *dbname,
! Oid *dboid,
Oid *dbtablespace);
#endif /* HBA_H */
--- 37,43 ----
extern int hba_getauthmethod(hbaPort *port);
extern int authident(hbaPort *port);
extern bool read_pg_database_line(FILE *fp, char *dbname,
! Oid *dboid,
Oid *dbtablespace,
! TransactionId
*dbfrozenxid);
#endif /* HBA_H */
---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match
