Re: [PATCHES] The vacuum-ignore-vacuum patch
Cc: pgsql-hackers removed, as this mail contains a patch.
Tom Lane wrote:
> Alvaro Herrera <[EMAIL PROTECTED]> writes:
> > Hannu Krossing asked me about his patch to ignore transactions running
> > VACUUM LAZY in other vacuum transactions. I attach a version of the
> > patch updated to the current sources.
>
> nonInVacuumXmin seems useless ... perhaps a vestige of some earlier
> version of the computation?
Yup -- I checked that code and found out that nonInVacuumXmin can be
taken out as it's not used anywhere. One upside of this is that taking
it out means we can remove all diffs to GetSnapshotData. New patch
attached; it's a bit smaller than the last one.
I'm currently testing it. Since it appears there are no further
objections, I intend to commit it.
--
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.
Index: src/backend/access/transam/twophase.c
===
RCS file: /home/alvherre/cvs/pgsql/src/backend/access/transam/twophase.c,v
retrieving revision 1.21
diff -c -r1.21 twophase.c
*** src/backend/access/transam/twophase.c 14 Jul 2006 14:52:17 -
1.21
--- src/backend/access/transam/twophase.c 24 Jul 2006 22:16:35 -
***
*** 279,284
--- 279,285
gxact->proc.pid = 0;
gxact->proc.databaseId = databaseid;
gxact->proc.roleId = owner;
+ gxact->proc.inVacuum = false;
gxact->proc.lwWaiting = false;
gxact->proc.lwExclusive = false;
gxact->proc.lwWaitLink = NULL;
Index: src/backend/access/transam/xact.c
===
RCS file: /home/alvherre/cvs/pgsql/src/backend/access/transam/xact.c,v
retrieving revision 1.224
diff -c -r1.224 xact.c
*** src/backend/access/transam/xact.c 24 Jul 2006 16:32:44 - 1.224
--- src/backend/access/transam/xact.c 24 Jul 2006 22:16:35 -
***
*** 1529,1534
--- 1529,1535
LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE);
MyProc->xid = InvalidTransactionId;
MyProc->xmin = InvalidTransactionId;
+ MyProc->inVacuum = false; /* must be cleared with
xid/xmin */
/* Clear the subtransaction-XID cache too while holding the
lock */
MyProc->subxids.nxids = 0;
***
*** 1764,1769
--- 1765,1771
LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE);
MyProc->xid = InvalidTransactionId;
MyProc->xmin = InvalidTransactionId;
+ MyProc->inVacuum = false; /* must be cleared with xid/xmin */
/* Clear the subtransaction-XID cache too while holding the lock */
MyProc->subxids.nxids = 0;
***
*** 1927,1932
--- 1929,1935
LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE);
MyProc->xid = InvalidTransactionId;
MyProc->xmin = InvalidTransactionId;
+ MyProc->inVacuum = false; /* must be cleared with
xid/xmin */
/* Clear the subtransaction-XID cache too while holding the
lock */
MyProc->subxids.nxids = 0;
Index: src/backend/access/transam/xlog.c
===
RCS file: /home/alvherre/cvs/pgsql/src/backend/access/transam/xlog.c,v
retrieving revision 1.244
diff -c -r1.244 xlog.c
*** src/backend/access/transam/xlog.c 14 Jul 2006 14:52:17 - 1.244
--- src/backend/access/transam/xlog.c 24 Jul 2006 22:16:35 -
***
*** 5413,5419
* StartupSUBTRANS hasn't been called yet.
*/
if (!InRecovery)
! TruncateSUBTRANS(GetOldestXmin(true));
if (!shutdown)
ereport(DEBUG2,
--- 5413,5419
* StartupSUBTRANS hasn't been called yet.
*/
if (!InRecovery)
! TruncateSUBTRANS(GetOldestXmin(true, false));
if (!shutdown)
ereport(DEBUG2,
Index: src/backend/catalog/index.c
===
RCS file: /home/alvherre/cvs/pgsql/src/backend/catalog/index.c,v
retrieving revision 1.269
diff -c -r1.269 index.c
*** src/backend/catalog/index.c 13 Jul 2006 16:49:13 - 1.269
--- src/backend/catalog/index.c 24 Jul 2006 22:16:35 -
***
*** 1367,1373
else
{
snapshot = SnapshotAny;
! OldestXmin = GetOldestXmin(heapRelation->rd_rel->relisshared);
}
scan = heap_beginscan(heapRelation, /* relation */
--- 1367,1374
else
{
snapshot = SnapshotAny;
! /* okay to ignore lazy VACUUMs here */
! OldestXmin = GetOldestXmin(heapRelation->rd_rel->relisshared,
true);
}
scan = heap_beginscan(heapRelation, /* relation */
Index: src/backend/command
Re: [PATCHES] The vacuum-ignore-vacuum patch
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> A possible objection to this is that it would foreclose running VACUUM >> and ANALYZE as a single transaction, exactly because of the point that >> we couldn't insert pg_statistic rows using a lazy vacuum's XID. > Hmm, what about having a single scan for both, and then starting a > normal transaction just for the sake of inserting the pg_statistics > tuple? We could, but I think memory consumption would be the issue. VACUUM wants a lotta memory for the dead-TIDs array, ANALYZE wants a lot for its statistics gathering ... even more if it's trying to take a larger sample than before. (This is probably why we kept them separate in the last rewrite.) > I think the interactions of Xids and vacuum and other stuff are starting > to get complex; IMHO it warrants having a README.vacuum, or something. Go for it ... regards, tom lane ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PATCHES] The vacuum-ignore-vacuum patch
Tom Lane wrote: > Alvaro Herrera <[EMAIL PROTECTED]> writes: > > Hannu Krossing asked me about his patch to ignore transactions running > > VACUUM LAZY in other vacuum transactions. I attach a version of the > > patch updated to the current sources. > > nonInVacuumXmin seems useless ... perhaps a vestige of some earlier > version of the computation? Hmm ... I remember removing a now-useless variable somewhere, but maybe this one escaped me. I don't have the code handy -- will check. > In general, it seems to me that a transaction running lazy vacuum could > be ignored for every purpose except truncating clog/subtrans. Since it > will never insert its own XID into the database (note: VACUUM ANALYZE is > run as two separate transactions, hence the pg_statistic rows inserted > by ANALYZE are not a counterexample), there's no need for anyone to > include it as running in their snapshots. So unless I'm missing > something, this is a safe change for lazy vacuum, but perhaps not for > full vacuum, which *does* put its XID into the database. But keep in mind that in the current code, clog truncation takes relminxid (actually datminxid) into account, not running transactions, so AFAICS this should affect anything. Subtrans truncation is different and it certainly should consider lazy vacuum's Xids. > A possible objection to this is that it would foreclose running VACUUM > and ANALYZE as a single transaction, exactly because of the point that > we couldn't insert pg_statistic rows using a lazy vacuum's XID. I think > there was some discussion of doing that in connection with enlarging > ANALYZE's sample greatly --- if ANALYZE goes back to being a full scan > or nearly so, it'd sure be nice to combine it with the VACUUM scan. > However maybe we should just accept that as the price of not having > multiple vacuums interfere with each other. Hmm, what about having a single scan for both, and then starting a normal transaction just for the sake of inserting the pg_statistics tuple? I think the interactions of Xids and vacuum and other stuff are starting to get complex; IMHO it warrants having a README.vacuum, or something. -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [PATCHES] The vacuum-ignore-vacuum patch
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Hannu Krossing asked me about his patch to ignore transactions running > VACUUM LAZY in other vacuum transactions. I attach a version of the > patch updated to the current sources. nonInVacuumXmin seems useless ... perhaps a vestige of some earlier version of the computation? In general, it seems to me that a transaction running lazy vacuum could be ignored for every purpose except truncating clog/subtrans. Since it will never insert its own XID into the database (note: VACUUM ANALYZE is run as two separate transactions, hence the pg_statistic rows inserted by ANALYZE are not a counterexample), there's no need for anyone to include it as running in their snapshots. So unless I'm missing something, this is a safe change for lazy vacuum, but perhaps not for full vacuum, which *does* put its XID into the database. A possible objection to this is that it would foreclose running VACUUM and ANALYZE as a single transaction, exactly because of the point that we couldn't insert pg_statistic rows using a lazy vacuum's XID. I think there was some discussion of doing that in connection with enlarging ANALYZE's sample greatly --- if ANALYZE goes back to being a full scan or nearly so, it'd sure be nice to combine it with the VACUUM scan. However maybe we should just accept that as the price of not having multiple vacuums interfere with each other. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PATCHES] The vacuum-ignore-vacuum patch
Am Dienstag, 11. Juli 2006 23:01 schrieb Alvaro Herrera: > One exception is that we can't do that with full vacuums. The reason is > that full vacuum may want to run user-defined functions to be able to > index the tuples it moves. This isn't a problem normally, except in the > case where the function tries to scan some other table: if we ignored > that transaction, then another lazy vacuum might delete tuples from that > table that we need to see. Functions in the index expression must be immutable, so I don't think that is a real concern. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---(end of broadcast)--- TIP 6: explain analyze is your friend
