Re: [HACKERS] truncating pg_multixact/members

2014-02-13 Thread Alvaro Herrera
Alvaro Herrera escribió: > So here are two patches -- the first one, for 9.3 and HEAD, introduce > the new aging variables and use them throughout vacuum and autovacuum, > including per-table options; the second one adjusts the struct > declarations to avoid the ABI break in VacuumStmt and StdRdOp

Re: [HACKERS] truncating pg_multixact/members

2014-02-13 Thread Alvaro Herrera
Alvaro Herrera escribió: > So here are two patches -- the first one, for 9.3 and HEAD, introduce > the new aging variables and use them throughout vacuum and autovacuum, > including per-table options; the second one adjusts the struct > declarations to avoid the ABI break in VacuumStmt and StdRdOp

Re: [HACKERS] truncating pg_multixact/members

2014-02-13 Thread Andres Freund
On 2014-02-13 14:40:39 -0300, Alvaro Herrera wrote: > Andres Freund escribió: > > On 2014-02-12 17:40:44 -0300, Alvaro Herrera wrote: > > > > > Also, AutoVacOpts (used as part of reloptions) gained three extra > > > > > fields. Since this is in the middle of StdRdOptions, it'd be somewhat > > > >

Re: [HACKERS] truncating pg_multixact/members

2014-02-13 Thread Alvaro Herrera
Alvaro Herrera escribió: > Yes, that's what I did --- see the attached patch, which I would apply > on top of the code for master and would be only in 9.3. (Of course, these changes affect other parts of the code, in particular autovacuum.c and reloptions.c. But that's not important here). --

Re: [HACKERS] truncating pg_multixact/members

2014-02-13 Thread Alvaro Herrera
Andres Freund escribió: > On 2014-02-12 17:40:44 -0300, Alvaro Herrera wrote: > > > > Also, AutoVacOpts (used as part of reloptions) gained three extra > > > > fields. Since this is in the middle of StdRdOptions, it'd be somewhat > > > > more involve to put these at the end of that struct. This m

Re: [HACKERS] truncating pg_multixact/members

2014-02-13 Thread Andres Freund
On 2014-02-12 17:40:44 -0300, Alvaro Herrera wrote: > > > Also, AutoVacOpts (used as part of reloptions) gained three extra > > > fields. Since this is in the middle of StdRdOptions, it'd be somewhat > > > more involve to put these at the end of that struct. This might be a > > > problem if someb

Re: [HACKERS] truncating pg_multixact/members

2014-02-12 Thread Alvaro Herrera
Tom Lane escribió: > Alvaro Herrera writes: > > In this new version, I added a couple of fields to VacuumStmt node. How > > strongly do we feel this would cause an ABI break? Would we be more > > comfortable if I put them at the end of the struct for 9.3 instead? > > In the past we've usually a

Re: [HACKERS] truncating pg_multixact/members

2014-02-12 Thread Tom Lane
Alvaro Herrera writes: > In this new version, I added a couple of fields to VacuumStmt node. How > strongly do we feel this would cause an ABI break? Would we be more > comfortable if I put them at the end of the struct for 9.3 instead? In the past we've usually added such members at the end of

Re: [HACKERS] truncating pg_multixact/members

2014-02-12 Thread Alvaro Herrera
In this new version, I added a couple of fields to VacuumStmt node. How strongly do we feel this would cause an ABI break? Would we be more comfortable if I put them at the end of the struct for 9.3 instead? Do we expect third-party code to be calling vacuum()? Also, AutoVacOpts (used as part of

Re: [HACKERS] truncating pg_multixact/members

2014-02-11 Thread Robert Haas
On Tue, Feb 11, 2014 at 5:16 PM, Alvaro Herrera wrote: > Robert Haas escribió: >> On Mon, Jan 20, 2014 at 1:39 PM, Alvaro Herrera >> wrote: >> > * I haven't introduced settings to tweak this per table for >> > autovacuum. I don't think those are needed. It's not hard to do, >> > however; if peo

Re: [HACKERS] truncating pg_multixact/members

2014-01-21 Thread Robert Haas
On Mon, Jan 20, 2014 at 1:39 PM, Alvaro Herrera wrote: > * I haven't introduced settings to tweak this per table for > autovacuum. I don't think those are needed. It's not hard to do, > however; if people opine against this, I will implement that. I can't think of any reason to believe that it

Re: [HACKERS] truncating pg_multixact/members

2014-01-20 Thread Andres Freund
Hi, On 2014-01-20 15:39:33 -0300, Alvaro Herrera wrote: > * The multixact_freeze_table_age value has been set to 5 million. > I feel this is a big enough number that shouldn't cause too much > vacuuming churn, while at the same time not leaving excessive storage > occupied by pg_multixact/members,

Re: [HACKERS] truncating pg_multixact/members

2014-01-20 Thread Alvaro Herrera
Alvaro Herrera escribió: > Here's a first cut at this. Note I have omitted a setting equivalent to > autovacuum_freeze_max_age, but I think we should have one too. Some more comments on the patch: * I haven't introduced settings to tweak this per table for autovacuum. I don't think those are n

Re: [HACKERS] truncating pg_multixact/members

2014-01-20 Thread Alvaro Herrera
Robert Haas escribió: > On Fri, Jan 3, 2014 at 9:11 AM, Alvaro Herrera > wrote: > Yeah, this stuff is definitely underdocumented relative to vacuum right now. I have added a paragraph or two. It's a (probably insufficient) start. I would like to add a sample query to monitor usage, but I just

Re: [HACKERS] truncating pg_multixact/members

2014-01-07 Thread Andres Freund
On 2014-01-06 20:51:57 -0500, Robert Haas wrote: > On Mon, Jan 6, 2014 at 7:50 PM, Jim Nasby wrote: > > On 1/4/14, 8:19 AM, Robert Haas wrote: > >> Also, while multixactid_freeze_min_age should be low, perhaps a > >> million as you suggest, multixactid_freeze_table_age should NOT be > >> lowered t

Re: [HACKERS] truncating pg_multixact/members

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 7:50 PM, Jim Nasby wrote: > On 1/4/14, 8:19 AM, Robert Haas wrote: >> Also, while multixactid_freeze_min_age should be low, perhaps a >> million as you suggest, multixactid_freeze_table_age should NOT be >> lowered to 3 million or anything like it. If you do that, people wh

Re: [HACKERS] truncating pg_multixact/members

2014-01-06 Thread Jim Nasby
On 1/4/14, 8:19 AM, Robert Haas wrote: Also, while multixactid_freeze_min_age should be low, perhaps a million as you suggest, multixactid_freeze_table_age should NOT be lowered to 3 million or anything like it. If you do that, people who are actually doing lots of row locking will start getting

Re: [HACKERS] truncating pg_multixact/members

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 2:53 PM, Tom Lane wrote: > Robert Haas writes: >> On Sat, Jan 4, 2014 at 12:38 PM, Tom Lane wrote: >>> Keep in mind that 9.3 is still wet behind the ears and many many people >>> haven't adopted it yet. If we do what you're suggesting then we're >>> creating a completely

Re: [HACKERS] truncating pg_multixact/members

2014-01-06 Thread Tom Lane
Robert Haas writes: > On Sat, Jan 4, 2014 at 12:38 PM, Tom Lane wrote: >> Keep in mind that 9.3 is still wet behind the ears and many many people >> haven't adopted it yet. If we do what you're suggesting then we're >> creating a completely useless inconsistency that will nonetheless affect >> a

Re: [HACKERS] truncating pg_multixact/members

2014-01-06 Thread Robert Haas
On Sat, Jan 4, 2014 at 12:38 PM, Tom Lane wrote: > Robert Haas writes: >> As far as back-patching the GUCs, my thought would be to back-patch >> them but mark them GUC_NOT_IN_SAMPLE in 9.3, so we don't have to touch >> the default postgresql.conf. > > That seems bizarre and pointless. > > Keep in

Re: [HACKERS] truncating pg_multixact/members

2014-01-04 Thread Tom Lane
Robert Haas writes: > As far as back-patching the GUCs, my thought would be to back-patch > them but mark them GUC_NOT_IN_SAMPLE in 9.3, so we don't have to touch > the default postgresql.conf. That seems bizarre and pointless. Keep in mind that 9.3 is still wet behind the ears and many many peo

Re: [HACKERS] truncating pg_multixact/members

2014-01-04 Thread Robert Haas
On Fri, Jan 3, 2014 at 9:11 AM, Alvaro Herrera wrote: > Robert Haas escribió: >> On Mon, Dec 30, 2013 at 10:59 PM, Alvaro Herrera >> wrote: >> > One problem I see is length of time before freezing multis: they live >> > for far too long, causing the SLRU files to eat way too much disk space. >> >

Re: [HACKERS] truncating pg_multixact/members

2014-01-03 Thread Andres Freund
Hi, On 2014-01-03 11:11:13 -0300, Alvaro Herrera wrote: > > Yeah. Since we expect mxids to be composed at a much lower rate than > > xids, we can keep pg_multixact small without needing to increase the > > rate of full table scans. I don't think that's necessarily true - there have been several

Re: [HACKERS] truncating pg_multixact/members

2014-01-03 Thread Alvaro Herrera
Robert Haas escribió: > On Mon, Dec 30, 2013 at 10:59 PM, Alvaro Herrera > wrote: > > One problem I see is length of time before freezing multis: they live > > for far too long, causing the SLRU files to eat way too much disk space. > > I ran burnmulti in a loop, creating multis of 3 members each,

Re: [HACKERS] truncating pg_multixact/members

2014-01-02 Thread Robert Haas
On Mon, Dec 30, 2013 at 10:59 PM, Alvaro Herrera wrote: > One problem I see is length of time before freezing multis: they live > for far too long, causing the SLRU files to eat way too much disk space. > I ran burnmulti in a loop, creating multis of 3 members each, with a min > freeze age of 50 m

Re: [HACKERS] truncating pg_multixact/members

2013-12-30 Thread Alvaro Herrera
Alvaro Herrera wrote: > 1. slru.c doesn't consider file names longer than 4 hexadecimal chars. > For 9.3, I propose we skip this and tweak the code to consider files > whose names are 4 or 5 chars in length, to remain compatible with > existing installations that have pg_multixact/member having a

Re: [HACKERS] truncating pg_multixact/members

2013-12-28 Thread Alvaro Herrera
Kevin Grittner wrote: > Alvaro Herrera wrote: > > > 1. slru.c doesn't consider file names longer than 4 hexadecimal chars. > > > Fixing (1) is simple: we can have each SLRU user declare how many digits > > to have in file names.  All existing users but pg_multixact/members > > should declare 4 d

Re: [HACKERS] truncating pg_multixact/members

2013-12-28 Thread Kevin Grittner
Alvaro Herrera wrote: > 1. slru.c doesn't consider file names longer than 4 hexadecimal chars. > Fixing (1) is simple: we can have each SLRU user declare how many digits > to have in file names.  All existing users but pg_multixact/members > should declare 4 digits; that one should declare 5.  T