Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-17 Thread Royce Ausburn
On 18/11/2011, at 10:44 AM, Robert Haas wrote: > On Thu, Nov 17, 2011 at 5:49 PM, Royce Ausburn wrote: >> Thanks for the discussion so far all. Would it be worthwhile to make >> another patch that addresses the points from Yeb's reviews? It's not >> sounding like this unremovable tuple count

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-17 Thread Robert Haas
On Thu, Nov 17, 2011 at 5:49 PM, Royce Ausburn wrote: > Thanks for the discussion so far all.  Would it be worthwhile to make another > patch that addresses the points from Yeb's reviews?  It's not sounding like > this unremovable tuple count is something that postgres wants, but I'm happy > to

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-17 Thread Royce Ausburn
On 17/11/2011, at 1:47 AM, Robert Haas wrote: > On Wed, Nov 16, 2011 at 9:34 AM, Tom Lane wrote: >> Robert Haas writes: >>> Not sure about the log line, but allowing pgstattuple to distinguish >>> between recently-dead and quite-thoroughly-dead seems useful. >> >> The dividing line is enormous

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-16 Thread Tom Lane
Yeb Havinga writes: > On 2011-11-16 15:34, Tom Lane wrote: >> The dividing line is enormously unstable though. pgstattuple's idea of >> RecentGlobalXmin could even be significantly different from that of a >> concurrently running VACUUM. I can see the point of having VACUUM log >> what it did, b

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-16 Thread Robert Haas
On Wed, Nov 16, 2011 at 9:34 AM, Tom Lane wrote: > Robert Haas writes: >> Not sure about the log line, but allowing pgstattuple to distinguish >> between recently-dead and quite-thoroughly-dead seems useful. > > The dividing line is enormously unstable though.  pgstattuple's idea of > RecentGloba

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-16 Thread Yeb Havinga
On 2011-11-16 15:34, Tom Lane wrote: Robert Haas writes: Not sure about the log line, but allowing pgstattuple to distinguish between recently-dead and quite-thoroughly-dead seems useful. The dividing line is enormously unstable though. pgstattuple's idea of RecentGlobalXmin could even be sig

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-16 Thread Tom Lane
Robert Haas writes: > Not sure about the log line, but allowing pgstattuple to distinguish > between recently-dead and quite-thoroughly-dead seems useful. The dividing line is enormously unstable though. pgstattuple's idea of RecentGlobalXmin could even be significantly different from that of a

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-16 Thread Robert Haas
On Tue, Nov 15, 2011 at 10:02 PM, Royce Ausburn wrote: > Fair enough -- someone knowledgable could set that up if they wanted.  My > goal was mostly to have something helpful in the logs.  If that's not > something postgres wants/needs Ill drop it =) IMHO, it's generally not desirable to provid

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-16 Thread Robert Haas
On Wed, Nov 16, 2011 at 3:31 AM, Yeb Havinga wrote: > Apparently pg_stat* counts the recently_dead tuple under n_live_tup, else 2 > is a wrong number, where pgstattuple counts recently_dead under > dead_tuple_count. This could be a source of confusion. If there is any > serious work considered her

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-16 Thread Yeb Havinga
On 2011-11-15 22:04, Tom Lane wrote: Robert Haas writes: Oh. I was thinking "dead" meant "no longer visible to anyone". But it sounds what we call "unremovable" here is what we elsewhere call "recently dead". Would have to look at the code to be sure, but I think that "nonremovable" is mean

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-15 Thread Royce Ausburn
On 16/11/2011, at 12:26 PM, Robert Haas wrote: > On Tue, Nov 15, 2011 at 7:59 PM, Royce Ausburn wrote: >>> Personally I think some log output, done better, would have been more >>> useful for me at the time. At the time I was trying to diagnose an >>> ineffective vacuum and postgres' logs wer

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-15 Thread Robert Haas
On Tue, Nov 15, 2011 at 7:59 PM, Royce Ausburn wrote: >> Personally I think some log output, done better, would have been more useful >> for me at the time.  At the time I was trying to diagnose an ineffective >> vacuum and postgres' logs weren't giving me any hints about what was wrong.   >> I

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-15 Thread Royce Ausburn
> > > Personally I think some log output, done better, would have been more useful > for me at the time. At the time I was trying to diagnose an ineffective > vacuum and postgres' logs weren't giving me any hints about what was wrong. > I turned to the mailing list and got immediate help, bu

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-15 Thread Royce Ausburn
On 16/11/2011, at 8:04 AM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Nov 15, 2011 at 10:29 AM, Alvaro Herrera >> wrote: >>> Excerpts from Robert Haas's message of mar nov 15 12:16:54 -0300 2011: I guess this is a dumb question, but why don't we remove all the dead tuples? > >>

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-15 Thread Royce Ausburn
On 16/11/2011, at 2:05 AM, Yeb Havinga wrote: > On 2011-10-05 00:45, Royce Ausburn wrote: >> Attached is v2 of my patch that doesn't update CATALOG_VERSION_NO. I've >> also fixed the name of an argument to pgstat_report_vacuum which I don't >> think was particularly good, and I've replace the

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-15 Thread Tom Lane
Robert Haas writes: > On Tue, Nov 15, 2011 at 10:29 AM, Alvaro Herrera > wrote: >> Excerpts from Robert Haas's message of mar nov 15 12:16:54 -0300 2011: >>> I guess this is a dumb question, but why don't we remove all the dead >>> tuples? >> They were deleted but there are transactions with old

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-15 Thread Christopher Browne
On Tue, Nov 15, 2011 at 1:33 PM, Robert Haas wrote: > "nondeletable" is surely terrible, since they may well have got into > this state by being deleted.  "nonremovable" is better, but still not > great. Bit of brain storm, including looking over to terminology used for garbage collection: - stil

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-15 Thread Robert Haas
On Tue, Nov 15, 2011 at 10:29 AM, Alvaro Herrera wrote: > Excerpts from Robert Haas's message of mar nov 15 12:16:54 -0300 2011: >> On Tue, Nov 15, 2011 at 10:05 AM, Yeb Havinga wrote: >> > I reviewed your patch. I think it is in good shape, my two main remarks >> > (name of n_unremovable_tup and

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-15 Thread Greg Smith
On 11/15/2011 10:29 AM, Alvaro Herrera wrote: They were deleted but there are transactions with older snapshots. I think vacuum uses the term "nondeletable" or "nonremovable". Not sure which one is less bad. Not being a native speaker, they all sound horrible to me. I would go more for so

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-15 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mar nov 15 12:16:54 -0300 2011: > > On Tue, Nov 15, 2011 at 10:05 AM, Yeb Havinga wrote: > > I reviewed your patch. I think it is in good shape, my two main remarks > > (name of n_unremovable_tup and a remark about documentation at the end of > > this revie

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-15 Thread Yeb Havinga
On 2011-11-15 16:16, Robert Haas wrote: On Tue, Nov 15, 2011 at 10:05 AM, Yeb Havinga wrote: I reviewed your patch. I think it is in good shape, my two main remarks (name of n_unremovable_tup and a remark about documentation at the end of this review) are highly subjective and I wouldn't spend

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-15 Thread Robert Haas
On Tue, Nov 15, 2011 at 10:05 AM, Yeb Havinga wrote: > I reviewed your patch. I think it is in good shape, my two main remarks > (name of n_unremovable_tup and a remark about documentation at the end of > this review) are highly subjective and I wouldn't spend time on it unless > other people have

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-11-15 Thread Yeb Havinga
On 2011-10-05 00:45, Royce Ausburn wrote: Attached is v2 of my patch that doesn't update CATALOG_VERSION_NO. I've also fixed the name of an argument to pgstat_report_vacuum which I don't think was particularly good, and I've replace the word "tuple" with "row" in some docs I added for consiste

Re: [HACKERS] [PATCH] Unremovable tuple monitoring (was: Addition of some trivial auto vacuum logging)

2011-10-05 Thread Robert Haas
On Tue, Oct 4, 2011 at 6:45 PM, Royce Ausburn wrote: > I'm not sure what my next step should be.  I've added this patch to the open > commit fest -- is that all for now until the commit fest begins review? Yep, except that it might be nice if you could volunteer to review someone else's patch in

Re: [HACKERS] [PATCH] Unremovable tuple monitoring

2011-10-04 Thread Greg Smith
On 10/04/2011 03:45 PM, Royce Ausburn wrote: I think I get this stats stuff now. Unless someone here thinks it's too hard for a new postgres dev's 2nd patch, I could take a stab. I might take a look at it tonight to get a feel for how hard, and what stats we could collect. I'll start a new thre

Re: [HACKERS] [PATCH] Unremovable tuple monitoring (was: Addition of some trivial auto vacuum logging)

2011-10-04 Thread Royce Ausburn
On 04/10/2011, at 11:26 PM, Robert Haas wrote: > On Mon, Oct 3, 2011 at 9:17 AM, Royce Ausburn wrote: >> - I'm not sure if I'm supposed to update CATALOG_VERSION_NO in catalog.h. >> In this patch I have. > > Generally that is left to the committer, as the correct value depends > on the value

Re: [HACKERS] [PATCH] Unremovable tuple monitoring (was: Addition of some trivial auto vacuum logging)

2011-10-04 Thread Robert Haas
On Mon, Oct 3, 2011 at 9:17 AM, Royce Ausburn wrote: > - I'm not sure if I'm supposed to update CATALOG_VERSION_NO in catalog.h.  In > this patch I have. Generally that is left to the committer, as the correct value depends on the value at the time of commit, not the time you submit the patch; a

Re: [HACKERS] [PATCH] Unremovable tuple monitoring (was: Addition of some trivial auto vacuum logging)

2011-10-03 Thread Royce Ausburn
On 28/09/2011, at 11:17 AM, Tom Lane wrote: > Alvaro Herrera writes: >> Excerpts from Royce Ausburn's message of mar sep 27 21:28:26 -0300 2011: >>> Tom's suggestion looks like it's less trivial that I can do just yet, but >>> I'll take a look and ask for help if I need it. > >> It's not that d