Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-05-12 Thread Andres Freund
On 2015-05-12 00:42:14 +0530, Abhijit Menon-Sen wrote: At 2015-05-11 19:15:47 +0200, and...@anarazel.de wrote: I don't really care how it's named, as long as it makes clear that it's not an exact measurement. Not having heard any better suggestions, I picked pgstatapprox as a compromise

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-05-12 Thread Abhijit Menon-Sen
At 2015-05-13 03:04:11 +0200, and...@anarazel.de wrote: I can live with that, although I'd personally go with pgstattuple_approx()... I could live with that too. Here's an incremental patch to rename the function. (I'm not resubmitting the whole thing since you said you've made some other

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-05-11 Thread Abhijit Menon-Sen
Hi Andres. I've attached an updated patch for pgstatbloat, as well as a patch to replace two uses of BuildTupleFromCStrings() elsewhere in pgstattuple. I've made the changes you mentioned in your earlier mail, except that I have not changed the name pending further suggestions about what would

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-05-11 Thread Abhijit Menon-Sen
At 2015-05-11 19:15:47 +0200, and...@anarazel.de wrote: TBH, I'd rather not touch unrelated things right now. We're pretty badly behind... OK. That patch is independent; just ignore it. I don't really care how it's named, as long as it makes clear that it's not an exact measurement. Not

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-05-11 Thread Andres Freund
Hi, On 2015-05-11 16:57:08 +0530, Abhijit Menon-Sen wrote: I've attached an updated patch for pgstatbloat, as well as a patch to replace two uses of BuildTupleFromCStrings() elsewhere in pgstattuple. TBH, I'd rather not touch unrelated things right now. We're pretty badly behind... I've made

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-05-09 Thread Abhijit Menon-Sen
At 2015-05-09 02:20:51 +0200, and...@anarazel.de wrote: + * Abhijit Menon-Sen a...@2ndquadrant.com + * Portions Copyright (c) 2001,2002Tatsuo Ishii (from pgstattuple) I think for new extension we don't really add authors and such anymore. OK, I'll get rid of the boilerplate. Hm.

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-05-08 Thread Andres Freund
Hi, On 2015-04-24 08:46:48 +0530, Abhijit Menon-Sen wrote: diff --git a/contrib/pgstattuple/pgstatbloat.c b/contrib/pgstattuple/pgstatbloat.c new file mode 100644 index 000..612e22b --- /dev/null +++ b/contrib/pgstattuple/pgstatbloat.c @@ -0,0 +1,389 @@ +/* + *

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-25 Thread Amit Kapila
On Fri, Apr 24, 2015 at 8:05 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: On 04/24/15 14:58, Amit Kapila wrote: On Fri, Apr 24, 2015 at 8:46 AM, Abhijit Menon-Sen a...@2ndquadrant.com mailto:a...@2ndquadrant.com wrote: At 2015-04-24 08:35:40 +0530, amit.kapil...@gmail.com

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-24 Thread Amit Kapila
On Fri, Apr 24, 2015 at 8:46 AM, Abhijit Menon-Sen a...@2ndquadrant.com wrote: At 2015-04-24 08:35:40 +0530, amit.kapil...@gmail.com wrote: Just stick a PG_RETURN_NULL() at the end? That should also work. OK, updated patch attached with just that one change. Patch looks good to me,

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-24 Thread Tomas Vondra
On 04/24/15 14:58, Amit Kapila wrote: On Fri, Apr 24, 2015 at 8:46 AM, Abhijit Menon-Sen a...@2ndquadrant.com mailto:a...@2ndquadrant.com wrote: At 2015-04-24 08:35:40 +0530, amit.kapil...@gmail.com mailto:amit.kapil...@gmail.com wrote: Just stick a PG_RETURN_NULL() at the end?

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-23 Thread Amit Kapila
On Wed, Apr 22, 2015 at 6:33 PM, Abhijit Menon-Sen a...@2ndquadrant.com wrote: At 2015-04-18 12:28:36 +0530, amit.kapil...@gmail.com wrote: I think you have missed to address the below point: 4) prefetch Updated patch attached, as well as the diff against the earlier version just to

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-23 Thread Abhijit Menon-Sen
At 2015-04-24 07:22:27 +0530, amit.kapil...@gmail.com wrote: Few minor issues: 1. warning on windows \pgstatbloat.c(193): warning C4715: 'pgstatbloat' : not all control paths return a value This is because the function ends with an ereport(ERROR, …). What would you suggest here? Just

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-23 Thread Amit Kapila
On Fri, Apr 24, 2015 at 8:04 AM, Abhijit Menon-Sen a...@2ndquadrant.com wrote: At 2015-04-24 07:22:27 +0530, amit.kapil...@gmail.com wrote: Few minor issues: 1. warning on windows \pgstatbloat.c(193): warning C4715: 'pgstatbloat' : not all control paths return a value This is

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-23 Thread Abhijit Menon-Sen
At 2015-04-24 08:35:40 +0530, amit.kapil...@gmail.com wrote: Just stick a PG_RETURN_NULL() at the end? That should also work. OK, updated patch attached with just that one change. I'm not doing anything about the rename. I don't know if it's possible to make patch(1) rename a file at all

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-22 Thread Abhijit Menon-Sen
At 2015-04-18 12:28:36 +0530, amit.kapil...@gmail.com wrote: I think you have missed to address the below point: 4) prefetch Updated patch attached, as well as the diff against the earlier version just to make it easier to see the exact change I made (which is to copy the skip logic from

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-18 Thread Amit Kapila
On Fri, Apr 3, 2015 at 9:07 PM, Abhijit Menon-Sen a...@2ndquadrant.com wrote: At 2015-03-31 22:43:49 +0530, a...@2ndquadrant.com wrote: I'm just posting this WIP patch where I've renamed fastbloat to pgstatbloat as suggested by Tomas, and added in the documentation, and so on. Here's

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-04-03 Thread Abhijit Menon-Sen
At 2015-03-31 22:43:49 +0530, a...@2ndquadrant.com wrote: I'm just posting this WIP patch where I've renamed fastbloat to pgstatbloat as suggested by Tomas, and added in the documentation, and so on. Here's the revised version that also adds the count of RECENTLY_DEAD and

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-03-31 Thread Abhijit Menon-Sen
Hi. I'm just posting this WIP patch where I've renamed fastbloat to pgstatbloat as suggested by Tomas, and added in the documentation, and so on. I still have to incorporate Amit's comments about the estimation of reltuples according to the way vacuum does it, and I expect to post that tomorrow

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-03-03 Thread Amit Kapila
On Fri, Dec 26, 2014 at 1:08 PM, Abhijit Menon-Sen a...@2ndquadrant.com wrote: At 2014-09-25 15:40:11 +0530, a...@2ndquadrant.com wrote: All right, then I'll post a version that addresses Amit's other points, adds a new file/function to pgstattuple, acquires content locks, and uses

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-03-03 Thread Amit Kapila
On Mon, Feb 23, 2015 at 7:11 AM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: On 28.1.2015 05:03, Abhijit Menon-Sen wrote: At 2015-01-27 17:00:27 -0600, jim.na...@bluetreble.com wrote: Otherwise, the code looks OK to me. Now, there are a few features I'd like to have for production use

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-02-25 Thread Tomas Vondra
On 24.2.2015 19:08, Jim Nasby wrote: On 2/22/15 8:32 PM, Tomas Vondra wrote: On 23.2.2015 03:20, Jim Nasby wrote: On 2/22/15 5:41 PM, Tomas Vondra wrote: Otherwise, the code looks OK to me. Now, there are a few features I'd like to have for production use (to minimize the impact): 1) no

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-02-25 Thread Alvaro Herrera
Mark Kirkwood wrote: On 26/02/15 13:32, Simon Riggs wrote: On 25 February 2015 at 23:30, Jim Nasby jim.na...@bluetreble.com wrote: That said, I don't want to block this; I think it's useful. Though, perhaps it would be better as an extension instead of in contrib? I don't think it should be

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-02-25 Thread Simon Riggs
On 25 February 2015 at 23:30, Jim Nasby jim.na...@bluetreble.com wrote: That said, I don't want to block this; I think it's useful. Though, perhaps it would be better as an extension instead of in contrib? I don't think it should be very version dependent? The whole point of this is to get it

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-02-25 Thread Jim Nasby
On 2/25/15 2:56 PM, Tomas Vondra wrote: On 24.2.2015 19:08, Jim Nasby wrote: On 2/22/15 8:32 PM, Tomas Vondra wrote: On 23.2.2015 03:20, Jim Nasby wrote: On 2/22/15 5:41 PM, Tomas Vondra wrote: Otherwise, the code looks OK to me. Now, there are a few features I'd like to have for production

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-02-25 Thread Mark Kirkwood
On 26/02/15 13:32, Simon Riggs wrote: On 25 February 2015 at 23:30, Jim Nasby jim.na...@bluetreble.com wrote: That said, I don't want to block this; I think it's useful. Though, perhaps it would be better as an extension instead of in contrib? I don't think it should be very version dependent?

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-02-24 Thread Jim Nasby
On 2/22/15 8:32 PM, Tomas Vondra wrote: On 23.2.2015 03:20, Jim Nasby wrote: On 2/22/15 5:41 PM, Tomas Vondra wrote: Otherwise, the code looks OK to me. Now, there are a few features I'd like to have for production use (to minimize the impact): 1) no index support:-( I'd like to see

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-02-22 Thread Tomas Vondra
Hi! On 28.1.2015 05:03, Abhijit Menon-Sen wrote: At 2015-01-27 17:00:27 -0600, jim.na...@bluetreble.com wrote: It would be best to get this into a commit fest so it's not lost. It's there already. -- Abhijit I looked at this patch today, so a few comments from me: 1) I believe the

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-02-22 Thread Tomas Vondra
On 23.2.2015 03:20, Jim Nasby wrote: On 2/22/15 5:41 PM, Tomas Vondra wrote: Otherwise, the code looks OK to me. Now, there are a few features I'd like to have for production use (to minimize the impact): 1) no index support:-( I'd like to see support for more relation types (at least

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-02-22 Thread Jim Nasby
On 2/22/15 5:41 PM, Tomas Vondra wrote: Otherwise, the code looks OK to me. Now, there are a few features I'd like to have for production use (to minimize the impact): 1) no index support:-( I'd like to see support for more relation types (at least btree indexes). Are there any plans

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-01-27 Thread Abhijit Menon-Sen
At 2015-01-26 18:47:29 -0600, jim.na...@bluetreble.com wrote: Anything happen with this? Nothing so far. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-01-27 Thread Jim Nasby
On 1/27/15 3:56 AM, Abhijit Menon-Sen wrote: At 2015-01-26 18:47:29 -0600, jim.na...@bluetreble.com wrote: Anything happen with this? Nothing so far. It would be best to get this into a commit fest so it's not lost. -- Jim Nasby, Data Architect, Blue Treble Consulting Data in Trouble? Get

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-01-27 Thread Abhijit Menon-Sen
At 2015-01-27 17:00:27 -0600, jim.na...@bluetreble.com wrote: It would be best to get this into a commit fest so it's not lost. It's there already. -- Abhijit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-01-26 Thread Jim Nasby
On 12/26/14 1:38 AM, Abhijit Menon-Sen wrote: At 2014-09-25 15:40:11 +0530,a...@2ndquadrant.com wrote: All right, then I'll post a version that addresses Amit's other points, adds a new file/function to pgstattuple, acquires content locks, and uses HeapTupleSatisfiesVacuum, hint-bit setting

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-12-25 Thread Abhijit Menon-Sen
At 2014-09-25 15:40:11 +0530, a...@2ndquadrant.com wrote: All right, then I'll post a version that addresses Amit's other points, adds a new file/function to pgstattuple, acquires content locks, and uses HeapTupleSatisfiesVacuum, hint-bit setting and all. Sorry, I forgot to post this patch.

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-09-26 Thread Amit Kapila
On Wed, Sep 24, 2014 at 2:26 PM, Abhijit Menon-Sen a...@2ndquadrant.com wrote: Hi Amit. Thanks for your comments, and I'm sorry it's taken me so long to respond. No issues. At 2014-08-03 11:18:57 +0530, amit.kapil...@gmail.com wrote: 7. HeapTupleSatisfiesVacuumNoHint() a. Why can't

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-09-25 Thread Andres Freund
On 2014-09-25 10:24:39 +0530, Abhijit Menon-Sen wrote: At 2014-09-24 11:09:24 +0200, and...@2ndquadrant.com wrote: I think it's completely unacceptable to copy a visibility routine. OK. Which visibility routine should I use, and should I try to create a variant that doesn't set hint bits?

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-09-25 Thread Abhijit Menon-Sen
At 2014-09-25 11:41:29 +0200, and...@2ndquadrant.com wrote: I've not yet followed your premise that you actually need one that doesn't set hint bits... Oh. All right, then I'll post a version that addresses Amit's other points, adds a new file/function to pgstattuple, acquires content locks,

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-09-25 Thread Simon Riggs
On 25 September 2014 10:41, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-25 10:24:39 +0530, Abhijit Menon-Sen wrote: At 2014-09-24 11:09:24 +0200, and...@2ndquadrant.com wrote: I think it's completely unacceptable to copy a visibility routine. OK. Which visibility routine should I

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-09-25 Thread Andres Freund
On 2014-09-25 14:43:14 +0100, Simon Riggs wrote: On 25 September 2014 10:41, Andres Freund and...@2ndquadrant.com wrote: On 2014-09-25 10:24:39 +0530, Abhijit Menon-Sen wrote: At 2014-09-24 11:09:24 +0200, and...@2ndquadrant.com wrote: I think it's completely unacceptable to copy a

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-09-24 Thread Abhijit Menon-Sen
Hi Amit. Thanks for your comments, and I'm sorry it's taken me so long to respond. At 2014-08-03 11:18:57 +0530, amit.kapil...@gmail.com wrote: After looking at code, I also felt that it is better to add this as a version of pg_stattuple. I started off trying to do that, but now I'm afraid I

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-09-24 Thread Andres Freund
Hi, On 2014-09-24 14:26:37 +0530, Abhijit Menon-Sen wrote: Thanks for your comments, and I'm sorry it's taken me so long to respond. At 2014-08-03 11:18:57 +0530, amit.kapil...@gmail.com wrote: After looking at code, I also felt that it is better to add this as a version of

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-09-24 Thread Abhijit Menon-Sen
At 2014-09-24 11:09:24 +0200, and...@2ndquadrant.com wrote: Why not add it to the stattuple extension, but as an independent function (and file)? Thanks, that's a good idea. I'll do that. I think it's completely unacceptable to copy a visibility routine. OK. Which visibility routine should

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-08-02 Thread Amit Kapila
On Thu, Apr 3, 2014 at 3:11 AM, Abhijit Menon-Sen a...@2ndquadrant.com wrote: This is a follow-up to the thread at http://www.postgresql.org/message-id/4eb5fa1b.1090...@2ndquadrant.com A quick summary: that thread proposed adding a relation_free_space() function to the pageinspect extension.

[HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-04-02 Thread Abhijit Menon-Sen
This is a follow-up to the thread at http://www.postgresql.org/message-id/4eb5fa1b.1090...@2ndquadrant.com A quick summary: that thread proposed adding a relation_free_space() function to the pageinspect extension. Various review comments were received, among which was the suggestion that the

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2014-04-02 Thread Robert Haas
On Wed, Apr 2, 2014 at 5:41 PM, Abhijit Menon-Sen a...@2ndquadrant.com wrote: I've attached an extension that produces largely pgstattuple-compatible numbers for a table without doing a full-table scan. It scans through the table, skipping blocks that have their visibility map bit set. For