Re: [HACKERS] Measuring relation free space

2012-03-13 Thread Robert Haas
On Mon, Mar 12, 2012 at 11:10 PM, Jaime Casanova ja...@2ndquadrant.com wrote: On Mon, Mar 12, 2012 at 9:41 PM, Noah Misch n...@leadboat.com wrote: I created a CF entry for this and marked it Ready for Committer. i wasn't sure if create an entry this late was a good idea or not... but now i

Re: [HACKERS] Measuring relation free space

2012-03-12 Thread Noah Misch
On Fri, Mar 09, 2012 at 02:18:02AM -0500, Jaime Casanova wrote: On Wed, Feb 22, 2012 at 12:27 AM, Noah Misch n...@leadboat.com wrote: On Tue, Feb 14, 2012 at 02:04:26AM -0500, Jaime Casanova wrote: 1) pgstattuple-gin_spgist.patch This first patch adds gin and spgist support to

Re: [HACKERS] Measuring relation free space

2012-03-12 Thread Jaime Casanova
On Mon, Mar 12, 2012 at 9:41 PM, Noah Misch n...@leadboat.com wrote: I created a CF entry for this and marked it Ready for Committer. i wasn't sure if create an entry this late was a good idea or not... but now i feel better because is less probable that it will fall out on the cracks, thanks

Re: [HACKERS] Measuring relation free space

2012-03-08 Thread Jaime Casanova
On Wed, Feb 22, 2012 at 12:27 AM, Noah Misch n...@leadboat.com wrote: On Tue, Feb 14, 2012 at 02:04:26AM -0500, Jaime Casanova wrote: 1) pgstattuple-gin_spgist.patch This first patch adds gin and spgist support to pgstattuple, also makes pgstattuple use a ring buffer when reading tables or

Re: [HACKERS] Measuring relation free space

2012-02-21 Thread Noah Misch
On Tue, Feb 14, 2012 at 02:04:26AM -0500, Jaime Casanova wrote: On Wed, Jan 25, 2012 at 9:47 PM, Noah Misch n...@leadboat.com wrote: With all that done, run some quick benchmarks: see how SELECT free_percent FROM pgstattuple(rel) fares compared to SELECT relation_free_space(rel) for a

Re: [HACKERS] Measuring relation free space

2012-02-13 Thread Jaime Casanova
On Wed, Jan 25, 2012 at 9:47 PM, Noah Misch n...@leadboat.com wrote: With all that done, run some quick benchmarks: see how SELECT free_percent FROM pgstattuple(rel) fares compared to SELECT relation_free_space(rel) for a large heap and for a large B-tree index.  If the timing difference is

Re: [HACKERS] Measuring relation free space

2012-01-25 Thread Noah Misch
On Tue, Jan 24, 2012 at 11:24:08AM -0500, Jaime Casanova wrote: On Mon, Jan 23, 2012 at 7:18 PM, Noah Misch n...@leadboat.com wrote: If someone feels like doing it, +1 for making pgstattuple() count non-leaf free space. actually i agreed that non-leaf pages are irrelevant... i just

Re: [HACKERS] Measuring relation free space

2012-01-24 Thread Jaime Casanova
On Mon, Jan 23, 2012 at 7:18 PM, Noah Misch n...@leadboat.com wrote: On Mon, Jan 23, 2012 at 04:56:24PM -0300, Alvaro Herrera wrote: Hm.  Leaf pages hold as much tuples as non-leaf pages, no?  I mean for each page element there's a value and a CTID.  In non-leaf those CTIDs point to other

Re: [HACKERS] Measuring relation free space

2012-01-23 Thread Alvaro Herrera
Excerpts from Noah Misch's message of vie ene 20 22:33:30 -0300 2012: On Fri, Jan 20, 2012 at 07:03:22PM -0500, Jaime Casanova wrote: On Wed, Jan 18, 2012 at 7:01 PM, Noah Misch n...@leadboat.com wrote: On Wed, Jan 18, 2012 at 09:46:20AM -0500, Jaime Casanova wrote: ignoring all

Re: [HACKERS] Measuring relation free space

2012-01-23 Thread Noah Misch
On Mon, Jan 23, 2012 at 04:56:24PM -0300, Alvaro Herrera wrote: Excerpts from Noah Misch's message of vie ene 20 22:33:30 -0300 2012: pgstattuple() figures the free_percent by adding up all space available to hold tuples and dividing that by the simple size of the relation. Non-leaf pages

Re: [HACKERS] Measuring relation free space

2012-01-20 Thread Jaime Casanova
On Wed, Jan 18, 2012 at 7:01 PM, Noah Misch n...@leadboat.com wrote: On Wed, Jan 18, 2012 at 09:46:20AM -0500, Jaime Casanova wrote: ignoring all non-leaf pages still gives a considerable difference between pgstattuple and relation_free_space() pgstattuple() counts the single B-tree meta

Re: [HACKERS] Measuring relation free space

2012-01-20 Thread Noah Misch
On Fri, Jan 20, 2012 at 07:03:22PM -0500, Jaime Casanova wrote: On Wed, Jan 18, 2012 at 7:01 PM, Noah Misch n...@leadboat.com wrote: On Wed, Jan 18, 2012 at 09:46:20AM -0500, Jaime Casanova wrote: ignoring all non-leaf pages still gives a considerable difference between pgstattuple and

Re: [HACKERS] Measuring relation free space

2012-01-18 Thread Jaime Casanova
On Mon, Jan 16, 2012 at 5:09 AM, Noah Misch n...@leadboat.com wrote: pgstattuple()'s decision to treat half-dead pages like deleted pages is better.  That transient state can only end in the page's deletion. the only page in that index has 200 records (all live 0 dead) using half the page

Re: [HACKERS] Measuring relation free space

2012-01-18 Thread Noah Misch
On Wed, Jan 18, 2012 at 09:46:20AM -0500, Jaime Casanova wrote: On Mon, Jan 16, 2012 at 5:09 AM, Noah Misch n...@leadboat.com wrote: pgstattuple()'s decision to treat half-dead pages like deleted pages is better. ?That transient state can only end in the page's deletion. the only page

Re: [HACKERS] Measuring relation free space

2012-01-16 Thread Noah Misch
On Sat, Jan 14, 2012 at 02:40:46PM -0500, Jaime Casanova wrote: On Sat, Jan 14, 2012 at 6:26 AM, Noah Misch n...@leadboat.com wrote: - pgstattuple() and relation_free_space() should emit the same number, even if ?that means improving pgstattuple() at the same time. yes, i just wanted

Re: [HACKERS] Measuring relation free space

2012-01-14 Thread Jaime Casanova
On Thu, Dec 15, 2011 at 4:11 PM, Noah Misch n...@leadboat.com wrote: On Sun, Nov 06, 2011 at 10:20:49PM +0100, Bernd Helmle wrote: --On 6. November 2011 01:08:11 -0200 Greg Smith g...@2ndquadrant.com wrote: Attached patch adds a new function to the pageinspect extension for measuring total

Re: [HACKERS] Measuring relation free space

2012-01-14 Thread Noah Misch
On Sat, Jan 14, 2012 at 04:41:57AM -0500, Jaime Casanova wrote: pgstattuple and relation_free_space are very close in all the numbers except for 2 indexes pgbench_branches_pkey and pgbench_tellers_pkey; after a VACUUM FULL and a REINDEX (and the difference persistence) i checked

Re: [HACKERS] Measuring relation free space

2012-01-14 Thread Jaime Casanova
On Sat, Jan 14, 2012 at 6:26 AM, Noah Misch n...@leadboat.com wrote: - pgstattuple() and relation_free_space() should emit the same number, even if  that means improving pgstattuple() at the same time. yes, i just wanted to understand which one was more accurate and why... and give the

Re: [HACKERS] Measuring relation free space

2011-12-18 Thread Noah Misch
On Fri, Dec 16, 2011 at 02:02:03AM -0500, Greg Smith wrote: On 12/15/2011 04:11 PM, Noah Misch wrote: Is one of those index figures simply wrong, or do they measure two senses of free space, both of which are interesting to DBAs? I think the bigger one--the one I was aiming to measure--also

Re: [HACKERS] Measuring relation free space

2011-12-15 Thread Greg Smith
On 11/28/2011 05:40 AM, Greg Smith wrote: Ignoring fillfactor seems to have even more downsides as I see it. Certainly deserves a doc improvement, as well as fixing the description of the value so it's clearly a ratio rather than a true percentage. So: I'm very clear on what to do here

Re: [HACKERS] Measuring relation free space

2011-12-15 Thread Noah Misch
On Sun, Nov 06, 2011 at 10:20:49PM +0100, Bernd Helmle wrote: --On 6. November 2011 01:08:11 -0200 Greg Smith g...@2ndquadrant.com wrote: Attached patch adds a new function to the pageinspect extension for measuring total free space, in either tables or indexes. I wonder if that should be

Re: [HACKERS] Measuring relation free space

2011-12-15 Thread Greg Smith
On 12/15/2011 04:11 PM, Noah Misch wrote: Is one of those index figures simply wrong, or do they measure two senses of free space, both of which are interesting to DBAs? I think the bigger one--the one I was aiming to measure--also includes fill-factor space. It should be possible to

Re: [HACKERS] Measuring relation free space

2011-12-07 Thread Jaime Casanova
On Mon, Nov 28, 2011 at 5:40 AM, Greg Smith g...@2ndquadrant.com wrote: Unless I am missing something, all indexes are handled via a procedure designed for BTree indices, GetBTRelationFreeSpace.  I don't know that the ultimate behavior of this is wrong, but it seems unusual.  If I get some

Re: [HACKERS] Measuring relation free space

2011-11-28 Thread Greg Smith
On 11/25/2011 04:42 PM, Jeff Janes wrote: It reports space that is free exclusively for updates as being free. In other words, it considers space free even if it is reserved against inserts in deference to fillfactor. This is in contrast to pg_freespace, which only reports space available for

Re: [HACKERS] Measuring relation free space

2011-11-25 Thread Jeff Janes
On Mon, Nov 14, 2011 at 2:02 PM, Jaime Casanova ja...@2ndquadrant.com wrote: On Wed, Nov 9, 2011 at 7:58 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Jaime Casanova's message of mar nov 08 18:12:25 -0300 2011: On Sun, Nov 6, 2011 at 5:38 AM, Magnus Hagander

Re: [HACKERS] Measuring relation free space

2011-11-14 Thread Jaime Casanova
On Wed, Nov 9, 2011 at 7:58 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Jaime Casanova's message of mar nov 08 18:12:25 -0300 2011: On Sun, Nov 6, 2011 at 5:38 AM, Magnus Hagander mag...@hagander.net wrote: Looks pretty useful. thanks for the review, attached is a

Re: [HACKERS] Measuring relation free space

2011-11-09 Thread Alvaro Herrera
Excerpts from Jaime Casanova's message of mar nov 08 18:12:25 -0300 2011: On Sun, Nov 6, 2011 at 5:38 AM, Magnus Hagander mag...@hagander.net wrote: Looks pretty useful. thanks for the review, attached is a new version of it Note that AFAIK you shouldn't update the 1.0 extension script

Re: [HACKERS] Measuring relation free space

2011-11-09 Thread Robert Treat
On Tue, Nov 8, 2011 at 7:19 PM, Greg Smith g...@2ndquadrant.com wrote: On 11/08/2011 05:07 PM, Robert Treat wrote: It's already easy to get good enough numbers based on user space tools with very little overhead, so I think it's more important that the server side tool be accurate rather than

Re: [HACKERS] Measuring relation free space

2011-11-08 Thread Greg Smith
On 11/06/2011 11:55 PM, Mark Kirkwood wrote: I am guessing (at this point very much guessing) that pg_freespace map may return its data faster, as pageinspect is gonna have to grovel through all the pages for itself (whereas pg_freespacemap relies on using info from the ... free space map

Re: [HACKERS] Measuring relation free space

2011-11-08 Thread Jaime Casanova
On Sun, Nov 6, 2011 at 5:38 AM, Magnus Hagander mag...@hagander.net wrote: Looks pretty useful. thanks for the review, attached is a new version of it One quick stylistic comment - we don't generally use * 1.0 to turn an int into a double - just use a cast. ok Hooking into a ring

Re: [HACKERS] Measuring relation free space

2011-11-08 Thread Robert Treat
On Tue, Nov 8, 2011 at 1:07 PM, Greg Smith g...@2ndquadrant.com wrote: On 11/06/2011 11:55 PM, Mark Kirkwood wrote: I am guessing (at this point very much guessing) that pg_freespace map may return its data faster, as pageinspect is gonna have to grovel through all the pages for itself

Re: [HACKERS] Measuring relation free space

2011-11-08 Thread Greg Smith
On 11/08/2011 05:07 PM, Robert Treat wrote: It's already easy to get good enough numbers based on user space tools with very little overhead, so I think it's more important that the server side tool be accurate rather than fast. What user space method do you consider good enough here? I

Re: [HACKERS] Measuring relation free space

2011-11-06 Thread Magnus Hagander
On Sun, Nov 6, 2011 at 04:08, Greg Smith g...@2ndquadrant.com wrote: Attached patch adds a new function to the pageinspect extension for measuring total free space, in either tables or indexes.  It returns the free space as a percentage, so higher numbers mean more bloat.  After trying a

Re: [HACKERS] Measuring relation free space

2011-11-06 Thread Bernd Helmle
--On 6. November 2011 01:08:11 -0200 Greg Smith g...@2ndquadrant.com wrote: Attached patch adds a new function to the pageinspect extension for measuring total free space, in either tables or indexes. I wonder if that should be done in the pgstattuple module, which output some similar

Re: [HACKERS] Measuring relation free space

2011-11-06 Thread Mark Kirkwood
On 07/11/11 10:20, Bernd Helmle wrote: --On 6. November 2011 01:08:11 -0200 Greg Smith g...@2ndquadrant.com wrote: Attached patch adds a new function to the pageinspect extension for measuring total free space, in either tables or indexes. I wonder if that should be done in the