Re: [PERFORM] vacuum full 100 mins plus?
Tom Lane wrote: Christopher Browne [EMAIL PROTECTED] writes: A long time ago, in a galaxy far, farpliers [EMAIL PROTECTED] (Patrick Hatcher) wrote: Answered my own question. I gave up the vacuum full after 150 mins. I was able to export to a file, vacuum full the empty table, and reimport in less than 10 mins. I suspect the empty item pointers and the sheer number of removable rows was causing an issue. In that case, you'd be a little further better off if the steps were: - drop indices; - copy table to file (perhaps via pg_dump -t my_table); - truncate the table, or drop-and-recreate, both of which make it unnecessary to do _any_ vacuum of the result; - recreate indices, probably with SORT_MEM set high, to minimize paging to disk - analyze the table (no need to vacuum if you haven't created any dead tuples) - cut SORT_MEM back down to normal sizes Rather than doing all this manually, you can just CLUSTER on any handy index. In 7.5, another possibility is to issue one of the forms of ALTER TABLE that force a table rewrite. The range of usefulness of VACUUM FULL is really looking narrower and narrower to me. I can foresee a day when we'll abandon it completely. I would love to see this 10lb sledge hammer go away when we have enough tiny screwdrivers and needlenose pliers to make it obsolete! regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
[PERFORM] vacuum full 100 mins plus?
Should I be concerned that my vacuum process has taken upwards of 100 + minutes to complete? I dropped all indexes before starting and also increased the vacuum_mem before starting. Looking at the output below, it appears that a vacuum full hasn't been done on this table for quite sometime. Would I be better off exporting the data vacuuming the table and reimporting the data? I cannot drop the table do to views attached to the table mdc_oz=# set vacuum_mem = 10240; SET mdc_oz=# vacuum full verbose cdm.cdm_ddw_Tran_item; INFO: vacuuming cdm.cdm_ddw_tran_item INFO: cdm_ddw_tran_item: found 15322404 removable, 10950460 nonremovable row versions in 934724 pages DETAIL: 0 dead row versions cannot be removed yet. Nonremovable row versions range from 233 to 308 bytes long. There were 1081 unused item pointers. Total free space (including removable row versions) is 4474020460 bytes. 544679 pages are or will become empty, including 0 at the end of the table. 692980 pages containing 4433398408 free bytes are potential move destinations. CPU 29.55s/4.13u sec elapsed 107.82 sec. TIA Patrick Hatcher ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PERFORM] vacuum full 100 mins plus?
Answered my own question. I gave up the vacuum full after 150 mins. I was able to export to a file, vacuum full the empty table, and reimport in less than 10 mins. I suspect the empty item pointers and the sheer number of removable rows was causing an issue. Patrick Hatcher [EMAIL PROTECTED] omTo Sent by: [EMAIL PROTECTED] pgsql-performance cc [EMAIL PROTECTED] .org Subject [PERFORM] vacuum full 100 mins plus? 07/14/2004 02:29 PM Should I be concerned that my vacuum process has taken upwards of 100 + minutes to complete? I dropped all indexes before starting and also increased the vacuum_mem before starting. Looking at the output below, it appears that a vacuum full hasn't been done on this table for quite sometime. Would I be better off exporting the data vacuuming the table and reimporting the data? I cannot drop the table do to views attached to the table mdc_oz=# set vacuum_mem = 10240; SET mdc_oz=# vacuum full verbose cdm.cdm_ddw_Tran_item; INFO: vacuuming cdm.cdm_ddw_tran_item INFO: cdm_ddw_tran_item: found 15322404 removable, 10950460 nonremovable row versions in 934724 pages DETAIL: 0 dead row versions cannot be removed yet. Nonremovable row versions range from 233 to 308 bytes long. There were 1081 unused item pointers. Total free space (including removable row versions) is 4474020460 bytes. 544679 pages are or will become empty, including 0 at the end of the table. 692980 pages containing 4433398408 free bytes are potential move destinations. CPU 29.55s/4.13u sec elapsed 107.82 sec. TIA Patrick Hatcher ---(end of broadcast)--- TIP 8: explain analyze is your friend ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PERFORM] vacuum full 100 mins plus?
Patrick, Answered my own question. I gave up the vacuum full after 150 mins. I was able to export to a file, vacuum full the empty table, and reimport in less than 10 mins. I suspect the empty item pointers and the sheer number of removable rows was causing an issue. Yeah. If you've a table that's not been vacuumed in a month, it's often faster to clean it out and import it. I've seen vacuums take up to 3 hours in really bad cases. -- -Josh Berkus Aglio Database Solutions San Francisco ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PERFORM] vacuum full 100 mins plus?
A long time ago, in a galaxy far, far away, [EMAIL PROTECTED] (Patrick Hatcher) wrote: Answered my own question. I gave up the vacuum full after 150 mins. I was able to export to a file, vacuum full the empty table, and reimport in less than 10 mins. I suspect the empty item pointers and the sheer number of removable rows was causing an issue. In that case, you'd be a little further better off if the steps were: - drop indices; - copy table to file (perhaps via pg_dump -t my_table); - truncate the table, or drop-and-recreate, both of which make it unnecessary to do _any_ vacuum of the result; - recreate indices, probably with SORT_MEM set high, to minimize paging to disk - analyze the table (no need to vacuum if you haven't created any dead tuples) - cut SORT_MEM back down to normal sizes -- output = reverse(gro.gultn @ enworbbc) http://www3.sympatico.ca/cbbrowne/spreadsheets.html Signs of a Klingon Programmer #6: Debugging? Klingons do not debug. Our software does not coddle the weak. ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PERFORM] vacuum full 100 mins plus?
Christopher Browne [EMAIL PROTECTED] writes: A long time ago, in a galaxy far, far away, [EMAIL PROTECTED] (Patrick Hatcher) wrote: Answered my own question. I gave up the vacuum full after 150 mins. I was able to export to a file, vacuum full the empty table, and reimport in less than 10 mins. I suspect the empty item pointers and the sheer number of removable rows was causing an issue. In that case, you'd be a little further better off if the steps were: - drop indices; - copy table to file (perhaps via pg_dump -t my_table); - truncate the table, or drop-and-recreate, both of which make it unnecessary to do _any_ vacuum of the result; - recreate indices, probably with SORT_MEM set high, to minimize paging to disk - analyze the table (no need to vacuum if you haven't created any dead tuples) - cut SORT_MEM back down to normal sizes Rather than doing all this manually, you can just CLUSTER on any handy index. In 7.5, another possibility is to issue one of the forms of ALTER TABLE that force a table rewrite. The range of usefulness of VACUUM FULL is really looking narrower and narrower to me. I can foresee a day when we'll abandon it completely. regards, tom lane ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
