Re: [HACKERS] TOAST versus VACUUM, or missing chunk number 0 for toast value identified

2011-11-04 Thread Bruce Momjian
Tom Lane wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Alvaro Herrera's message of vie oct 28 16:47:13 -0300 2011: BTW we had previous discussions about dropping pg_database's toast table. Maybe this is a good time to do it, even if there's no risk of this bug

Re: [HACKERS] TOAST versus VACUUM, or missing chunk number 0 for toast value identified

2011-10-28 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Wait a minute: I'm confused. What's at issue here, at least AIUI, is taking a TOAST pointer and fetching the corresponding value. But that must involve reading from the TOAST table, and our usual paradigm for reading from system catalogs is (1) take

Re: [HACKERS] TOAST versus VACUUM, or missing chunk number 0 for toast value identified

2011-10-28 Thread Alvaro Herrera
Excerpts from Tom Lane's message of vie oct 28 15:37:43 -0300 2011: The main concern I had about detoast before caching is the risk of circularity, ie, needing detoastable cache entries in order to figure out how to detoast. But I think it's probably okay. The current list of catalogs with

Re: [HACKERS] TOAST versus VACUUM, or missing chunk number 0 for toast value identified

2011-10-28 Thread Alvaro Herrera
Excerpts from Alvaro Herrera's message of vie oct 28 16:47:13 -0300 2011: BTW we had previous discussions about dropping pg_database's toast table. Maybe this is a good time to do it, even if there's no risk of this bug (or the hypothetical circularity detoasting problem) showing up there.

Re: [HACKERS] TOAST versus VACUUM, or missing chunk number 0 for toast value identified

2011-10-28 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: The risk factor comes in when we hold a syscache entry across transactions; then this guarantee is lost. (In both the original example and the pg_proc case above, the second backend is only at risk when it starts its transaction after the first one's

Re: [HACKERS] TOAST versus VACUUM, or missing chunk number 0 for toast value identified

2011-10-28 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: If we made the commit sequence number more generally available, incrementing it at the point of visibility change under cover of ProcArrayLock, and including the then-current value in a Snapshot object when built, would that help with this at

Re: [HACKERS] TOAST versus VACUUM, or missing chunk number 0 for toast value identified

2011-10-28 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Alvaro Herrera's message of vie oct 28 16:47:13 -0300 2011: BTW we had previous discussions about dropping pg_database's toast table. Maybe this is a good time to do it, even if there's no risk of this bug (or the hypothetical

Re: [HACKERS] TOAST versus VACUUM, or missing chunk number 0 for toast value identified

2011-10-28 Thread Merlin Moncure
On Fri, Oct 28, 2011 at 3:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Alvaro Herrera's message of vie oct 28 16:47:13 -0300 2011: BTW we had previous discussions about dropping pg_database's toast table.  Maybe this is a good time

Re: [HACKERS] TOAST versus VACUUM, or missing chunk number 0 for toast value identified

2011-10-26 Thread Simon Riggs
On Wed, Oct 26, 2011 at 4:24 AM, Tom Lane t...@sss.pgh.pa.us wrote: Even given your recent changes to reduce the overhead of checking for sinval messages, I'm not sure that it'd be practical to move the sinval message processing to just-before-we-look-up-a-cache-entry.  Right now, we do

Re: [HACKERS] TOAST versus VACUUM, or missing chunk number 0 for toast value identified

2011-10-26 Thread Robert Haas
On Tue, Oct 25, 2011 at 11:24 PM, Tom Lane t...@sss.pgh.pa.us wrote: Even given your recent changes to reduce the overhead of checking for sinval messages, I'm not sure that it'd be practical to move the sinval message processing to just-before-we-look-up-a-cache-entry. Wait a minute: I'm

[HACKERS] TOAST versus VACUUM, or missing chunk number 0 for toast value identified

2011-10-25 Thread Tom Lane
I believe I have reproduced the behavior described by Andrew Hammond in http://archives.postgresql.org/pgsql-general/2011-10/msg00928.php This is using the regression database: 1. In session 1, do set default_statistics_target TO 1; analyze tenk1; (We need the large stats

Re: [HACKERS] TOAST versus VACUUM, or missing chunk number 0 for toast value identified

2011-10-25 Thread Robert Haas
On Tue, Oct 25, 2011 at 9:06 PM, Tom Lane t...@sss.pgh.pa.us wrote: What has happened here is that the second ANALYZE has marked itself committed in pg_clog and no longer running in the ProcArray, so VACUUM feels entitled to remove toast tuples that the ANALYZE deleted.  However, the ANALYZE

Re: [HACKERS] TOAST versus VACUUM, or missing chunk number 0 for toast value identified

2011-10-25 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Oct 25, 2011 at 9:06 PM, Tom Lane t...@sss.pgh.pa.us wrote: Ordinarily, sending out sinval messages post-commit is okay because we don't release locks until after that, and we suppose that our locks prevent any other transactions from getting