Re: [HACKERS] Remaining VACUUM patches

2007-04-18 Thread ITAGAKI Takahiro
Tom Lane <[EMAIL PROTECTED]> wrote: > > The other patch was ITAGAKI Takahiro's patch to fix n_dead_tuples in > > pgstats after VACUUM when there is concurrent update activity. > > I objected (and still object) to this patch because it allows > n_dead_tuples to drift arbitrarily far away from rea

Re: [HACKERS] Background LRU Writer/free list

2007-04-18 Thread Greg Smith
On Wed, 18 Apr 2007, Gregory Stark wrote: In particular I'm worried about what happens on a very busy cpu-bound system where adjusting the sleep times would result in it deciding to not sleep at all. On such a system sleeping for even 10ms might be too long... Anyways, if we have a working pat

Re: [HACKERS] Background LRU Writer/free list

2007-04-18 Thread Greg Smith
On Wed, 18 Apr 2007, Jim C. Nasby wrote: So if you make this change will BgBufferSync start incrementing StrategyControl->nextVictimBuffer and decrementing buf->usage_count like StrategyGetBuffer does now? Something will need to keep advancing the nextVictimBuffer, I hadn't really finished im

Re: [HACKERS] Background LRU Writer/free list

2007-04-18 Thread Greg Smith
On Wed, 18 Apr 2007, Tom Lane wrote: Furthermore, if the page was dirty, then it's probably been accessed more recently than adjacent pages that are clean, so preferentially zapping just-written pages seems backwards. The LRU background writer only writes out pages that have a usage_count of

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-18 Thread Jim C. Nasby
On Thu, Apr 19, 2007 at 10:07:08AM +1200, Martin Langhoff wrote: > Jim C. Nasby wrote: > > Then how do you tell what version a file is if it's outside of a > > checkout? > > It's trivial for git to answer that - the file will either be pristine, > and then we can just scan for the matching SHA1, o

Re: [HACKERS] modifying the table function

2007-04-18 Thread Tom Lane
"Islam Hegazy" <[EMAIL PROTECTED]> writes: > I wonder if I am on the right track or not and how to know such kind of > message sent from the server? Seems like you're doing it the hard way. Wouldn't it be easier to fix the client to display data before it's received the whole query result?

[HACKERS] Report on "PANIC: unexpected hash relation size" problem

2007-04-18 Thread Tom Lane
I finally got a chance to look at a reproducible case of this, thanks to Vlastimil Krejcir. It's now blindingly obvious that my patch of 2006-11-19, which was intended to fix bug #2737, has broken things. The problem is partly a misordering of operations in _hash_expandtable(): it calls _hash_all

Re: [HACKERS] modifying the table function

2007-04-18 Thread Islam Hegazy
Thanks for the documentation link. It helped me to understand how data are passed back to the client. I figured out that data is sent back to the client using the 'printtup' function. It is called by ExecSelect, called by ExecutorRun, etc. What I understand now is that the data is sent to the

Re: [HACKERS] Remaining VACUUM patches

2007-04-18 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > The other patch was ITAGAKI Takahiro's patch to fix n_dead_tuples in > pgstats after VACUUM when there is concurrent update activity. This > patch is still on hold largely because the above patch would cause it to > be a bit obsolete. I objected (and s

Re: [HACKERS] Remaining VACUUM patches

2007-04-18 Thread Heikki Linnakangas
Alvaro Herrera wrote: There are two additional patches in the VACUUM code. One is Heikki's patch to recalculate OldestXmin in the vacuum run. http://groups.google.es/group/pgsql.patches/browse_thread/thread/b2cfc901534d8990/40ba5b2fbb8f5b91 (much nicer than our archives because the whole thread

[HACKERS] Remaining VACUUM patches

2007-04-18 Thread Alvaro Herrera
There are two additional patches in the VACUUM code. One is Heikki's patch to recalculate OldestXmin in the vacuum run. http://groups.google.es/group/pgsql.patches/browse_thread/thread/b2cfc901534d8990/40ba5b2fbb8f5b91 (much nicer than our archives because the whole thread is there, not just mont

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-18 Thread Alvaro Herrera
Aidan Van Dyk wrote: > * Aidan Van Dyk <[EMAIL PROTECTED]> [070418 15:03]: > > > > Then how do you tell what version a file is if it's outside of a > > > checkout? > > > > That's what all the fun is about ;-) Some would say that "labelling" the > > file is the job of the release processes. Othe

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-18 Thread Aidan Van Dyk
* Aidan Van Dyk <[EMAIL PROTECTED]> [070418 15:03]: > > Then how do you tell what version a file is if it's outside of a > > checkout? > > That's what all the fun is about ;-) Some would say that "labelling" the > file is the job of the release processes. Others say it's the job of > the SCM sy

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-18 Thread Aidan Van Dyk
* Jim C. Nasby <[EMAIL PROTECTED]> [070418 14:39]: > On Wed, Apr 18, 2007 at 06:39:34PM +1200, Martin Langhoff wrote: > > Keyword expansions are generally bad because SCM tools should track > > _content_ - and keyword expansions _modify_ it to add metadata that is > > somewhat redundant, obtainable

Re: [HACKERS] [RFC] PostgreSQL Access Control Extension (PGACE)

2007-04-18 Thread Tom Lane
KaiGai Kohei <[EMAIL PROTECTED]> writes: >>> There are also >>> some interesting questions about SQL spec compliance and whether a >>> database that silently hides some rows from you will give semantically >>> consistent results. >> >> Yeah -- that's a potentially serious issue; KaiGai, have you l

Re: [HACKERS] [RFC] PostgreSQL Access Control Extension (PGACE)

2007-04-18 Thread KaiGai Kohei
... which presumably wouldn't involve any added dependency on outside code. For people who are already using SELinux or Trusted Solaris, making the database dependent on that infrastructure might be seen as a plus, but I'm not sure the rest of the world would be pleased. Yes, I was thinking t

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-18 Thread Jim C. Nasby
On Wed, Apr 18, 2007 at 06:39:34PM +1200, Martin Langhoff wrote: > Keyword expansions are generally bad because SCM tools should track > _content_ - and keyword expansions _modify_ it to add metadata that is > somewhat redundant, obtainable in other ways, and should just not be in > the middle of t

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-18 Thread Martin Langhoff
Tom Lane wrote: > Um ... why do either of you feel there's an issue there? > > We switched over to $PostgreSQL$ a few years ago specifically to avoid > creating merge problems for downstream repositories. If there are any > other keyword expansions left in the source text I'd vote to remove > the

Re: [HACKERS] Background LRU Writer/free list

2007-04-18 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > Why? What you're really trying to determine, I think, is the I/O load > imposed by the bgwriter, and pages-per-second seems a pretty natural > way to think about that; percentage of shared buffers not so much. What I'm saying is that pages/s will vary fro

Re: [HACKERS] Can't ri_KeysEqual() consider two nulls as equal?

2007-04-18 Thread Tom Lane
Stephan Szabo <[EMAIL PROTECTED]> writes: > On Tue, 17 Apr 2007, Tom Lane wrote: >> I claim that both ri_KeysEqual() and ri_OneKeyEqual() could consider >> two nulls to be equal. > Do you have any suggestions for alternate names? Keeping them using Equal > seems to be dangerous since people would

Re: [HACKERS] Can't ri_KeysEqual() consider two nulls as equal?

2007-04-18 Thread David Fetter
On Wed, Apr 18, 2007 at 07:51:48AM -0700, Stephan Szabo wrote: > On Tue, 17 Apr 2007, Tom Lane wrote: > > > A recent discussion led me to the idea that FK triggers are fired > > unnecessarily during an UPDATE if the foreign-key column(s) > > contain any NULLs, because ri_KeysEqual() treats two nul

Re: [HACKERS] Background LRU Writer/free list

2007-04-18 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > I had a thought on this. Instead of sleeping for a constant amount of time and > then estimating the number of pages needed for that constant amount of time > perhaps what bgwriter should be doing is sleeping for a variable amount of > time and estimating

Re: [HACKERS] Backend Crash

2007-04-18 Thread Tom Lane
Harvell F <[EMAIL PROTECTED]> writes: > I've got a database corruption/backend crash problem with my 8.1.3 > database on Mac OS X Server 10.4. I'm beginning the process of > trying to recover it. If anyone is interested in trying to fully > understand the what, where, and why of the crash,

Re: [HACKERS] Background LRU Writer/free list

2007-04-18 Thread Tom Lane
Greg Smith <[EMAIL PROTECTED]> writes: > With the new patch, the LRU writer is fairly well bounded in that it > doesn't write out more than it thinks it will need; you shouldn't get into > a situation where many more pages are written than will be used in the > near future. Given that mindset,

Re: [HACKERS] Backend Crash

2007-04-18 Thread Gregory Stark
"Harvell F" <[EMAIL PROTECTED]> writes: > Just as a follow up, it turns out that our fiberchannel RAID was power > cycled > while the systems were up and running. There are several write errors in the > postgresql log. > > Now I'm off to try to recover the data... That's still a problem, i

Re: [HACKERS] Background LRU Writer/free list

2007-04-18 Thread Gregory Stark
"Greg Smith" <[EMAIL PROTECTED]> writes: > I'm mostly done with my review of the "Automatic adjustment of > bgwriter_lru_maxpages" patch. In addition to issues already brought up with > that code, there are some small things that need to be done to merge it with > the recent pg_stat_bgwriter pat

Re: [HACKERS] [RFC] PostgreSQL Access Control Extension (PGACE)

2007-04-18 Thread Tom Lane
Josh Berkus <[EMAIL PROTECTED]> writes: > Give the team some credit, though; they've managed to come up with a > system that integrates OS-level ACLs for both SElinux and TxSol, are not > asking us to incorporate two different sets, and are coming to us with a > serious proposal that has a lot o

Re: [HACKERS] utf8 COPY DELIMITER?

2007-04-18 Thread Andrew Dunstan
Jim C. Nasby wrote: On Tue, Apr 17, 2007 at 02:28:18PM -0400, Tom Lane wrote: I doubt that supporting a single multibyte character would be an interesting extension --- if we wanted to do anything at all there, we'd just generalize the delimiter to be an arbitrary string. But it would certai

Re: [HACKERS] Background LRU Writer/free list

2007-04-18 Thread Jim C. Nasby
On Wed, Apr 18, 2007 at 09:09:11AM -0400, Greg Smith wrote: > I'm mostly done with my review of the "Automatic adjustment of > bgwriter_lru_maxpages" patch. In addition to issues already brought up > with that code, there are some small things that need to be done to merge > it with the recent

Re: [HACKERS] Autovacuum vs statement_timeout

2007-04-18 Thread Robert Treat
On Wednesday 18 April 2007 11:30, Alvaro Herrera wrote: > Robert Treat wrote: > > On Tuesday 17 April 2007 20:54, Tom Lane wrote: > > > I'm not excited about the other ones but I can see the argument for > > > making pg_dump force the timeout to 0. > > > > Allowing pg_dump to run un-checked could a

Re: [HACKERS] utf8 COPY DELIMITER?

2007-04-18 Thread Jim C. Nasby
On Tue, Apr 17, 2007 at 02:28:18PM -0400, Tom Lane wrote: > I doubt that supporting a single multibyte character would be an > interesting extension --- if we wanted to do anything at all there, we'd > just generalize the delimiter to be an arbitrary string. But it would > certainly slow down COPY

Re: [HACKERS] Autovacuum vs statement_timeout

2007-04-18 Thread Joshua D. Drake
Alvaro Herrera wrote: Robert Treat wrote: On Tuesday 17 April 2007 20:54, Tom Lane wrote: I'm not excited about the other ones but I can see the argument for making pg_dump force the timeout to 0. Allowing pg_dump to run un-checked could also lead to problems such as exceeding maintenence wi

[HACKERS] Background LRU Writer/free list

2007-04-18 Thread Greg Smith
I'm mostly done with my review of the "Automatic adjustment of bgwriter_lru_maxpages" patch. In addition to issues already brought up with that code, there are some small things that need to be done to merge it with the recent pg_stat_bgwriter patch, and I have some concerns about its unbounde

Re: [HACKERS] Autovacuum vs statement_timeout

2007-04-18 Thread Alvaro Herrera
> > Tom Lane wrote: > > > I seem to remember that we'd agreed that autovacuum should ignore any > > > globally set statement_timeout, on the grounds that a poorly chosen > > > setting could indefinitely prevent large tables from being vacuumed. FWIW in testing, I just noticed that autovacuum does

Re: [HACKERS] Autovacuum vs statement_timeout

2007-04-18 Thread Alvaro Herrera
Robert Treat wrote: > On Tuesday 17 April 2007 20:54, Tom Lane wrote: > > I'm not excited about the other ones but I can see the argument for > > making pg_dump force the timeout to 0. > > Allowing pg_dump to run un-checked could also lead to problems such as > exceeding maintenence windows caus

Re: [HACKERS] Backend Crash

2007-04-18 Thread Harvell F
Just as a follow up, it turns out that our fiberchannel RAID was power cycled while the systems were up and running. There are several write errors in the postgresql log. Now I'm off to try to recover the data... -- F Harvell 407 467-1919 On 18 Apr 2007, at 10:08, Harvell F wrote:

[HACKERS] Backend Crash

2007-04-18 Thread Harvell F
I've got a database corruption/backend crash problem with my 8.1.3 database on Mac OS X Server 10.4. I'm beginning the process of trying to recover it. If anyone is interested in trying to fully understand the what, where, and why of the crash, please contact me. I've provided the basic

Re: [HACKERS] Autovacuum vs statement_timeout

2007-04-18 Thread Alvaro Herrera
Robert Treat wrote: > On Tuesday 17 April 2007 21:25, Alvaro Herrera wrote: > > I think that is too strong an assumption, which is why I'm planning to > > back-patch the change to reset statement_timeout to 0 on autovacuum till > > 8.0, as discussed. I think I should also backpatch the change to s

Re: [HACKERS] Autovacuum vs statement_timeout

2007-04-18 Thread Robert Treat
On Tuesday 17 April 2007 20:54, Tom Lane wrote: > Robert Treat <[EMAIL PROTECTED]> writes: > > I'm with Joshua on this one. Statement_timeout is often used as a means > > for protection from long running statements due to server load and > > locking and all of the above commands can certainly fall

Re: [HACKERS] Can't ri_KeysEqual() consider two nulls as equal?

2007-04-18 Thread Stephan Szabo
On Tue, 17 Apr 2007, Tom Lane wrote: > A recent discussion led me to the idea that FK triggers are fired > unnecessarily during an UPDATE if the foreign-key column(s) contain > any NULLs, because ri_KeysEqual() treats two nulls as unequal, > and therefore we conclude the row has changed when it ha

Re: [HACKERS] Autovacuum vs statement_timeout

2007-04-18 Thread Robert Treat
On Tuesday 17 April 2007 21:25, Alvaro Herrera wrote: > I think that is too strong an assumption, which is why I'm planning to > back-patch the change to reset statement_timeout to 0 on autovacuum till > 8.0, as discussed. I think I should also backpatch the change to set > zero_damaged_pages as w

Re: [HACKERS] schema creation during initdb

2007-04-18 Thread Andrew Dunstan
NikhilS wrote: One way that I can think of doing this would be by adding the details about the new schema and the tables that lie therein in the src/backend/catalog/information_schema.sql file (I dont know if this is the recommended way though). These will end up becoming a part of template

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-18 Thread Alvaro Herrera
Tom Lane wrote: > Martin Langhoff <[EMAIL PROTECTED]> writes: > > Aidan Van Dyk wrote: > >> And remember the warning I gave that my conversion is *not* a direct CVS > >> import - I intentionally *unexpand* all Keywords before stuffing them > >> into GIT so that merging and branching can ignore all

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-18 Thread Aidan Van Dyk
* Tom Lane <[EMAIL PROTECTED]> [070418 01:33]: > Um ... why do either of you feel there's an issue there? > > We switched over to $PostgreSQL$ a few years ago specifically to avoid > creating merge problems for downstream repositories. If there are any > other keyword expansions left in the sou

Re: [HACKERS] schema creation during initdb

2007-04-18 Thread NikhilS
Hi, On 4/18/07, sharath kumar <[EMAIL PROTECTED]> wrote: Sorry i have put the question wrongly. I wanted to ask if we can create a schema during createdb time so that i have to hide the following from the user. psql -c 'create schema foo' mytemplate psql -c 'create table foo.bar ...' mytempla

Re: [HACKERS] schema creation during initdb

2007-04-18 Thread Heikki Linnakangas
sharath kumar wrote: Sorry i have put the question wrongly. I wanted to ask if we can create a schema during createdb time so that i have to hide the following from the user. psql -c 'create schema foo' mytemplate psql -c 'create table foo.bar ...' mytemplate Whenever a user runs createdb command

Re: [HACKERS] schema creation during initdb

2007-04-18 Thread sharath kumar
Sorry i have put the question wrongly. I wanted to ask if we can create a schema during createdb time so that i have to hide the following from the user. psql -c 'create schema foo' mytemplate psql -c 'create table foo.bar ...' mytemplate Whenever a user runs createdb command, the above schema and

Re: [HACKERS] Can't ri_KeysEqual() consider two nulls as equal?

2007-04-18 Thread Richard Huxton
Simon Riggs wrote: On Tue, 2007-04-17 at 17:16 -0400, Tom Lane wrote: A recent discussion led me to the idea that FK triggers are fired unnecessarily during an UPDATE if the foreign-key column(s) contain any NULLs, because ri_KeysEqual() treats two nulls as unequal, and therefore we conclude the

Re: [HACKERS] Autovacuum vs statement_timeout

2007-04-18 Thread Magnus Hagander
On Tue, Apr 17, 2007 at 10:33:21PM -0400, Bruce Momjian wrote: > Alvaro Herrera wrote: > > I think that is too strong an assumption, which is why I'm planning to > > back-patch the change to reset statement_timeout to 0 on autovacuum till > > 8.0, as discussed. I think I should also backpatch the