Re: [HACKERS] Autovacuum vs statement_timeout

2007-04-18 Thread Magnus Hagander
On Tue, Apr 17, 2007 at 10:33:21PM -0400, Bruce Momjian wrote: Alvaro Herrera wrote: I think that is too strong an assumption, which is why I'm planning to back-patch the change to reset statement_timeout to 0 on autovacuum till 8.0, as discussed. I think I should also backpatch the change

Re: [HACKERS] Can't ri_KeysEqual() consider two nulls as equal?

2007-04-18 Thread Richard Huxton
Simon Riggs wrote: On Tue, 2007-04-17 at 17:16 -0400, Tom Lane wrote: A recent discussion led me to the idea that FK triggers are fired unnecessarily during an UPDATE if the foreign-key column(s) contain any NULLs, because ri_KeysEqual() treats two nulls as unequal, and therefore we conclude

Re: [HACKERS] schema creation during initdb

2007-04-18 Thread sharath kumar
Sorry i have put the question wrongly. I wanted to ask if we can create a schema during createdb time so that i have to hide the following from the user. psql -c 'create schema foo' mytemplate psql -c 'create table foo.bar ...' mytemplate Whenever a user runs createdb command, the above schema

Re: [HACKERS] schema creation during initdb

2007-04-18 Thread Heikki Linnakangas
sharath kumar wrote: Sorry i have put the question wrongly. I wanted to ask if we can create a schema during createdb time so that i have to hide the following from the user. psql -c 'create schema foo' mytemplate psql -c 'create table foo.bar ...' mytemplate Whenever a user runs createdb

Re: [HACKERS] schema creation during initdb

2007-04-18 Thread NikhilS
Hi, On 4/18/07, sharath kumar [EMAIL PROTECTED] wrote: Sorry i have put the question wrongly. I wanted to ask if we can create a schema during createdb time so that i have to hide the following from the user. psql -c 'create schema foo' mytemplate psql -c 'create table foo.bar ...'

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-18 Thread Aidan Van Dyk
* Tom Lane [EMAIL PROTECTED] [070418 01:33]: Um ... why do either of you feel there's an issue there? We switched over to $PostgreSQL$ a few years ago specifically to avoid creating merge problems for downstream repositories. If there are any other keyword expansions left in the source

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-18 Thread Alvaro Herrera
Tom Lane wrote: Martin Langhoff [EMAIL PROTECTED] writes: Aidan Van Dyk wrote: And remember the warning I gave that my conversion is *not* a direct CVS import - I intentionally *unexpand* all Keywords before stuffing them into GIT so that merging and branching can ignore all the Keyword

Re: [HACKERS] schema creation during initdb

2007-04-18 Thread Andrew Dunstan
NikhilS wrote: One way that I can think of doing this would be by adding the details about the new schema and the tables that lie therein in the src/backend/catalog/information_schema.sql file (I dont know if this is the recommended way though). These will end up becoming a part of

Re: [HACKERS] Autovacuum vs statement_timeout

2007-04-18 Thread Robert Treat
On Tuesday 17 April 2007 21:25, Alvaro Herrera wrote: I think that is too strong an assumption, which is why I'm planning to back-patch the change to reset statement_timeout to 0 on autovacuum till 8.0, as discussed. I think I should also backpatch the change to set zero_damaged_pages as well

Re: [HACKERS] Can't ri_KeysEqual() consider two nulls as equal?

2007-04-18 Thread Stephan Szabo
On Tue, 17 Apr 2007, Tom Lane wrote: A recent discussion led me to the idea that FK triggers are fired unnecessarily during an UPDATE if the foreign-key column(s) contain any NULLs, because ri_KeysEqual() treats two nulls as unequal, and therefore we conclude the row has changed when it has

Re: [HACKERS] Autovacuum vs statement_timeout

2007-04-18 Thread Robert Treat
On Tuesday 17 April 2007 20:54, Tom Lane wrote: Robert Treat [EMAIL PROTECTED] writes: I'm with Joshua on this one. Statement_timeout is often used as a means for protection from long running statements due to server load and locking and all of the above commands can certainly fall into

Re: [HACKERS] Autovacuum vs statement_timeout

2007-04-18 Thread Alvaro Herrera
Robert Treat wrote: On Tuesday 17 April 2007 21:25, Alvaro Herrera wrote: I think that is too strong an assumption, which is why I'm planning to back-patch the change to reset statement_timeout to 0 on autovacuum till 8.0, as discussed. I think I should also backpatch the change to set

[HACKERS] Backend Crash

2007-04-18 Thread Harvell F
I've got a database corruption/backend crash problem with my 8.1.3 database on Mac OS X Server 10.4. I'm beginning the process of trying to recover it. If anyone is interested in trying to fully understand the what, where, and why of the crash, please contact me. I've provided the basic

Re: [HACKERS] Backend Crash

2007-04-18 Thread Harvell F
Just as a follow up, it turns out that our fiberchannel RAID was power cycled while the systems were up and running. There are several write errors in the postgresql log. Now I'm off to try to recover the data... -- F Harvell 407 467-1919 On 18 Apr 2007, at 10:08, Harvell F wrote:

Re: [HACKERS] Autovacuum vs statement_timeout

2007-04-18 Thread Alvaro Herrera
Robert Treat wrote: On Tuesday 17 April 2007 20:54, Tom Lane wrote: I'm not excited about the other ones but I can see the argument for making pg_dump force the timeout to 0. Allowing pg_dump to run un-checked could also lead to problems such as exceeding maintenence windows causing

Re: [HACKERS] Autovacuum vs statement_timeout

2007-04-18 Thread Alvaro Herrera
Tom Lane wrote: I seem to remember that we'd agreed that autovacuum should ignore any globally set statement_timeout, on the grounds that a poorly chosen setting could indefinitely prevent large tables from being vacuumed. FWIW in testing, I just noticed that autovacuum does not pay

[HACKERS] Background LRU Writer/free list

2007-04-18 Thread Greg Smith
I'm mostly done with my review of the Automatic adjustment of bgwriter_lru_maxpages patch. In addition to issues already brought up with that code, there are some small things that need to be done to merge it with the recent pg_stat_bgwriter patch, and I have some concerns about its unbounded

Re: [HACKERS] Autovacuum vs statement_timeout

2007-04-18 Thread Joshua D. Drake
Alvaro Herrera wrote: Robert Treat wrote: On Tuesday 17 April 2007 20:54, Tom Lane wrote: I'm not excited about the other ones but I can see the argument for making pg_dump force the timeout to 0. Allowing pg_dump to run un-checked could also lead to problems such as exceeding maintenence

Re: [HACKERS] utf8 COPY DELIMITER?

2007-04-18 Thread Jim C. Nasby
On Tue, Apr 17, 2007 at 02:28:18PM -0400, Tom Lane wrote: I doubt that supporting a single multibyte character would be an interesting extension --- if we wanted to do anything at all there, we'd just generalize the delimiter to be an arbitrary string. But it would certainly slow down COPY by

Re: [HACKERS] Autovacuum vs statement_timeout

2007-04-18 Thread Robert Treat
On Wednesday 18 April 2007 11:30, Alvaro Herrera wrote: Robert Treat wrote: On Tuesday 17 April 2007 20:54, Tom Lane wrote: I'm not excited about the other ones but I can see the argument for making pg_dump force the timeout to 0. Allowing pg_dump to run un-checked could also lead to

Re: [HACKERS] Background LRU Writer/free list

2007-04-18 Thread Jim C. Nasby
On Wed, Apr 18, 2007 at 09:09:11AM -0400, Greg Smith wrote: I'm mostly done with my review of the Automatic adjustment of bgwriter_lru_maxpages patch. In addition to issues already brought up with that code, there are some small things that need to be done to merge it with the recent

Re: [HACKERS] utf8 COPY DELIMITER?

2007-04-18 Thread Andrew Dunstan
Jim C. Nasby wrote: On Tue, Apr 17, 2007 at 02:28:18PM -0400, Tom Lane wrote: I doubt that supporting a single multibyte character would be an interesting extension --- if we wanted to do anything at all there, we'd just generalize the delimiter to be an arbitrary string. But it would

Re: [HACKERS] [RFC] PostgreSQL Access Control Extension (PGACE)

2007-04-18 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: Give the team some credit, though; they've managed to come up with a system that integrates OS-level ACLs for both SElinux and TxSol, are not asking us to incorporate two different sets, and are coming to us with a serious proposal that has a lot of

Re: [HACKERS] Background LRU Writer/free list

2007-04-18 Thread Gregory Stark
Greg Smith [EMAIL PROTECTED] writes: I'm mostly done with my review of the Automatic adjustment of bgwriter_lru_maxpages patch. In addition to issues already brought up with that code, there are some small things that need to be done to merge it with the recent pg_stat_bgwriter patch, and I

Re: [HACKERS] Backend Crash

2007-04-18 Thread Gregory Stark
Harvell F [EMAIL PROTECTED] writes: Just as a follow up, it turns out that our fiberchannel RAID was power cycled while the systems were up and running. There are several write errors in the postgresql log. Now I'm off to try to recover the data... That's still a problem, it

Re: [HACKERS] Background LRU Writer/free list

2007-04-18 Thread Tom Lane
Greg Smith [EMAIL PROTECTED] writes: With the new patch, the LRU writer is fairly well bounded in that it doesn't write out more than it thinks it will need; you shouldn't get into a situation where many more pages are written than will be used in the near future. Given that mindset,

Re: [HACKERS] Backend Crash

2007-04-18 Thread Tom Lane
Harvell F [EMAIL PROTECTED] writes: I've got a database corruption/backend crash problem with my 8.1.3 database on Mac OS X Server 10.4. I'm beginning the process of trying to recover it. If anyone is interested in trying to fully understand the what, where, and why of the crash,

Re: [HACKERS] Background LRU Writer/free list

2007-04-18 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: I had a thought on this. Instead of sleeping for a constant amount of time and then estimating the number of pages needed for that constant amount of time perhaps what bgwriter should be doing is sleeping for a variable amount of time and estimating the

Re: [HACKERS] Can't ri_KeysEqual() consider two nulls as equal?

2007-04-18 Thread David Fetter
On Wed, Apr 18, 2007 at 07:51:48AM -0700, Stephan Szabo wrote: On Tue, 17 Apr 2007, Tom Lane wrote: A recent discussion led me to the idea that FK triggers are fired unnecessarily during an UPDATE if the foreign-key column(s) contain any NULLs, because ri_KeysEqual() treats two nulls as

Re: [HACKERS] Can't ri_KeysEqual() consider two nulls as equal?

2007-04-18 Thread Tom Lane
Stephan Szabo [EMAIL PROTECTED] writes: On Tue, 17 Apr 2007, Tom Lane wrote: I claim that both ri_KeysEqual() and ri_OneKeyEqual() could consider two nulls to be equal. Do you have any suggestions for alternate names? Keeping them using Equal seems to be dangerous since people would likely

Re: [HACKERS] Background LRU Writer/free list

2007-04-18 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Why? What you're really trying to determine, I think, is the I/O load imposed by the bgwriter, and pages-per-second seems a pretty natural way to think about that; percentage of shared buffers not so much. What I'm saying is that pages/s will vary from

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-18 Thread Martin Langhoff
Tom Lane wrote: Um ... why do either of you feel there's an issue there? We switched over to $PostgreSQL$ a few years ago specifically to avoid creating merge problems for downstream repositories. If there are any other keyword expansions left in the source text I'd vote to remove them.

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-18 Thread Jim C. Nasby
On Wed, Apr 18, 2007 at 06:39:34PM +1200, Martin Langhoff wrote: Keyword expansions are generally bad because SCM tools should track _content_ - and keyword expansions _modify_ it to add metadata that is somewhat redundant, obtainable in other ways, and should just not be in the middle of the

Re: [HACKERS] [RFC] PostgreSQL Access Control Extension (PGACE)

2007-04-18 Thread KaiGai Kohei
... which presumably wouldn't involve any added dependency on outside code. For people who are already using SELinux or Trusted Solaris, making the database dependent on that infrastructure might be seen as a plus, but I'm not sure the rest of the world would be pleased. Yes, I was thinking

Re: [HACKERS] [RFC] PostgreSQL Access Control Extension (PGACE)

2007-04-18 Thread Tom Lane
KaiGai Kohei [EMAIL PROTECTED] writes: There are also some interesting questions about SQL spec compliance and whether a database that silently hides some rows from you will give semantically consistent results. Yeah -- that's a potentially serious issue; KaiGai, have you looked into it?

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-18 Thread Aidan Van Dyk
* Jim C. Nasby [EMAIL PROTECTED] [070418 14:39]: On Wed, Apr 18, 2007 at 06:39:34PM +1200, Martin Langhoff wrote: Keyword expansions are generally bad because SCM tools should track _content_ - and keyword expansions _modify_ it to add metadata that is somewhat redundant, obtainable in

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-18 Thread Aidan Van Dyk
* Aidan Van Dyk [EMAIL PROTECTED] [070418 15:03]: Then how do you tell what version a file is if it's outside of a checkout? That's what all the fun is about ;-) Some would say that labelling the file is the job of the release processes. Others say it's the job of the SCM system...

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-18 Thread Alvaro Herrera
Aidan Van Dyk wrote: * Aidan Van Dyk [EMAIL PROTECTED] [070418 15:03]: Then how do you tell what version a file is if it's outside of a checkout? That's what all the fun is about ;-) Some would say that labelling the file is the job of the release processes. Others say it's the

[HACKERS] Remaining VACUUM patches

2007-04-18 Thread Alvaro Herrera
There are two additional patches in the VACUUM code. One is Heikki's patch to recalculate OldestXmin in the vacuum run. http://groups.google.es/group/pgsql.patches/browse_thread/thread/b2cfc901534d8990/40ba5b2fbb8f5b91 (much nicer than our archives because the whole thread is there, not just

Re: [HACKERS] Remaining VACUUM patches

2007-04-18 Thread Heikki Linnakangas
Alvaro Herrera wrote: There are two additional patches in the VACUUM code. One is Heikki's patch to recalculate OldestXmin in the vacuum run. http://groups.google.es/group/pgsql.patches/browse_thread/thread/b2cfc901534d8990/40ba5b2fbb8f5b91 (much nicer than our archives because the whole

Re: [HACKERS] Remaining VACUUM patches

2007-04-18 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: The other patch was ITAGAKI Takahiro's patch to fix n_dead_tuples in pgstats after VACUUM when there is concurrent update activity. This patch is still on hold largely because the above patch would cause it to be a bit obsolete. I objected (and still

Re: [HACKERS] modifying the table function

2007-04-18 Thread Islam Hegazy
Thanks for the documentation link. It helped me to understand how data are passed back to the client. I figured out that data is sent back to the client using the 'printtup' function. It is called by ExecSelect, called by ExecutorRun, etc. What I understand now is that the data is sent to the

[HACKERS] Report on PANIC: unexpected hash relation size problem

2007-04-18 Thread Tom Lane
I finally got a chance to look at a reproducible case of this, thanks to Vlastimil Krejcir. It's now blindingly obvious that my patch of 2006-11-19, which was intended to fix bug #2737, has broken things. The problem is partly a misordering of operations in _hash_expandtable(): it calls

Re: [HACKERS] modifying the table function

2007-04-18 Thread Tom Lane
Islam Hegazy [EMAIL PROTECTED] writes: I wonder if I am on the right track or not and how to know such kind of message sent from the server? Seems like you're doing it the hard way. Wouldn't it be easier to fix the client to display data before it's received the whole query result?

Re: [HACKERS] Hacking on PostgreSQL via GIT

2007-04-18 Thread Jim C. Nasby
On Thu, Apr 19, 2007 at 10:07:08AM +1200, Martin Langhoff wrote: Jim C. Nasby wrote: Then how do you tell what version a file is if it's outside of a checkout? It's trivial for git to answer that - the file will either be pristine, and then we can just scan for the matching SHA1, or

Re: [HACKERS] Background LRU Writer/free list

2007-04-18 Thread Greg Smith
On Wed, 18 Apr 2007, Tom Lane wrote: Furthermore, if the page was dirty, then it's probably been accessed more recently than adjacent pages that are clean, so preferentially zapping just-written pages seems backwards. The LRU background writer only writes out pages that have a usage_count of

Re: [HACKERS] Background LRU Writer/free list

2007-04-18 Thread Greg Smith
On Wed, 18 Apr 2007, Jim C. Nasby wrote: So if you make this change will BgBufferSync start incrementing StrategyControl-nextVictimBuffer and decrementing buf-usage_count like StrategyGetBuffer does now? Something will need to keep advancing the nextVictimBuffer, I hadn't really finished

Re: [HACKERS] Background LRU Writer/free list

2007-04-18 Thread Greg Smith
On Wed, 18 Apr 2007, Gregory Stark wrote: In particular I'm worried about what happens on a very busy cpu-bound system where adjusting the sleep times would result in it deciding to not sleep at all. On such a system sleeping for even 10ms might be too long... Anyways, if we have a working