Re: [HACKERS] changing MyDatabaseId

2010-11-17 Thread Markus Wanner
Robert, On 11/15/2010 05:39 AM, Robert Haas wrote: I've spent a few hours pouring over the source code with coarse-toothed comb, trying to figure out just exactly what might break if we changed MyDatabaseId after backend startup time, or in other words, allowed a backend to unbind from the

Re: [HACKERS] changing MyDatabaseId

2010-11-17 Thread Andres Freund
On Wednesday 17 November 2010 11:04:04 Markus Wanner wrote: Robert, On 11/15/2010 05:39 AM, Robert Haas wrote: I've spent a few hours pouring over the source code with coarse-toothed comb, trying to figure out just exactly what might break if we changed MyDatabaseId after backend startup

Re: [HACKERS] multi-platform, multi-locale regression tests

2010-11-17 Thread Markus Wanner
Kevin, On 11/13/2010 01:28 AM, Kevin Grittner wrote: Should anyone else run into this, it's controlled by this in the test scheduling definitions (the tdef values): 'xfail': True There are other test flags you can override here, like 'skip' to skip a test. Correct. Looks like dtester

Re: [HACKERS] changing MyDatabaseId

2010-11-17 Thread Markus Wanner
Andreas, On 11/17/2010 11:38 AM, Andres Freund wrote: Well, one could optimize most of the resetting away if the the old MyDatabaseId and the new one are the same - an optimization which is hardly possible with forking new backends. Uh? Why not simply re-use the same backend, then? Or do

Re: [HACKERS] changing MyDatabaseId

2010-11-17 Thread Andres Freund
On Wednesday 17 November 2010 11:58:33 Markus Wanner wrote: Andreas, On 11/17/2010 11:38 AM, Andres Freund wrote: Well, one could optimize most of the resetting away if the the old MyDatabaseId and the new one are the same - an optimization which is hardly possible with forking new

Re: [HACKERS] changing MyDatabaseId

2010-11-17 Thread Markus Wanner
On 11/17/2010 12:09 PM, Andres Freund wrote: I am thinking of a connection-pooler like setup. Quite often your main-load goes towards a single database - in that situation you don't have to reset the database id most of the time. Okay, so that's what I'd call a connection-reset or

[HACKERS] Overlap operators for ARRAY and NULLs

2010-11-17 Thread Itagaki Takahiro
There might be inconsistency in overlap operators for array types ( @, @, and ) when the argument arrays contain NULLs. - SELECT 2 = ANY (ARRAY[1, NULL]) = NULL - SELECT ARRAY[2] @ ARRAY[1, NULL] = false NULL means unknown in definition, so should it return NULL rather than false? I found

Re: [HACKERS] changing MyDatabaseId

2010-11-17 Thread Alvaro Herrera
Excerpts from Markus Wanner's message of mié nov 17 07:04:04 -0300 2010: Thoughts? The question obviously is whether or not this is faster than just terminating one backend and starting a new one. Which basically costs an additional termination and re-creation of a process (i.e. fork())

Re: [HACKERS] changing MyDatabaseId

2010-11-17 Thread Markus Wanner
On 11/17/2010 01:27 PM, Alvaro Herrera wrote: I don't think it's a speed thing only. It would be a great thing to have in autovacuum, for example, where we have constant problem reports because the system failed to fork a new backend. If we could simply reuse an already existing one, it

[HACKERS] Review: rollback sequence reset for TRUNCATE ... RESTART IDENTITY

2010-11-17 Thread Marc Cousin
Hi, Here is my review of 'rollback sequence reset for TRUNCATE ... RESTART IDENTITY' patch. - Is the patch in context diff format? It's in git diff format. I guess it's OK ? - Does it apply cleanly to the current git master? Yes - Does it include reasonable tests, necessary doc patches, etc?

Re: [HACKERS] changing MyDatabaseId

2010-11-17 Thread Alvaro Herrera
Excerpts from Markus Wanner's message of mié nov 17 09:57:18 -0300 2010: On 11/17/2010 01:27 PM, Alvaro Herrera wrote: I don't think it's a speed thing only. It would be a great thing to have in autovacuum, for example, where we have constant problem reports because the system failed to

[HACKERS] describe objects, as in pg_depend

2010-11-17 Thread Alvaro Herrera
Hi, A customer of ours (Enova Financial) requested the ability to describe objects in pg_depend. The wiki contains a simplistic SQL snippet that does the task, but only for some of the object types, and it's rather ugly. It struck me that we could fulfill this very easily by exposing the

Re: [HACKERS] changing MyDatabaseId

2010-11-17 Thread Markus Wanner
On 11/17/2010 02:19 PM, Alvaro Herrera wrote: Well, the autovacuum mechanism involves a lot of back-and-forth between launcher and postmaster, which includes some signals, a fork() and backend initialization. The failure possibilities are endless. Fork failure communication is similarly

Re: [HACKERS] Review: rollback sequence reset for TRUNCATE ... RESTART IDENTITY

2010-11-17 Thread Jaime Casanova
On Wed, Nov 17, 2010 at 8:13 AM, Marc Cousin cousinm...@gmail.com wrote: - Does the feature work as advertised? Yes. It works consistently, isn't fooled by savepoints or multiple serials in a table, or concurrent transactions i haven't tested this nor readed the patch but i wondering what

Re: [HACKERS] Review: rollback sequence reset for TRUNCATE ... RESTART IDENTITY

2010-11-17 Thread Marc Cousin
The Wednesday 17 November 2010 15:50:36, Jaime Casanova wrote : On Wed, Nov 17, 2010 at 8:13 AM, Marc Cousin cousinm...@gmail.com wrote: - Does the feature work as advertised? Yes. It works consistently, isn't fooled by savepoints or multiple serials in a table, or concurrent

Re: [HACKERS] unlogged tables

2010-11-17 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: fsync()ing the file at shutdown doesn't seem too bad to me from performance point of view, we tolerate that for all other tables. And you can always truncate the table yourself before shutdown. The objection to that was not about

Re: [HACKERS] describe objects, as in pg_depend

2010-11-17 Thread Tom Lane
Alvaro Herrera alvhe...@alvh.no-ip.org writes: A customer of ours (Enova Financial) requested the ability to describe objects in pg_depend. The wiki contains a simplistic SQL snippet that does the task, but only for some of the object types, and it's rather ugly. It struck me that we could

Re: [HACKERS] Isn't HANDLE 64 bits on Win64?

2010-11-17 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: Do you still have a reference to the page that said they will never be assigned that high? http://msdn.microsoft.com/en-us/library/ms810720.aspx which says USER and GDI handles are sign extended 32b values To facilitate the porting, a

Re: [HACKERS] changing MyDatabaseId

2010-11-17 Thread Tom Lane
Markus Wanner mar...@bluegap.ch writes: On 11/17/2010 02:19 PM, Alvaro Herrera wrote: Well, the autovacuum mechanism involves a lot of back-and-forth between launcher and postmaster, which includes some signals, a fork() and backend initialization. The failure possibilities are endless.

[HACKERS] Indent authentication overloading

2010-11-17 Thread Magnus Hagander
Currently, we overload indent meaning both unix socket authentication and ident over tcp, depending on what type of connection it is. This is quite unfortunate - one of them being one of the most secure options we have, the other one being one of the most *insecure* ones (really? ident over tcp?

Re: [HACKERS] Indent authentication overloading

2010-11-17 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: Currently, we overload indent meaning both unix socket authentication and ident over tcp, depending on what type of connection it is. This is quite unfortunate - one of them being one of the most secure options we have, the other one being one of the

Re: [HACKERS] describe objects, as in pg_depend

2010-11-17 Thread Alvaro Herrera
Excerpts from Tom Lane's message of mié nov 17 12:20:06 -0300 2010: Alvaro Herrera alvhe...@alvh.no-ip.org writes: A customer of ours (Enova Financial) requested the ability to describe objects in pg_depend. The wiki contains a simplistic SQL snippet that does the task, but only for some

Re: [HACKERS] Indent authentication overloading

2010-11-17 Thread Magnus Hagander
On Wed, Nov 17, 2010 at 16:39, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: Currently, we overload indent meaning both unix socket authentication and ident over tcp, depending on what type of connection it is. This is quite unfortunate - one of them being one

Re: [HACKERS] describe objects, as in pg_depend

2010-11-17 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Tom Lane's message of mié nov 17 12:20:06 -0300 2010: What's the point of the InvalidOid check? If the check is not there, the calling query will have to prevent the function from being called on rows having OID=0 in pg_depend.

Re: [HACKERS] changing MyDatabaseId

2010-11-17 Thread Markus Wanner
On 11/17/2010 04:25 PM, Tom Lane wrote: I'm afraid that any such change would trade a visible, safe failure mechanism (no avworker) for invisible, impossible-to-debug data corruption scenarios (due to failure to reset some bit of cached state). It certainly won't give me any warm fuzzy feeling

Re: [HACKERS] unlogged tables

2010-11-17 Thread Greg Stark
On Wed, Nov 17, 2010 at 3:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: fsync()ing the file at shutdown doesn't seem too bad to me from performance point of view, we tolerate that for all other tables. And you can always truncate the

Re: [HACKERS] contrib: auth_delay module

2010-11-17 Thread Ross J. Reedstrom
On Tue, Nov 16, 2010 at 09:41:37PM -0500, Robert Haas wrote: On Tue, Nov 16, 2010 at 8:15 PM, KaiGai Kohei kai...@ak.jp.nec.com wrote: If we don't need a PoC module for each new hooks, I'm not strongly motivated to push it into contrib tree. How about your opinion? I'd say let it go,

Re: [HACKERS] duplicate connection failure messages

2010-11-17 Thread Bruce Momjian
Alvaro Herrera wrote: Excerpts from Bruce Momjian's message of s??b nov 13 22:36:31 -0300 2010: OK, I found out how to get the IP address with the attached patch. The problem is that only pghost is set, never pghostaddr. I am not even sure how that would get set for this code because my

Re: [HACKERS] Indent authentication overloading

2010-11-17 Thread David Fetter
On Wed, Nov 17, 2010 at 04:43:00PM +0100, Magnus Hagander wrote: On Wed, Nov 17, 2010 at 16:39, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: Currently, we overload indent meaning both unix socket authentication and ident over tcp, depending on what type of

Re: [HACKERS] Indent authentication overloading

2010-11-17 Thread Jeroen Vermeulen
On 2010-11-17 22:43, Magnus Hagander wrote: at the advantage of not confusing new users. We could of course also just drop ident-over-tcp completely, but there might be some poor guy out there who actually *uses* it :-) As far as I know, companies do use it in their internal networks where

Re: [HACKERS] Indent authentication overloading

2010-11-17 Thread Peter Eisentraut
On ons, 2010-11-17 at 16:35 +0100, Magnus Hagander wrote: Currently, we overload indent meaning both unix socket authentication and ident over tcp, depending on what type of connection it is. This is quite unfortunate - one of them being one of the most secure options we have, the other one

Re: [HACKERS] unlogged tables

2010-11-17 Thread Kevin Grittner
Greg Stark gsst...@mit.edu wrote: If this is a clean shutdown then all the non-unlogged tables have been checkpointed so they should have no dirty pages in them anyways. So we could just fsync everything. Or just all the unlogged tables. -Kevin -- Sent via pgsql-hackers mailing list

Re: [HACKERS] changing MyDatabaseId

2010-11-17 Thread Robert Haas
On Wed, Nov 17, 2010 at 5:04 AM, Markus Wanner mar...@bluegap.ch wrote: The question obviously is whether or not this is faster than just terminating one backend and starting a new one. I agree. Which basically costs an additional termination and re-creation of a process (i.e. fork())

Re: [HACKERS] Indent authentication overloading

2010-11-17 Thread Magnus Hagander
On Wed, Nov 17, 2010 at 17:10, Jeroen Vermeulen j...@xs4all.nl wrote: On 2010-11-17 22:43, Magnus Hagander wrote: at the advantage of not confusing new users. We could of course also just drop ident-over-tcp completely, but there might be some poor guy out there who actually *uses* it :-)

Re: [HACKERS] unlogged tables

2010-11-17 Thread Robert Haas
On Wed, Nov 17, 2010 at 11:00 AM, Greg Stark gsst...@mit.edu wrote: On Wed, Nov 17, 2010 at 3:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: fsync()ing the file at shutdown doesn't seem too bad to me from performance point of view, we

Re: [HACKERS] Indent authentication overloading

2010-11-17 Thread Magnus Hagander
On Wed, Nov 17, 2010 at 17:31, Peter Eisentraut pete...@gmx.net wrote: On ons, 2010-11-17 at 16:35 +0100, Magnus Hagander wrote: Currently, we overload indent meaning both unix socket authentication and ident over tcp, depending on what type of connection it is. This is quite unfortunate - one

Re: [HACKERS] contrib: auth_delay module

2010-11-17 Thread Robert Haas
On Wed, Nov 17, 2010 at 10:32 AM, Ross J. Reedstrom reeds...@rice.edu wrote: On Tue, Nov 16, 2010 at 09:41:37PM -0500, Robert Haas wrote: On Tue, Nov 16, 2010 at 8:15 PM, KaiGai Kohei kai...@ak.jp.nec.com wrote: If we don't need a PoC module for each new hooks, I'm not strongly motivated to

Re: [HACKERS] Extensible executor nodes for preparation of SQL/MED

2010-11-17 Thread Robert Haas
On Wed, Nov 17, 2010 at 2:13 AM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: On Wed, Nov 17, 2010 at 10:51, Itagaki Takahiro itagaki.takah...@gmail.com wrote: On Wed, Nov 17, 2010 at 03:36, Robert Haas robertmh...@gmail.com wrote: On Tue, Nov 16, 2010 at 1:28 PM, Tom Lane

Re: [HACKERS] HOT updates in index-less tables

2010-11-17 Thread Merlin Moncure
On Sun, Nov 14, 2010 at 1:12 PM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 14.11.2010 00:29, Robert Haas wrote: On Sat, Nov 13, 2010 at 12:13 PM, Tom Lanet...@sss.pgh.pa.us  wrote: Hannu Krosingha...@2ndquadrant.com  writes: On Sat, 2010-11-13 at 10:51 -0500, Tom Lane

Re: [HACKERS] GiST insert algorithm rewrite

2010-11-17 Thread Teodor Sigaev
Hmm, will have to do some benchmarking on that. I'm using the Consistent function when walking down to check if the downlink needs to be updated, and assumed that it would be insignificant compared to the cost of calling Penalty on all the keys on the page. Why consistent?! It's impossible - you

Re: [HACKERS] changing MyDatabaseId

2010-11-17 Thread Greg Stark
On Wed, Nov 17, 2010 at 4:52 PM, Robert Haas robertmh...@gmail.com wrote: However, that test doesn't capture everything.  For example, imagine a connection pooler sitting in front of PG.  Rebinding to a new database means disconnecting a TCP connection and establishing a new one. Switching

Re: [HACKERS] GiST insert algorithm rewrite

2010-11-17 Thread Teodor Sigaev
Sorry, I missed beginning of discussion on GiST, so I read it on the web mail archive. You wrote: http://archives.postgresql.org/pgsql-hackers/2010-11/msg00939.php [skip] 0. (the child page is locked) 1. The parent page is locked. 2. The child page is split. The original page becomes the left

Re: [HACKERS] unlogged tables

2010-11-17 Thread Josh Berkus
Robert, All: I hope you're following the thread on -general about this feature. We're getting a lot of feedback. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com --

Re: [HACKERS] Indent authentication overloading

2010-11-17 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: If it was a matter of changing it for those who use ident over tcp, I really wouldn't hesitate - they're few :-) But the problem is that it's the ident-over-tcp that's correctly named, not the other one... Yeah, renaming the TCP version would be

Re: [HACKERS] GiST insert algorithm rewrite

2010-11-17 Thread Heikki Linnakangas
On 17.11.2010 19:46, Teodor Sigaev wrote: I disagree with that opinion: if we crash between 2 and 3 then why will somebody update parent before WAL replay? WAL replay process in this case should complete child split by inserting invalid pointer and tree become correct again, although it needs to

Re: [HACKERS] unlogged tables

2010-11-17 Thread Heikki Linnakangas
On 17.11.2010 17:11, Tom Lane wrote: Heikki Linnakangasheikki.linnakan...@enterprisedb.com writes: fsync()ing the file at shutdown doesn't seem too bad to me from performance point of view, we tolerate that for all other tables. And you can always truncate the table yourself before shutdown.

Re: [HACKERS] changing MyDatabaseId

2010-11-17 Thread Robert Haas
On Wed, Nov 17, 2010 at 12:42 PM, Greg Stark gsst...@mit.edu wrote: On Wed, Nov 17, 2010 at 4:52 PM, Robert Haas robertmh...@gmail.com wrote: However, that test doesn't capture everything.  For example, imagine a connection pooler sitting in front of PG.  Rebinding to a new database means

Re: [HACKERS] Review: rollback sequence reset for TRUNCATE ... RESTART IDENTITY

2010-11-17 Thread Tom Lane
Marc Cousin cousinm...@gmail.com writes: - Does the feature work as advertised? Yes. It works consistently, isn't fooled by savepoints or multiple serials in a table, or concurrent transactions I think there's a rather nasty problem here, which is what to do with the cached nextval/currval

Re: [HACKERS] unlogged tables

2010-11-17 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 17.11.2010 17:11, Tom Lane wrote: The objection to that was not about performance. It was about how to find out what needs to be fsync'd. I must be missing something: we handle that just fine with normal tables, why is it a

Re: [HACKERS] unlogged tables

2010-11-17 Thread Robert Haas
On Wed, Nov 17, 2010 at 1:11 PM, Josh Berkus j...@agliodbs.com wrote: Robert, All: I hope you're following the thread on -general about this feature. We're getting a lot of feedback. I haven't been; I'm not subscribed to general; it'd be useful to CC me next time. Reading through the thread

Re: [HACKERS] Re: [BUGS] BUG #5650: Postgres service showing as stopped when in fact it is running

2010-11-17 Thread Bruce Momjian
Magnus Hagander wrote: I basically report the connection error string if it starts with FATAL:. I originally tried to check for an ERRCODE_INVALID_PASSWORD error field (see // comments), but it seems there is no way to access this, i.e. PQgetResult(conn) on a connection failure is always

Re: [HACKERS] Re: [BUGS] BUG #5650: Postgres service showing as stopped when in fact it is running

2010-11-17 Thread Magnus Hagander
On Wed, Nov 17, 2010 at 19:50, Bruce Momjian br...@momjian.us wrote: Magnus Hagander wrote: Does this actually solve the *problem*, though? The problem is not what is reported  on stdout/stderr, the problem is that the net result is that the server is reported as not started (by the service

[HACKERS] git diff script is not portable

2010-11-17 Thread Bruce Momjian
Looking here: http://wiki.postgresql.org/wiki/Working_with_Git the script we recommend for doing context diffs for git, git-external-diff, is not portable: http://anarazel.de/pg/git-external-diff It uses diff -L, which is not supported by FreeBSD, and I imagine many other

Re: [HACKERS] unlogged tables

2010-11-17 Thread Robert Haas
On Wed, Nov 17, 2010 at 1:46 PM, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: On 17.11.2010 17:11, Tom Lane wrote: The objection to that was not about performance.  It was about how to find out what needs to be fsync'd. I must be missing

Re: [HACKERS] Re: [BUGS] BUG #5650: Postgres service showing as stopped when in fact it is running

2010-11-17 Thread Bruce Momjian
Magnus Hagander wrote: On Wed, Nov 17, 2010 at 19:50, Bruce Momjian br...@momjian.us wrote: Magnus Hagander wrote: Does this actually solve the *problem*, though? The problem is not what is reported ?on stdout/stderr, the problem is that the net result is that the server is reported as

Re: [HACKERS] git diff script is not portable

2010-11-17 Thread Magnus Hagander
On Wed, Nov 17, 2010 at 19:54, Bruce Momjian br...@momjian.us wrote: Looking here:        http://wiki.postgresql.org/wiki/Working_with_Git the script we recommend for doing context diffs for git, git-external-diff, is not portable:        http://anarazel.de/pg/git-external-diff It uses

[HACKERS] MergeAppend costing

2010-11-17 Thread Robert Haas
See the attached test case. With that setup, this uses MergeAppend: explain select * from ma_parent order by id limit 10; But this one does not: explain select * from ma_parent order by name limit 10; ...which seems odd, because the index on ma_child1 and sorting the other two ought to still

Re: [HACKERS] git diff script is not portable

2010-11-17 Thread Andres Freund
On Wednesday 17 November 2010 19:54:49 Bruce Momjian wrote: Looking here: http://wiki.postgresql.org/wiki/Working_with_Git the script we recommend for doing context diffs for git, git-external-diff, is not portable: http://anarazel.de/pg/git-external-diff It uses diff -L,

Re: [HACKERS] Re: [BUGS] BUG #5650: Postgres service showing as stopped when in fact it is running

2010-11-17 Thread Magnus Hagander
On Wed, Nov 17, 2010 at 19:57, Bruce Momjian br...@momjian.us wrote: Magnus Hagander wrote: On Wed, Nov 17, 2010 at 19:50, Bruce Momjian br...@momjian.us wrote: Magnus Hagander wrote: Does this actually solve the *problem*, though? The problem is not what is reported ?on stdout/stderr, the

Re: [HACKERS] changing MyDatabaseId

2010-11-17 Thread Greg Stark
On Wed, Nov 17, 2010 at 6:33 PM, Robert Haas robertmh...@gmail.com wrote: I think you're missing the point.  If we switch databases, all cached relations and plans have to be flushed anyway.  We're talking about what might NOT need to be flushed on switching databases. Oh sorry, yes, I missed

Re: [HACKERS] unlogged tables

2010-11-17 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Nov 17, 2010 at 1:46 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert is probably going to object that he wanted to prevent any fsyncing for unlogged tables, but the discussion over in pgsql-general is crystal clear that people do NOT want to lose

Re: [HACKERS] Re: [BUGS] BUG #5650: Postgres service showing as stopped when in fact it is running

2010-11-17 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Wed, Nov 17, 2010 at 19:57, Bruce Momjian br...@momjian.us wrote: Is FATAL, in general, enough to conclude the server is running? No - specifically, we will send FATAL when the database system is starting up, which is exactly the one we want to

Re: [HACKERS] unlogged tables

2010-11-17 Thread Kenneth Marshall
On Wed, Nov 17, 2010 at 02:16:06PM -0500, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Nov 17, 2010 at 1:46 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert is probably going to object that he wanted to prevent any fsyncing for unlogged tables, but the discussion over in

Re: [HACKERS] unlogged tables

2010-11-17 Thread Andrew Dunstan
On 11/17/2010 02:22 PM, Kenneth Marshall wrote: On Wed, Nov 17, 2010 at 02:16:06PM -0500, Tom Lane wrote: Robert Haasrobertmh...@gmail.com writes: On Wed, Nov 17, 2010 at 1:46 PM, Tom Lanet...@sss.pgh.pa.us wrote: Robert is probably going to object that he wanted to prevent any fsyncing

Re: [HACKERS] unlogged tables

2010-11-17 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mié nov 17 15:48:56 -0300 2010: On Wed, Nov 17, 2010 at 1:11 PM, Josh Berkus j...@agliodbs.com wrote: Robert, All: I hope you're following the thread on -general about this feature. We're getting a lot of feedback. I haven't been; I'm not

Re: [HACKERS] unlogged tables

2010-11-17 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 11/17/2010 02:22 PM, Kenneth Marshall wrote: I would be fine with only having a safe shutdown with unlogged tables and skip the checkpoint I/O all other times. Yeah, I was just thinking something like that would be good, and should overcome

Re: [HACKERS] unlogged tables

2010-11-17 Thread Josh Berkus
Now, a few minutes ago Robert was muttering about supporting more than one kind of degraded-reliability table. I could see inventing unlogged tables, which means exactly that (no xlog support, but we still checkpoint/fsync as usual), and unsynced tables which also/instead suppress fsync

Re: [HACKERS] unlogged tables

2010-11-17 Thread Tom Lane
[ forgot to comment on this part ] Andrew Dunstan and...@dunslane.net writes: To answer another point I see Tom made on the -general list: while individual backends may crash from time to time, crashes of the whole Postgres server are very rare in my experience in production environments.

Re: [HACKERS] Re: [BUGS] BUG #5650: Postgres service showing as stopped when in fact it is running

2010-11-17 Thread Bruce Momjian
Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: On Wed, Nov 17, 2010 at 19:57, Bruce Momjian br...@momjian.us wrote: Is FATAL, in general, enough to conclude the server is running? No - specifically, we will send FATAL when the database system is starting up, which is

Re: [HACKERS] git diff script is not portable

2010-11-17 Thread Bruce Momjian
Magnus Hagander wrote: On Wed, Nov 17, 2010 at 19:54, Bruce Momjian br...@momjian.us wrote: Looking here: ? ? ? ?http://wiki.postgresql.org/wiki/Working_with_Git the script we recommend for doing context diffs for git, git-external-diff, is not portable: ? ? ?

Re: [HACKERS] unlogged tables

2010-11-17 Thread Andrew Dunstan
On 11/17/2010 02:44 PM, Tom Lane wrote: [ forgot to comment on this part ] Andrew Dunstanand...@dunslane.net writes: To answer another point I see Tom made on the -general list: while individual backends may crash from time to time, crashes of the whole Postgres server are very rare in my

Re: [HACKERS] unlogged tables

2010-11-17 Thread Robert Haas
On Wed, Nov 17, 2010 at 2:31 PM, Andrew Dunstan and...@dunslane.net wrote: The customer is always right, and I think we are hearing loud and clear what the customers want.  Please let's not go out of our way to create a feature that isn't what they want. I would be fine with only having a

Re: [HACKERS] Re: [BUGS] BUG #5650: Postgres service showing as stopped when in fact it is running

2010-11-17 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: Agreed. So how do we pass that info to libpq without exceeding the value of fixing this problem? Should we parse pg_controldata output? pg_upgrade could use machine-readable output from that too. pg_controldata seems 100% unrelated to this problem.

Re: [HACKERS] Review: rollback sequence reset for TRUNCATE ... RESTART IDENTITY

2010-11-17 Thread Marc Cousin
The Wednesday 17 November 2010 19:41:19, Tom Lane wrote : Marc Cousin cousinm...@gmail.com writes: - Does the feature work as advertised? Yes. It works consistently, isn't fooled by savepoints or multiple serials in a table, or concurrent transactions I think there's a rather nasty

Re: [HACKERS] Re: [BUGS] BUG #5650: Postgres service showing as stopped when in fact it is running

2010-11-17 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Agreed. So how do we pass that info to libpq without exceeding the value of fixing this problem? Should we parse pg_controldata output? pg_upgrade could use machine-readable output from that too. pg_controldata seems 100%

Re: [HACKERS] ALTER TYPE recursion to typed tables

2010-11-17 Thread Dimitri Fontaine
Peter Eisentraut pete...@gmx.net writes: Here is the patch that adds [RESTRICT|CASCADE] to ALTER TYPE ... ADD/ALTER/DROP/RENAME ATTRIBUTE, so that recurses to typed tables. And here's my commitfest review of it: - patch applies cleanly - adds regression tests - passes them - is useful and

Re: [HACKERS] unlogged tables

2010-11-17 Thread Robert Haas
On Wed, Nov 17, 2010 at 2:42 PM, Josh Berkus j...@agliodbs.com wrote: Now, a few minutes ago Robert was muttering about supporting more than one kind of degraded-reliability table.  I could see inventing unlogged tables, which means exactly that (no xlog support, but we still checkpoint/fsync

Re: [HACKERS] Review: rollback sequence reset for TRUNCATE ... RESTART IDENTITY

2010-11-17 Thread Steve Singer
On 10-11-17 03:00 PM, Marc Cousin wrote: The Wednesday 17 November 2010 19:41:19, Tom Lane wrote : Marc Cousincousinm...@gmail.com writes: - Does the feature work as advertised? Yes. It works consistently, isn't fooled by savepoints or multiple serials in a table, or concurrent transactions

Re: [HACKERS] unlogged tables

2010-11-17 Thread Andres Freund
On Wednesday 17 November 2010 20:54:14 Robert Haas wrote: On Wed, Nov 17, 2010 at 2:31 PM, Andrew Dunstan and...@dunslane.net wrote: The customer is always right, and I think we are hearing loud and clear what the customers want. Please let's not go out of our way to create a feature that

Re: [HACKERS] unlogged tables

2010-11-17 Thread Robert Haas
On Wed, Nov 17, 2010 at 2:37 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andrew Dunstan and...@dunslane.net writes: On 11/17/2010 02:22 PM, Kenneth Marshall wrote: I would be fine with only having a safe shutdown with unlogged tables and skip the checkpoint I/O all other times. Yeah, I was just

Re: [HACKERS] unlogged tables

2010-11-17 Thread Andrew Dunstan
On 11/17/2010 03:37 PM, Robert Haas wrote: I don't particularly care for the name UNSYNCED, and I'm starting not to like UNLOGGED much either, although at least that one is an actual word. PERMANENT and the flavors of TEMPORARY are a reasonably comprehensible as a description of

Re: [HACKERS] unlogged tables

2010-11-17 Thread Robert Haas
On Wed, Nov 17, 2010 at 3:35 PM, Andres Freund and...@anarazel.de wrote: The customer is always right, but the informed customer makes better decisions than the uninformed customer.  This idea, as proposed, does not work.  If you only include dirty buffers at the final checkpoint before

Re: [HACKERS] unlogged tables

2010-11-17 Thread David Fetter
On Wed, Nov 17, 2010 at 03:48:52PM -0500, Andrew Dunstan wrote: On 11/17/2010 03:37 PM, Robert Haas wrote: I don't particularly care for the name UNSYNCED, and I'm starting not to like UNLOGGED much either, although at least that one is an actual word. PERMANENT and the flavors of TEMPORARY

Re: [HACKERS] unlogged tables

2010-11-17 Thread Steve Crawford
On 11/17/2010 12:48 PM, Andrew Dunstan wrote: Maybe VOLATILE for UNSYNCED? Not sure about UNLOGGED. UNSAFE and EXTREMELY_UNSAFE?? :) Cheers, Steve -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] unlogged tables

2010-11-17 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: OK, so we're proposing a hierarchy like this. 1. PERMANENT (already exists). 2. UNLOGGED (what this patch currently implements). 3. UNSYNCED (future work). 4. GLOBAL TEMPORARY (future work). 5. LOCAL TEMPORARY (our current temp tables).

Re: [HACKERS] unlogged tables

2010-11-17 Thread A.M.
On Nov 17, 2010, at 4:00 PM, Kevin Grittner wrote: Robert Haas robertmh...@gmail.com wrote: OK, so we're proposing a hierarchy like this. 1. PERMANENT (already exists). 2. UNLOGGED (what this patch currently implements). 3. UNSYNCED (future work). 4. GLOBAL TEMPORARY (future

Re: [HACKERS] unlogged tables

2010-11-17 Thread Robert Haas
On Wed, Nov 17, 2010 at 4:00 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Robert Haas robertmh...@gmail.com wrote: OK, so we're proposing a hierarchy like this. 1. PERMANENT (already exists). 2. UNLOGGED (what this patch currently implements). 3. UNSYNCED (future work). 4.

Re: [HACKERS] unlogged tables

2010-11-17 Thread Andrew Dunstan
On 11/17/2010 04:00 PM, Kevin Grittner wrote: Actually, the UNSYNCED and UNLOGGED seem fairly clear I think Robert's right. These names won't convey much to someone not steeped in our technology. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] unlogged tables

2010-11-17 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mié nov 17 17:51:37 -0300 2010: On Wed, Nov 17, 2010 at 3:35 PM, Andres Freund and...@anarazel.de wrote: How can you get a buffer which was no written out *at all*? Do you want to force all such pages to stay in shared_buffers? That sounds quite a bit

Re: [HACKERS] unlogged tables

2010-11-17 Thread David Fetter
On Wed, Nov 17, 2010 at 04:05:56PM -0500, Robert Haas wrote: On Wed, Nov 17, 2010 at 4:00 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Robert Haas robertmh...@gmail.com wrote: OK, so we're proposing a hierarchy like this. 1. PERMANENT (already exists). 2. UNLOGGED (what

Re: [HACKERS] unlogged tables

2010-11-17 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mié nov 17 18:05:56 -0300 2010: CREATE VOLATILE TABLE blow_me_away (k text, v text) SOME OTHER WORDS THAT EXPLAIN THE DETAILS GO HERE; What about some reloptions? -- Álvaro Herrera alvhe...@commandprompt.com The PostgreSQL Company - Command Prompt, Inc.

Re: [HACKERS] unlogged tables

2010-11-17 Thread Steve Crawford
On 11/17/2010 11:44 AM, Tom Lane wrote: ...because a backend crash has to be assumed to have corrupted unlogged tables... So in a typical use-case, say storing session data on a web-site, one crashed backend could wreck sessions for some or all of the site? Is there a mechanism in the

Re: [HACKERS] Review: rollback sequence reset for TRUNCATE ... RESTART IDENTITY

2010-11-17 Thread Tom Lane
Steve Singer ssin...@ca.afilias.info writes: I will look at addressing this in an updated patch. I've nearly finished revising the patch already, don't worry about it. Should currval really be used after a failed transaction ? Right now, we can have a value that has been generated inside a

Re: [HACKERS] Re: [BUGS] BUG #5650: Postgres service showing as stopped when in fact it is running

2010-11-17 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: Tom Lane wrote: Possibly the cleanest fix is to implement pg_ping as a libpq function. You do have to distinguish connection failures (ie connection refused) from errors that came back from the postmaster, and the easiest place to be doing that is inside

Re: [HACKERS] We need to log aborted autovacuums

2010-11-17 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: I've been trying to diagnose in a production database why certain tables never get autovacuumed despite having a substantial % of updates. The obvious reason is locks blocking autovacuum from vacuuming the table ... but the trick is we don't log such

Re: [HACKERS] MergeAppend costing

2010-11-17 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: See the attached test case. With that setup, this uses MergeAppend: explain select * from ma_parent order by id limit 10; But this one does not: explain select * from ma_parent order by name limit 10; ...which seems odd, because the index on

Re: [HACKERS] We need to log aborted autovacuums

2010-11-17 Thread Josh Berkus
It's hard to tell, because you're just handwaving about what it is you think isn't being logged; nor is it clear whether you have any evidence that locks are the problem. Offhand I'd think it at least as likely that autovacuum thinks it doesn't need to do anything, perhaps because of a

Re: [HACKERS] We need to log aborted autovacuums

2010-11-17 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: There *is* an elog(DEBUG3) in autovacuum.c that reports whether autovac thinks a table needs vacuumed/analyzed ... maybe that needs to be a tad more user-accessible. Yeah, it would be really good to be able to log that without bumping the log levels of

Re: [HACKERS] We need to log aborted autovacuums

2010-11-17 Thread Itagaki Takahiro
On Thu, Nov 18, 2010 at 08:35, Tom Lane t...@sss.pgh.pa.us wrote: Yeah, it would be really good to be able to log that without bumping the log levels of the server in general to DEBUG3. Well, the way to deal with that would be to add a GUC that enables reporting of those messages at LOG

Re: [HACKERS] We need to log aborted autovacuums

2010-11-17 Thread Tom Lane
Itagaki Takahiro itagaki.takah...@gmail.com writes: On Thu, Nov 18, 2010 at 08:35, Tom Lane t...@sss.pgh.pa.us wrote: Well, the way to deal with that would be to add a GUC that enables reporting of those messages at LOG level.  But it's a bit hard to argue that we need such a thing without

  1   2   >