Re: [HACKERS] Listen / Notify - what to do when the queue is full

2010-02-14 Thread Simon Riggs
On Thu, 2010-02-11 at 00:52 +0100, Joachim Wieland wrote: On Mon, Feb 8, 2010 at 5:16 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: These are the on-disk notifications, right? It seems to me a bit wasteful to store channel name always as NAMEDATALEN bytes. Can we truncate it at

Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

2010-02-14 Thread Greg Stark
On Fri, Feb 12, 2010 at 3:49 PM, Robert Haas robertmh...@gmail.com wrote: Greg Stark, have you managed to get your access issues sorted out?  If Yep, will look at this today. -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] [NOVICE] Python verison for build in config.pl (Win32)

2010-02-14 Thread Magnus Hagander
2010/1/20 James William Pye li...@jwp.name: On Jan 20, 2010, at 12:27 PM, Magnus Hagander wrote: Well, it needs the version to match it to the DLL name. For python 2.6, it needs python26.dll. But yes, there should probably be some way to ask python itself about that - that would be the

Re: [HACKERS] Streaming Replication on win32

2010-02-14 Thread Magnus Hagander
2010/2/8 Fujii Masao masao.fu...@gmail.com: On Mon, Jan 18, 2010 at 11:46 PM, Magnus Hagander mag...@hagander.net wrote: From what I can tell, this indicates that pq_getbyte_if_available() is not working - because it's supposed to never block, right? Right, it's not supposed to block. This

[HACKERS] Optimizing GetConflictingVirtualXIDs()

2010-02-14 Thread Simon Riggs
Optimize GetConflictingVirtualXIDs() in roughly the same manner we optimize TransactionIdIsInProgress(). Views? -- Simon Riggs www.2ndQuadrant.com diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c index 12de877..7f9b10e 100644 ---

[HACKERS] function to display ddl

2010-02-14 Thread Little, Douglas
Hi, Is there a PG command or fuction that will return table ddl? Thanks Doug Little Sr. Data Warehouse Architect | Enterprise Data Management | Orbitz Worldwide 500 W. Madison, Suite 1000 Chicago IL 60661| Office 312.260.2588 | Fax 312.894.5164 | Cell 847-997-5741

Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

2010-02-14 Thread Greg Stark
On Sun, Feb 14, 2010 at 2:03 PM, Greg Stark gsst...@mit.edu wrote: On Fri, Feb 12, 2010 at 3:49 PM, Robert Haas robertmh...@gmail.com wrote: Greg Stark, have you managed to get your access issues sorted out?  If Yep, will look at this today. So I think we have a bigger problem than just

Re: [HACKERS] knngist patch support

2010-02-14 Thread Yeb Havinga
Dimitri Fontaine wrote: Then there's the metric space which is a data type with a distance function. This function must be non-negative, commutative, etc. So I guess what we need here is a Operator Group to define our plus and minus operators, and the fact that it's a group says (by convention,

Re: [HACKERS] function to display ddl

2010-02-14 Thread Yeb Havinga
Little, Douglas wrote: Hi, Is there a PG command or fuction that will return table ddl? If you just want the definition,in psql type \d tablename. To dump ddl the pg_dump with proper arguments can dump just the ddl of a single table. (see pg_dump --help) I'm not aware of a command or

Re: [HACKERS] Optimizing GetConflictingVirtualXIDs()

2010-02-14 Thread Greg Stark
On Sun, Feb 14, 2010 at 2:59 PM, Simon Riggs si...@2ndquadrant.com wrote: Optimize GetConflictingVirtualXIDs() in roughly the same manner we optimize TransactionIdIsInProgress(). Views? EINSUFFICIENTEXPLANATION :) -- greg -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

2010-02-14 Thread Tom Lane
Greg Stark gsst...@mit.edu writes: So I think we have a bigger problem than just copydir.c. It seems to me we should be fsyncing the table space data directories on every checkpoint. Is there any evidence that anyone anywhere has ever lost data because of a lack of directory fsyncs? I sure

Re: [HACKERS] function to display ddl

2010-02-14 Thread Ross J. Reedstrom
On Sun, Feb 14, 2010 at 05:08:05PM +0100, Yeb Havinga wrote: Little, Douglas wrote: Hi, Is there a PG command or fuction that will return table ddl? If you just want the definition,in psql type \d tablename. To dump ddl the pg_dump with proper arguments can dump just the ddl of a

Re: [HACKERS] knngist patch support

2010-02-14 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: OK. In that case, any objections to my applying the attached patch, which I believe implements this as you suggested? Um, did you test this version? It looks like the macros are still defined according to the idea that SearchSysCache takes five

Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

2010-02-14 Thread Andres Freund
On Sunday 14 February 2010 18:11:39 Tom Lane wrote: Greg Stark gsst...@mit.edu writes: So I think we have a bigger problem than just copydir.c. It seems to me we should be fsyncing the table space data directories on every checkpoint. Is there any evidence that anyone anywhere has ever

Re: [HACKERS] psycopg2 license changed

2010-02-14 Thread Josh Berkus
On 2/13/10 5:13 PM, Federico Di Gregorio wrote: Hi *, I just wanted all interested people know that psycopg2 2.0.14 to be released in the next few days will be under the LGPL3 + OpenSSL exception (example code and tests under the LGPL3 alone because they are never linked to OpenSSL).

Re: [HACKERS] Streaming Replication docs

2010-02-14 Thread Josh Berkus
On 2/13/10 10:17 AM, Greg Stark wrote: But having the separate recovery.conf seems kind of pointless legacy structure at this point. Agreed, but I've been trying to minimize my tinkering with HS/SR. Several things would need to be changed to get rid of Recovery.conf for HS and PITR. I expect

Re: [HACKERS] psycopg2 license changed

2010-02-14 Thread Federico Di Gregorio
On 14/02/2010 18:33, Josh Berkus wrote: I just wanted all interested people know that psycopg2 2.0.14 to be released in the next few days will be under the LGPL3 + OpenSSL exception (example code and tests under the LGPL3 alone because they are never linked to OpenSSL). Yaaay! Of

Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

2010-02-14 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On Sunday 14 February 2010 18:11:39 Tom Lane wrote: It seems to me that we're talking about a huge hit in both code complexity and performance to deal with a problem that doesn't actually occur in the field; and which furthermore is trivially solved on

Re: [HACKERS] Optimizing GetConflictingVirtualXIDs()

2010-02-14 Thread Simon Riggs
On Sun, 2010-02-14 at 17:06 +, Greg Stark wrote: On Sun, Feb 14, 2010 at 2:59 PM, Simon Riggs si...@2ndquadrant.com wrote: Optimize GetConflictingVirtualXIDs() in roughly the same manner we optimize TransactionIdIsInProgress(). Views? EINSUFFICIENTEXPLANATION :) ...I like that

Re: [HACKERS] psycopg2 license changed

2010-02-14 Thread Devrim GÜNDÜZ
On Sun, 2010-02-14 at 18:37 +0100, Federico Di Gregorio wrote: Of course, now I'm on the hook to fix bugs. Bugs? Which bugs? :) Come on, you know, some people report bugs sometimes ;) -- Devrim GÜNDÜZ, RHCE PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer devrim~gunduz.org,

Re: [HACKERS] Listen / Notify - what to do when the queue is full

2010-02-14 Thread Simon Riggs
On Sun, 2010-02-14 at 17:22 +0100, Joachim Wieland wrote: * There is no mention of what to do with pg_notify at checkpoint. Look at how pg_subtrans handles this. Should pg_notify do the same? Actually we don't care... We even hope that the pg_notify pages are not flushed at all.

Re: [HACKERS] Optimizing GetConflictingVirtualXIDs()

2010-02-14 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Sun, 2010-02-14 at 17:06 +, Greg Stark wrote: EINSUFFICIENTEXPLANATION :) ...I like that error code. I think more EINSUFFICIENTCOMMENTS. The patch should also add a comment to the function, along the lines of While we have the lock, also

Re: [HACKERS] CommitFest Status Summary - 2010-02-14

2010-02-14 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: * Listen / Notify rewrite. This is the only one of the remaining patches that is not marked as Ready for Committer, but I think it would be good if someone (probably Tom) at least took a look at it. I'm not sure if it's committable at this point, but

Re: [HACKERS] knngist patch support

2010-02-14 Thread Robert Haas
On Sun, Feb 14, 2010 at 12:24 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: OK.  In that case, any objections to my applying the attached patch, which I believe implements this as you suggested? Um, did you test this version?  It looks like the macros are

Re: [HACKERS] CommitFest Status Summary - 2010-02-14

2010-02-14 Thread Josh Berkus
I will look at this one. It'd be nice to get it in if at all possible, because the existing listen/notify infrastructure won't play very nicely with HS --- eg, inspecting pg_listener on the slave might yield the false impression that some of the slave-side backends had active LISTENs

Re: [HACKERS] Listen / Notify - what to do when the queue is full

2010-02-14 Thread Tom Lane
Joachim Wieland j...@mcknight.de writes: + #define ERRCODE_TOO_MANY_ENTRIESMAKE_SQLSTATE('5','4', '0','3','1') Do you have any evidence that there is actually a DB2 error code matching this, or is this errcode just invented? The one page Google finds doesn't list it:

Re: [HACKERS] Confusion over Python drivers

2010-02-14 Thread Florian Weimer
* Jeff Davis: Agreed. Ultimately, the conversion has to be done somewhere, but I don't believe the driver is the place for it. Type conversions are always going to be imperfect, and this has some important consequences: * The type conversion system will be endlessly tweaked to improve it *

Re: [HACKERS] Streaming Replication docs

2010-02-14 Thread Peter Eisentraut
On fre, 2010-02-12 at 11:14 +0200, Heikki Linnakangas wrote: So the new layout would be: III. Server Administration ... 21. Managing Databases 22. Localization 23. Routine Database Maintenance Tasks 24. Backup and Restore 25. High Availability, Load Balancing,

[HACKERS] LISTEN/NOTIFY versus encoding conversion

2010-02-14 Thread Tom Lane
There's been a lot of thrashing about whether LISTEN/NOTIFY should restrict payload strings to 7-bit ASCII to avoid possible encoding conversion failures. I was on the side of yes but I'm having second thoughts about it. The point I had failed to think about is that we already restrict notifies

Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync

2010-02-14 Thread Florian Weimer
* Tom Lane: Which options would that be? I am not aware that there any for any of the recent linux filesystems. Shouldn't journaling of metadata be sufficient? You also need to enforce ordering between the directory update and the file update. The file metadata is flushed with fsync(), but

Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync

2010-02-14 Thread Mark Mielke
On 02/14/2010 03:24 PM, Florian Weimer wrote: * Tom Lane: Which options would that be? I am not aware that there any for any of the recent linux filesystems. Shouldn't journaling of metadata be sufficient? You also need to enforce ordering between the directory update and

Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync

2010-02-14 Thread Andres Freund
On Sunday 14 February 2010 21:41:02 Mark Mielke wrote: On 02/14/2010 03:24 PM, Florian Weimer wrote: * Tom Lane: Which options would that be? I am not aware that there any for any of the recent linux filesystems. Shouldn't journaling of metadata be sufficient? You also need to

Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

2010-02-14 Thread Robert Haas
On Sun, Feb 14, 2010 at 10:31 AM, Greg Stark gsst...@mit.edu wrote: On Sun, Feb 14, 2010 at 2:03 PM, Greg Stark gsst...@mit.edu wrote: On Fri, Feb 12, 2010 at 3:49 PM, Robert Haas robertmh...@gmail.com wrote: Greg Stark, have you managed to get your access issues sorted out?  If Yep, will

Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

2010-02-14 Thread Andres Freund
On Sunday 14 February 2010 21:57:08 Robert Haas wrote: On Sun, Feb 14, 2010 at 10:31 AM, Greg Stark gsst...@mit.edu wrote: On Sun, Feb 14, 2010 at 2:03 PM, Greg Stark gsst...@mit.edu wrote: On Fri, Feb 12, 2010 at 3:49 PM, Robert Haas robertmh...@gmail.com wrote: Greg Stark, have you

[HACKERS] subtransaction cancellation in HS

2010-02-14 Thread Andres Freund
Hi Simon, Hi all, I am not sure this is 9.0 material - even if I would like it to get included... If HS wouldnt be new I wouldnt even consider suggesting it, but as its stands its a pretty small change... It simply allows queries involving subtransaction not to get FATALed but canceled. This

Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

2010-02-14 Thread Greg Stark
On Sun, Feb 14, 2010 at 8:57 PM, Robert Haas robertmh...@gmail.com wrote: On a pragmatic note, if this does turn out to be a problem, it's a bug: and we can and do fix bugs whenever we discover them.  But the other part of this patch - to speed up createdb - is a feature - and we are very

Re: [HACKERS] Re: Faster CREATE DATABASE by delaying fsync

2010-02-14 Thread Mark Mielke
On 02/14/2010 03:49 PM, Andres Freund wrote: On Sunday 14 February 2010 21:41:02 Mark Mielke wrote: The widely reported problems, though, did not tend to be a problem with directory changes written too late - but directory changes being written too early. That is, the directory change is

[HACKERS] Explain buffers display units.

2010-02-14 Thread Greg Stark
So this is what I did about my two complaints earlier about the explain buffer patch. a) Changed the line description to Total Buffer Usage which at least hints that it's something more akin to the Total runtime listed at the bottom than the actual time. b) Used units of memory -- I formatted

[HACKERS] LISTEN/NOTIFY and notification timing guarantees

2010-02-14 Thread Tom Lane
The proposed new implementation of listen/notify works by shoving all of a transaction's outgoing notifies into the global queue during pre-commit, then sending PROCSIG_NOTIFY_INTERRUPT to listening backends post-commit. When a listening backend scans the queue, if it hits a message from a

Re: [HACKERS] psql tab completion for DO blocks

2010-02-14 Thread Takahiro Itagaki
David Fetter da...@fetter.org wrote: DO { [ LANGUAGE lang_name ] | code } ... Good catch :) The tab completion patch and documentation fix were committed. Thanks. Regards, --- Takahiro Itagaki NTT Open Source Software Center -- Sent via pgsql-hackers mailing list

Re: [HACKERS] CommitFest Status Summary - 2010-02-14

2010-02-14 Thread Andrew Dunstan
Robert Haas wrote: We're down to 5 patches remaining, and 1 day remaining, so it's time to try to wrap things up. * Package namespace and Safe init cleanup for plperl. Andrew Dunstan is taking care of this one, I believe. I will get this in, with changes as discussed recently. I also

Re: [HACKERS] Explain buffers display units.

2010-02-14 Thread Greg Smith
Greg Stark wrote: b) Used units of memory -- I formatted them with 3 significant digits (unless the unit is bytes or kB where that would be silly). It's just what looked best to my eye. How does this compare with what comes out of pg_size_pretty (src/backend/utils/adt/dbsize.c)? I already

Re: [HACKERS] Streaming Replication on win32

2010-02-14 Thread Fujii Masao
On Sun, Feb 14, 2010 at 11:52 PM, Magnus Hagander mag...@hagander.net wrote: Sorry about the delay in responding to this. Thanks for the response. Remember that the win32 code *always* puts the socket in non-blocking mode. So we can't just teach the layer about it. We need some way to pass

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make standby server continuously retry restoring the next WAL

2010-02-14 Thread Fujii Masao
On Sat, Feb 13, 2010 at 1:10 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Are you thinking of a scenario where remove_command gets stuck, and prevents bgwriter from performing restartpoints while it's stuck? Yes. If there is the archive in the remote server and the network

Re: [HACKERS] Parameter name standby_mode

2010-02-14 Thread Fujii Masao
On Fri, Feb 12, 2010 at 11:46 PM, Tom Lane t...@sss.pgh.pa.us wrote: Even more to the point is that some of them, like PGPORT, are highly likely to be set in a server's environment to point to the server itself.  It would be extremely dangerous to automatically try to start replication just

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-14 Thread Tom Lane
st...@postgresql.org (Greg Stark) writes: Log Message: --- Speed up CREATE DATABASE by deferring the fsyncs until after copying all the data and using posix_fadvise to nudge the OS into flushing it earlier. This also hopefully makes CREATE DATABASE avoid spamming the cache. The

Re: [HACKERS] [COMMITTERS] pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

2010-02-14 Thread Tom Lane
I wrote: The buildfarm indicates that this patch has got some serious issues. Actually, looking closer, some of the Windows machines started failing after the *earlier* patch to add directory fsyncs. regards, tom lane -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Re: [COMMITTERS] pgsql: Reduce the chatter to the log when starting a standby server.

2010-02-14 Thread Fujii Masao
On Sat, Feb 13, 2010 at 4:33 AM, Richard Huxton d...@archonet.com wrote: 2. Why have a finish.replication file, rather than SELECT pg_finish_replication()? I thought that the trigger file method was more easy-to-use for the existing users since it had been used in pg_standby for a long time so

Re: [HACKERS] PostgreSQL::PLPerl::Call - Simple interface for calling SQL functions from PostgreSQL PL/Perl

2010-02-14 Thread Richard Huxton
On 12/02/10 23:10, Tim Bunce wrote: There was some discussion a few weeks ago about inter-stored-procedure calling from PL/Perl. I'd greatly appreciate any feedback. Looks great. PostgreSQL::PLPerl::Call - Simple interface for calling SQL functions from PostgreSQL PL/Perl I don't think

Re: [HACKERS] Re: [COMMITTERS] pgsql: Reduce the chatter to the log when starting a standby server.

2010-02-14 Thread Heikki Linnakangas
Fujii Masao wrote: On Sat, Feb 13, 2010 at 4:33 AM, Richard Huxton d...@archonet.com wrote: 2. Why have a finish.replication file, rather than SELECT pg_finish_replication()? I thought that the trigger file method was more easy-to-use for the existing users since it had been used in