Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-25 Thread Martin Pihlak
On 07/24/2011 11:33 PM, Tom Lane wrote: I've applied the simplified fix (just set SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER) as well as a patch to improve the error reporting situation. Cool that this turned out to be a one-line fix. Thanks! regards, Martin -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Environment checks prior to regression tests?

2011-07-25 Thread Kohei KaiGai
The attached patch enables to check prerequisites to run regression test of sepgsql module. It adds a dependency to installcheck that allows us to launch a script to check environment of operating system. I'd like to add this patch next commit-fest. E.g, this example does not turn on

Re: Initial Review: JSON contrib modul was: Re: [HACKERS] Another swing at JSON

2011-07-25 Thread Florian Pflug
On Jul25, 2011, at 07:35 , Joey Adams wrote: On Mon, Jul 25, 2011 at 1:05 AM, Joey Adams joeyadams3.14...@gmail.com wrote: Should we mimic IEEE floats and preserve -0 versus +0 while treating them as equal? Or should we treat JSON floats like numeric and convert -0 to 0 on input? Or should

Re: Initial Review: JSON contrib modul was: Re: [HACKERS] Another swing at JSON

2011-07-25 Thread Florian Pflug
On Jul25, 2011, at 02:03 , Florian Pflug wrote: On Jul25, 2011, at 00:48 , Joey Adams wrote: Should we follow the JavaScript standard for rendering numbers (which my suggestion approximates)? Or should we use the shortest encoding as Florian suggests? In the light of the above, consider my

Re: [HACKERS] Policy on pulling in code from other projects?

2011-07-25 Thread Robert Haas
On Sun, Jul 24, 2011 at 10:12 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Dave Page's message of sáb jul 23 02:25:30 -0400 2011: Also consider if the library is widely available on common distros or not. If not, packagers are going to have to start packaging that first,

Re: [HACKERS] Policy on pulling in code from other projects?

2011-07-25 Thread Dave Page
On Mon, Jul 25, 2011 at 3:12 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Dave Page's message of sáb jul 23 02:25:30 -0400 2011: Also consider if the library is widely available on common distros or not. If not, packagers are going to have to start packaging that first,

[HACKERS] Access to current database from C-language function

2011-07-25 Thread Achim Domma
Hi, I have read http://www.postgresql.org/docs/9.1/static/xfunc-c.html and my idea is, to write a C function which returns a set of rows. To generate the result set, I would like to access indexes directly using the information I found at http://www.postgresql.org/docs/9.1/static/indexam.html.

Re: [HACKERS] Access to current database from C-language function

2011-07-25 Thread Florian Pflug
On Jul25, 2011, at 13:40 , Achim Domma wrote: I have read http://www.postgresql.org/docs/9.1/static/xfunc-c.html and my idea is, to write a C function which returns a set of rows. To generate the result set, I would like to access indexes directly using the information I found at

Re: [HACKERS] pgbench --unlogged-tables

2011-07-25 Thread Robert Haas
On Fri, Jul 22, 2011 at 5:15 PM, Greg Smith g...@2ndquadrant.com wrote: That looks straightforward enough. OK, committed. The other thing I keep realizing would be useful recently is to allow specifying a different tablespace to switch to when creating all of the indexes.  The old data here,

Re: [HACKERS] vacuumlo patch

2011-07-25 Thread Tim
Updated the patch to also apply when the no-action flag is enabled. git diff HEAD -- contrib/vacuumlo/vacuumlo.c diff --git a/contrib/vacuumlo/vacuumlo.c b/contrib/vacuumlo/vacuumlo.c index f6e2a28..8e9c342 100644 --- a/contrib/vacuumlo/vacuumlo.c +++ b/contrib/vacuumlo/vacuumlo.c @@ -48,6 +48,7

Re: [HACKERS] Questions and experiences writing a Foreign Data Wrapper

2011-07-25 Thread Robert Haas
On Fri, Jul 22, 2011 at 8:09 AM, Albe Laurenz laurenz.a...@wien.gv.at wrote: Heikki Linnakangas wrote: I guess I misunderstood the concept of user mapping. I guess it is time to read my SQL Standard, but some clarification in the documentation sure wouldn't hurt. Agreed, there doesn't seem

Re: [HACKERS] python cleanup

2011-07-25 Thread Andrew Dunstan
On 07/24/2011 11:46 PM, Tom Lane wrote: [python headers set _POSIX_C_SOURCE and _XOPEN_SOURCE] What in the world are the python headers doing fooling with these macros, anyway?? Good question. It seems unfriendly. It looks like you're just about guaranteed to get a warning if you include

Re: [HACKERS] python cleanup

2011-07-25 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 07/24/2011 11:46 PM, Tom Lane wrote: [python headers set _POSIX_C_SOURCE and _XOPEN_SOURCE] What in the world are the python headers doing fooling with these macros, anyway?? The reason we get warnings about these and not about many other things

Re: [HACKERS] python cleanup

2011-07-25 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: [python headers set _POSIX_C_SOURCE and _XOPEN_SOURCE] BTW ... so far as I can find, there is no attempt anywhere in the Postgres sources to set either of these macros. And my understanding of their purpose is that *system* headers should not be

Re: [HACKERS] python cleanup

2011-07-25 Thread Andrew Dunstan
On 07/25/2011 10:36 AM, Tom Lane wrote: Andrew Dunstanand...@dunslane.net writes: [python headers set _POSIX_C_SOURCE and _XOPEN_SOURCE] BTW ... so far as I can find, there is no attempt anywhere in the Postgres sources to set either of these macros. And my understanding of their purpose

Re: [HACKERS] Environment checks prior to regression tests?

2011-07-25 Thread Robert Haas
On Mon, Jul 25, 2011 at 4:36 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: The attached patch enables to check prerequisites to run regression test of sepgsql module. It adds a dependency to installcheck that allows us to launch a script to check environment of operating system. Committed. --

Re: [HACKERS] python cleanup

2011-07-25 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 07/25/2011 10:36 AM, Tom Lane wrote: Andrew Dunstanand...@dunslane.net writes: [python headers set _POSIX_C_SOURCE and _XOPEN_SOURCE] BTW ... so far as I can find, there is no attempt anywhere in the Postgres sources to set either of these

Re: [HACKERS] python cleanup

2011-07-25 Thread Andrew Dunstan
On 07/25/2011 10:52 AM, Tom Lane wrote: Andrew Dunstanand...@dunslane.net writes: On 07/25/2011 10:36 AM, Tom Lane wrote: Andrew Dunstanand...@dunslane.net writes: [python headers set _POSIX_C_SOURCE and _XOPEN_SOURCE] BTW ... so far as I can find, there is no attempt anywhere in the

Re: [HACKERS] Tracing in Postgres

2011-07-25 Thread Robert Haas
On Mon, Jul 25, 2011 at 12:59 AM, Harshitha S hershe...@gmail.com wrote: I want to retain all the error messages, error report that is used by Postgres. I don't intend to log any information extra other than what is provided by Postgres. But I just want to replace the implementation of the

Re: [HACKERS] WIP fix proposal for bug #6123

2011-07-25 Thread Robert Haas
On Fri, Jul 22, 2011 at 5:01 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Your scenario is a BEFORE DELETE trigger that does an UPDATE on the same row, but I think this problem also occurs if you have a BEFORE UPDATE trigger that does an UPDATE on the same row.  I believe the second

Re: [HACKERS] python cleanup

2011-07-25 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 07/25/2011 10:52 AM, Tom Lane wrote: What is features.h, and have its authors read the POSIX standard? AFAICS they have no business defining this symbol. [andrew@emma ~]$ rpm -q -f /usr/include/features.h glibc-headers-2.13-1.x86_64 Oh,

Re: [HACKERS] WIP fix proposal for bug #6123

2011-07-25 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: Well, it seems to me that if the trigger update and the main update were executed as separate commands (with no triggers involved) it would often be the case that they'd dovetail nicely. When this has come up for me, it's usually been the case that

Re: [HACKERS] psql: display of object comments

2011-07-25 Thread Robert Haas
On Fri, Jul 22, 2011 at 10:44 PM, Josh Kupershmidt schmi...@gmail.com wrote: On Sat, Jul 9, 2011 at 1:16 PM, Josh Kupershmidt schmi...@gmail.com wrote: Attached is an updated version of this patch, lifted out of the recent pg_comments patch. With this v2 patch, \dd should properly show just

Re: [HACKERS] Another issue with invalid XML values

2011-07-25 Thread Bernd Helmle
--On 20. Juli 2011 13:06:17 -0400 Tom Lane t...@sss.pgh.pa.us wrote: I've committed this patch with the discussed changes and some other editorialization. I have to leave for an appointment and can't write anything now about the changes, but feel free to ask questions if you have any. Hmm,

Re: [HACKERS] WIP fix proposal for bug #6123

2011-07-25 Thread Robert Haas
On Mon, Jul 25, 2011 at 12:26 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: There's no doubt that it would be better the way you're suggesting; but it looks to me like about five times as many lines of code, harder to be sure it's right, and probably forcing me to learn a few new

Re: [HACKERS] Another issue with invalid XML values

2011-07-25 Thread Florian Pflug
On Jul25, 2011, at 18:53 , Bernd Helmle wrote: --On 20. Juli 2011 13:06:17 -0400 Tom Lane t...@sss.pgh.pa.us wrote: I've committed this patch with the discussed changes and some other editorialization. I have to leave for an appointment and can't write anything now about the changes, but feel

Re: [HACKERS] WIP fix proposal for bug #6123

2011-07-25 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: On Mon, Jul 25, 2011 at 12:26 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: There's no doubt that it would be better the way you're suggesting; but it looks to me like about five times as many lines of code, harder to be sure it's right, and

Re: [HACKERS] Another issue with invalid XML values

2011-07-25 Thread Bernd Helmle
--On 25. Juli 2011 19:07:50 +0200 Florian Pflug f...@phlo.org wrote: Hm, I have libxml2 2.7.8, installed via Mac Ports, and I cannot reproduce this. Maybe Mac Ports uses a modified libxml2, though. I'll check that. Where did you obtain libxml2 from? This is MacPorts, too: % port installed

Re: [HACKERS] Problem with pg_upgrade's directory write check on Windows

2011-07-25 Thread Robert Haas
On Sun, Jul 24, 2011 at 5:27 PM, Bruce Momjian br...@momjian.us wrote: Alvaro Herrera wrote: Excerpts from Bruce Momjian's message of dom jul 24 01:46:08 -0400 2011: Robert Haas wrote: Should I fix this in pg_upgrade 9.1 for Windows or just in 9.2? ?The check works fine on

Re: [HACKERS] Another issue with invalid XML values

2011-07-25 Thread Florian Pflug
On Jul25, 2011, at 19:37 , Bernd Helmle wrote: --On 25. Juli 2011 19:07:50 +0200 Florian Pflug f...@phlo.org wrote: Hm, I have libxml2 2.7.8, installed via Mac Ports, and I cannot reproduce this. Maybe Mac Ports uses a modified libxml2, though. I'll check that. Where did you obtain libxml2

Re: [HACKERS] Deferred partial/expression unique constraints

2011-07-25 Thread Jeff Davis
On Fri, 2011-07-22 at 23:35 +0300, Peter Eisentraut wrote: On ons, 2011-07-13 at 11:26 -0400, Tom Lane wrote: Our standard reason for not implementing UNIQUE constraints on expressions has been that then you would have a thing that claims to be a UNIQUE constraint but isn't representable in

Re: [HACKERS] Another issue with invalid XML values

2011-07-25 Thread Bernd Helmle
--On 25. Juli 2011 19:57:40 +0200 Florian Pflug f...@phlo.org wrote: I got a theory. We do distinguish between libxml2 versions for which the structured and the generic error context handler share the error context (older ones), and those with don't (newer ones). Our configure scripts checks

Re: [HACKERS] WIP: Fast GiST index build

2011-07-25 Thread Heikki Linnakangas
On 22.07.2011 12:38, Alexander Korotkov wrote: Patch with my try to detect ordered datasets is attached. The implemented idea is desribed below. Index tuples are divided by chunks of 128. On each chunk we measure how much leaf pages where index tuples was inserted don't match those of previous

Re: [HACKERS] proposal: a validator for configuration files

2011-07-25 Thread Alexey Klyukin
On Jul 16, 2011, at 9:55 PM, Tom Lane wrote: I wrote: I think that it might be sensible to have the following behavior: 1. Parse the file, where parse means collect all the name = value pairs. Bail out if we find any syntax errors at that level of detail. (With this patch, we could

Re: [HACKERS] problem with compiling beta3 on mingw32+WinXP

2011-07-25 Thread pasman pasmański
After googling i found that mingw's gcc works with 64 bit integers. But printf is incompatible :( . Possible workaround: include inttypes.h , define macros and convert printf strings: printf(% LL,(long long)100) 2011/7/25, pasman pasmański pasma...@gmail.com: Hi. When i try to compile

Re: [HACKERS] Another issue with invalid XML values

2011-07-25 Thread Florian Pflug
On Jul25, 2011, at 20:37 , Bernd Helmle wrote: Ah, but i got now what's wrong here: configure is confusing both libxml2 installations, and a quick look into config.log proves that: it uses the xml2-config from the OSX libs (my $PATH has /usr in front of the bindir of MacPorts, though i seem to

Re: [HACKERS] problem with compiling beta3 on mingw32+WinXP

2011-07-25 Thread Andrew Dunstan
On 07/25/2011 02:56 PM, pasman pasmański wrote: After googling i found that mingw's gcc works with 64 bit integers. But printf is incompatible :( . Possible workaround: include inttypes.h , define macros and convert printf strings: printf(% LL,(long long)100) Postgres builds under

[HACKERS] write scalability

2011-07-25 Thread Robert Haas
I've long harbored a suspicion, based on some testing I did on my home machine, that WALInsertLock is a big performance bottleneck. But I just did some benchmarking that doesn't entirely support that contention. This is on Nate Boley's 32-core machine, with the following settings:

Re: [HACKERS] Access to current database from C-language function

2011-07-25 Thread Achim Domma
Am 25.07.2011 um 14:48 schrieb Florian Pflug: A more low-level API is provided by {heap,index}_{beginscan,endscan}, heap_{insert,update,delete} and index_insert. However, correct handling of transactions using this API isn't easy - for example, to update a row you'd first have to find the

Re: [HACKERS] pgbench --unlogged-tables

2011-07-25 Thread Greg Smith
On 07/25/2011 09:23 AM, Robert Haas wrote: At some point, we also need to sort out the scale factor limit issues, so you can make these things bigger. I had a patch to improve that whole situation, but it hasn't seem to nag at me recently. I forget why it seemed less important, but I

Re: [HACKERS] write scalability

2011-07-25 Thread Merlin Moncure
On Mon, Jul 25, 2011 at 3:07 PM, Robert Haas robertmh...@gmail.com wrote: I've long harbored a suspicion, based on some testing I did on my home machine, that WALInsertLock is a big performance bottleneck.  But I just did some benchmarking that doesn't entirely support that contention.  This

Re: [HACKERS] write scalability

2011-07-25 Thread Alvaro Herrera
Excerpts from Merlin Moncure's message of lun jul 25 17:19:58 -0400 2011: On Mon, Jul 25, 2011 at 3:07 PM, Robert Haas robertmh...@gmail.com wrote:  Experience with the read scalability stuff has taught me also to look at which LWLocks have the most shared acquisitions, as that can cause

Re: [HACKERS] sinval synchronization considered harmful

2011-07-25 Thread Noah Misch
On Fri, Jul 22, 2011 at 03:54:03PM -0400, Robert Haas wrote: On Fri, Jul 22, 2011 at 3:28 PM, Noah Misch n...@2ndquadrant.com wrote: This is attractive, and I don't see any problems with it.  (In theory, you could hit a case where the load of resetState gives an ancient false just as the

Re: [HACKERS] storing TZ along timestamps

2011-07-25 Thread Jim Nasby
On Jul 22, 2011, at 10:33 AM, Robert Haas wrote: On Thu, Jul 21, 2011 at 7:05 PM, Jim Nasby j...@nasby.net wrote: On Jul 21, 2011, at 5:30 PM, Christopher Browne wrote: - I'd commend capturing NOW() in a timestamptz field. That gives you: 1. What time the DB server thought it was, in terms

Re: [HACKERS] libedit memory stomp is apparently fixed in OS X Lion

2011-07-25 Thread Jim Nasby
On Jul 22, 2011, at 11:01 AM, Peter Geoghegan wrote: On 22 July 2011 03:24, Tom Lane t...@sss.pgh.pa.us wrote: I had a bug filed with Apple about that, and today I got some auto-mail indicating they'd fixed that bug as of OS X 10.7 (Lion). I don't have Lion installed here, but I grabbed the

Re: [HACKERS] Access to current database from C-language function

2011-07-25 Thread Florian Pflug
On Jul25, 2011, at 22:31 , Achim Domma wrote: Am 25.07.2011 um 14:48 schrieb Florian Pflug: A more low-level API is provided by {heap,index}_{beginscan,endscan}, heap_{insert,update,delete} and index_insert. However, correct handling of transactions using this API isn't easy - for example,

Re: [HACKERS] Update releases

2011-07-25 Thread David Fetter
On Fri, Jul 01, 2011 at 11:59:55PM +0100, Thom Brown wrote: On 1 July 2011 23:57, David Fetter da...@fetter.org wrote: Folks, Now that there's a (very minor) crypto fix and a new DST ruleset, when can we get the next set of minor revs out the door? Do we know how many identified bugs

[HACKERS] Error calling PG_RETURN_NULL()

2011-07-25 Thread Alexandre Savaris
Hi! I'm working on an implementation for a new data type (PostgreSQL version 9.1 beta 3 on Windows 7 32 bits), according to the following rules: - 1. NULL values are stored as is; - 2. character strings (up to 16 bytes) are stored without leading or trailing spaces; - 3. empty character

Re: [HACKERS] pgbench --unlogged-tables

2011-07-25 Thread David Fetter
On Fri, Jul 22, 2011 at 10:15:08PM -0400, Greg Smith wrote: On 07/22/2011 08:15 PM, David Fetter wrote: Do you have any theories as to how indexing on SSD speeds things up? IIRC you found only marginal benefit in putting WALs there. Are there cases that SSD helps more than others when it

Re: [HACKERS] storing TZ along timestamps

2011-07-25 Thread Robert Haas
On Mon, Jul 25, 2011 at 6:26 PM, Jim Nasby j...@nasby.net wrote: Hrm, don't we only pull in ZIC info on a reload? Or do we actually refer to it dynamically all the time? Perhaps we can enforce that we'll only recognize new TZ info as part of a config reload? Hmm. That might work in theory,

Re: [HACKERS] vacuumlo patch

2011-07-25 Thread Robert Haas
On Mon, Jul 25, 2011 at 9:37 AM, Tim elatl...@gmail.com wrote: Updated the patch to also apply when the no-action flag is enabled. You may want to read this: http://wiki.postgresql.org/wiki/Submitting_a_Patch And add your patch here:

Re: [HACKERS] Another issue with invalid XML values

2011-07-25 Thread Noah Misch
On Mon, Jul 25, 2011 at 09:06:41PM +0200, Florian Pflug wrote: On Jul25, 2011, at 20:37 , Bernd Helmle wrote: Ah, but i got now what's wrong here: configure is confusing both libxml2 installations, and a quick look into config.log proves that: it uses the xml2-config from the OSX libs (my

Re: [HACKERS] pgbench--new transaction type

2011-07-25 Thread Jeff Janes
On Sun, Jun 19, 2011 at 9:34 PM, Itagaki Takahiro itagaki.takah...@gmail.com wrote: On Mon, Jun 20, 2011 at 07:30, Greg Smith g...@2ndquadrant.com wrote: I applied Jeff's patch but changed this to address concerns about the program getting stuck running for too long in the function: #define

Re: [HACKERS] Problem with pg_upgrade's directory write check on Windows

2011-07-25 Thread Bruce Momjian
Robert Haas wrote: On Sun, Jul 24, 2011 at 5:27 PM, Bruce Momjian br...@momjian.us wrote: Alvaro Herrera wrote: Excerpts from Bruce Momjian's message of dom jul 24 01:46:08 -0400 2011: Robert Haas wrote: Should I fix this in pg_upgrade 9.1 for Windows or just in 9.2? ?The check

Re: [HACKERS] pgbench--new transaction type

2011-07-25 Thread Greg Smith
On 07/25/2011 08:12 PM, Jeff Janes wrote: In the absence of -s and presence of -f, :scale gets set to 1, rather than to select count(*) from pgbench_branches. I don't think it is nice to rely on people to correctly specify -s. I would like to change -f so that in the absence of -s it uses the

Re: [HACKERS] write scalability

2011-07-25 Thread Robert Haas
On Mon, Jul 25, 2011 at 4:07 PM, Robert Haas robertmh...@gmail.com wrote: As to what that something might be, I reran this last test with LWLOCK_STATS enabled and here are the top things that are blocking: lwlock 310: shacq 96846 exacq 108433 blk 16275 lwlock 3: shacq 64 exacq 2628381 blk

Re: [HACKERS] write scalability

2011-07-25 Thread Greg Smith
On 07/25/2011 04:07 PM, Robert Haas wrote: I did 5-minute pgbench runs with unlogged tables and with permanent tables, restarting the database server and reinitializing the tables between each run. Database scale? One or multiple pgbench worker threads? A reminder on the amount of RAM in

Re: [HACKERS] psql: bogus descriptions displayed by \d+

2011-07-25 Thread Josh Kupershmidt
On Fri, Jul 22, 2011 at 12:44 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jul 21, 2011 at 9:17 PM, Josh Kupershmidt schmi...@gmail.com wrote: Here's a small patch against branch 8.4 to mention support for COMMENT ON index_name.column_name. I am not in favor of this - because we'd

[HACKERS] Check constraints on partition parents only?

2011-07-25 Thread Jerry Sievers
Hackers; I just noticed that somewhere between 8.2 and 8.4, an exception is raised trying to alter table ONLY some_partition_parent ADD CHECK (foo). I can understand why it makes sense to handle this as an error. Howeverin practice on a few systems that I used to manage this would be a problem.

Re: [HACKERS] Check constraints on partition parents only?

2011-07-25 Thread Andrew Dunstan
On 07/25/2011 10:31 PM, Jerry Sievers wrote: Hackers; I just noticed that somewhere between 8.2 and 8.4, an exception is raised trying to alter table ONLY some_partition_parent ADD CHECK (foo). I can understand why it makes sense to handle this as an error. Howeverin practice on a few

Re: [HACKERS] Error calling PG_RETURN_NULL()

2011-07-25 Thread Tom Lane
Alexandre Savaris alexandre.sava...@gmail.com writes: Hi! I'm working on an implementation for a new data type (PostgreSQL version 9.1 beta 3 on Windows 7 32 bits), according to the following rules: - 1. NULL values are stored as is; - 2. character strings (up to 16 bytes) are stored without

Re: [HACKERS] Check constraints on partition parents only?

2011-07-25 Thread Alvaro Herrera
Excerpts from Andrew Dunstan's message of lun jul 25 22:44:32 -0400 2011: On 07/25/2011 10:31 PM, Jerry Sievers wrote: Hackers; I just noticed that somewhere between 8.2 and 8.4, an exception is raised trying to alter table ONLY some_partition_parent ADD CHECK (foo). 8.4 had this