Re: [HACKERS] libpq SSL with non-blocking sockets

2011-07-25 Thread Martin Pihlak
On 07/24/2011 11:33 PM, Tom Lane wrote: > I've applied the simplified fix (just set SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER) > as well as a patch to improve the error reporting situation. > Cool that this turned out to be a one-line fix. Thanks! regards, Martin -- Sent via pgsql-hackers mailing li

Re: [HACKERS] Environment checks prior to regression tests?

2011-07-25 Thread Kohei KaiGai
The attached patch enables to check prerequisites to run regression test of sepgsql module. It adds a dependency to installcheck that allows us to launch a script to check environment of operating system. I'd like to add this patch next commit-fest. E.g, this example does not turn on sepgsql_regre

Re: Initial Review: JSON contrib modul was: Re: [HACKERS] Another swing at JSON

2011-07-25 Thread Florian Pflug
On Jul25, 2011, at 07:35 , Joey Adams wrote: > On Mon, Jul 25, 2011 at 1:05 AM, Joey Adams > wrote: >> Should we mimic IEEE floats and preserve -0 versus +0 while treating >> them as equal? Or should we treat JSON floats like numeric and >> convert -0 to 0 on input? Or should we do something el

Re: Initial Review: JSON contrib modul was: Re: [HACKERS] Another swing at JSON

2011-07-25 Thread Florian Pflug
On Jul25, 2011, at 02:03 , Florian Pflug wrote: > On Jul25, 2011, at 00:48 , Joey Adams wrote: >> Should we follow the JavaScript standard for rendering numbers (which >> my suggestion approximates)? Or should we use the shortest encoding >> as Florian suggests? > > In the light of the above, con

Re: [HACKERS] Policy on pulling in code from other projects?

2011-07-25 Thread Robert Haas
On Sun, Jul 24, 2011 at 10:12 PM, Alvaro Herrera wrote: > Excerpts from Dave Page's message of sáb jul 23 02:25:30 -0400 2011: > >> Also consider if the library is widely available on common distros or >> not. If not, packagers are going to have to start packaging that >> first, in order to build

Re: [HACKERS] Policy on pulling in code from other projects?

2011-07-25 Thread Dave Page
On Mon, Jul 25, 2011 at 3:12 AM, Alvaro Herrera wrote: > Excerpts from Dave Page's message of sáb jul 23 02:25:30 -0400 2011: > >> Also consider if the library is widely available on common distros or >> not. If not, packagers are going to have to start packaging that >> first, in order to build t

[HACKERS] Access to current database from C-language function

2011-07-25 Thread Achim Domma
Hi, I have read http://www.postgresql.org/docs/9.1/static/xfunc-c.html and my idea is, to write a C function which returns a set of rows. To generate the result set, I would like to access indexes directly using the information I found at http://www.postgresql.org/docs/9.1/static/indexam.html.

Re: [HACKERS] Access to current database from C-language function

2011-07-25 Thread Florian Pflug
On Jul25, 2011, at 13:40 , Achim Domma wrote: > I have read http://www.postgresql.org/docs/9.1/static/xfunc-c.html and my > idea is, to write a C function which returns a set of rows. To generate the > result set, I would like to access indexes directly using the information I > found at http://

Re: [HACKERS] pgbench --unlogged-tables

2011-07-25 Thread Robert Haas
On Fri, Jul 22, 2011 at 5:15 PM, Greg Smith wrote: > That looks straightforward enough. OK, committed. > The other thing I keep realizing would > be useful recently is to allow specifying a different tablespace to switch > to when creating all of the indexes.  The old "data here, indexes on fast

Re: [HACKERS] vacuumlo patch

2011-07-25 Thread Tim
Updated the patch to also apply when the no-action flag is enabled. git diff HEAD -- contrib/vacuumlo/vacuumlo.c diff --git a/contrib/vacuumlo/vacuumlo.c b/contrib/vacuumlo/vacuumlo.c index f6e2a28..8e9c342 100644 --- a/contrib/vacuumlo/vacuumlo.c +++ b/contrib/vacuumlo/vacuumlo.c @@ -48,6 +48,7 @

Re: [HACKERS] Questions and experiences writing a Foreign Data Wrapper

2011-07-25 Thread Robert Haas
On Fri, Jul 22, 2011 at 8:09 AM, Albe Laurenz wrote: > Heikki Linnakangas wrote: >>> I guess I misunderstood the concept of user mapping. > >>> I guess it is time to read my SQL Standard, but some clarification >>> in the documentation sure wouldn't hurt. >> >> Agreed, there doesn't seem to be any

Re: [HACKERS] python cleanup

2011-07-25 Thread Andrew Dunstan
On 07/24/2011 11:46 PM, Tom Lane wrote: [python headers set _POSIX_C_SOURCE and _XOPEN_SOURCE] What in the world are the python headers doing fooling with these macros, anyway?? Good question. It seems unfriendly. It looks like you're just about guaranteed to get a warning if you include a

Re: [HACKERS] python cleanup

2011-07-25 Thread Tom Lane
Andrew Dunstan writes: > On 07/24/2011 11:46 PM, Tom Lane wrote: >>> [python headers set _POSIX_C_SOURCE and _XOPEN_SOURCE] >> What in the world are the python headers doing fooling with these >> macros, anyway?? > The reason we get warnings about these and not about many other things it > defi

Re: [HACKERS] python cleanup

2011-07-25 Thread Tom Lane
Andrew Dunstan writes: > [python headers set _POSIX_C_SOURCE and _XOPEN_SOURCE] BTW ... so far as I can find, there is no attempt anywhere in the Postgres sources to set either of these macros. And my understanding of their purpose is that *system* headers should not be setting them at all, rath

Re: [HACKERS] python cleanup

2011-07-25 Thread Andrew Dunstan
On 07/25/2011 10:36 AM, Tom Lane wrote: Andrew Dunstan writes: [python headers set _POSIX_C_SOURCE and _XOPEN_SOURCE] BTW ... so far as I can find, there is no attempt anywhere in the Postgres sources to set either of these macros. And my understanding of their purpose is that *system* head

Re: [HACKERS] Environment checks prior to regression tests?

2011-07-25 Thread Robert Haas
On Mon, Jul 25, 2011 at 4:36 AM, Kohei KaiGai wrote: > The attached patch enables to check prerequisites to run regression > test of sepgsql module. > It adds a dependency to installcheck that allows us to launch > a script to check environment of operating system. Committed. -- Robert Haas Ent

Re: [HACKERS] python cleanup

2011-07-25 Thread Tom Lane
Andrew Dunstan writes: > On 07/25/2011 10:36 AM, Tom Lane wrote: >> Andrew Dunstan writes: >>> [python headers set _POSIX_C_SOURCE and _XOPEN_SOURCE] >> BTW ... so far as I can find, there is no attempt anywhere in the >> Postgres sources to set either of these macros. And my understanding of >

Re: [HACKERS] python cleanup

2011-07-25 Thread Andrew Dunstan
On 07/25/2011 10:52 AM, Tom Lane wrote: Andrew Dunstan writes: On 07/25/2011 10:36 AM, Tom Lane wrote: Andrew Dunstan writes: [python headers set _POSIX_C_SOURCE and _XOPEN_SOURCE] BTW ... so far as I can find, there is no attempt anywhere in the Postgres sources to set either of these m

Re: [HACKERS] Tracing in Postgres

2011-07-25 Thread Robert Haas
On Mon, Jul 25, 2011 at 12:59 AM, Harshitha S wrote: > I want to retain all the error messages, error report that is used by > Postgres. > I don't intend to log any information extra other than what is provided by > Postgres. > But I just want to replace the implementation of the logging/tracing i

Re: [HACKERS] WIP fix proposal for bug #6123

2011-07-25 Thread Robert Haas
On Fri, Jul 22, 2011 at 5:01 PM, Kevin Grittner wrote: >> Your scenario is a BEFORE DELETE trigger that does an UPDATE on >> the same row, but I think this problem also occurs if you have a >> BEFORE UPDATE trigger that does an UPDATE on the same row.  I >> believe the second update gets silently

Re: [HACKERS] python cleanup

2011-07-25 Thread Tom Lane
Andrew Dunstan writes: > On 07/25/2011 10:52 AM, Tom Lane wrote: >> What is features.h, and have its authors read the POSIX standard? >> AFAICS they have no business defining this symbol. > [andrew@emma ~]$ rpm -q -f /usr/include/features.h > glibc-headers-2.13-1.x86_64 Oh, for some reas

Re: [HACKERS] WIP fix proposal for bug #6123

2011-07-25 Thread Kevin Grittner
Robert Haas wrote: > Well, it seems to me that if the trigger update and the main > update were executed as separate commands (with no triggers > involved) it would often be the case that they'd dovetail nicely. > When this has come up for me, it's usually been the case that the > sets of field

Re: [HACKERS] psql: display of object comments

2011-07-25 Thread Robert Haas
On Fri, Jul 22, 2011 at 10:44 PM, Josh Kupershmidt wrote: > On Sat, Jul 9, 2011 at 1:16 PM, Josh Kupershmidt wrote: > > Attached is an updated version of this patch, lifted out of the recent > pg_comments patch. With this v2 patch, \dd should properly show just > its five object types, and the ps

Re: [HACKERS] Another issue with invalid XML values

2011-07-25 Thread Bernd Helmle
--On 20. Juli 2011 13:06:17 -0400 Tom Lane wrote: I've committed this patch with the discussed changes and some other editorialization. I have to leave for an appointment and can't write anything now about the changes, but feel free to ask questions if you have any. Hmm, when building agai

Re: [HACKERS] WIP fix proposal for bug #6123

2011-07-25 Thread Robert Haas
On Mon, Jul 25, 2011 at 12:26 PM, Kevin Grittner wrote: > There's no doubt that it would be better the way you're suggesting; > but it looks to me like about five times as many lines of code, > harder to be sure it's right, and probably forcing me to learn a few > new subsystems of PostgreSQL inte

Re: [HACKERS] Another issue with invalid XML values

2011-07-25 Thread Florian Pflug
On Jul25, 2011, at 18:53 , Bernd Helmle wrote: > --On 20. Juli 2011 13:06:17 -0400 Tom Lane wrote: >> I've committed this patch with the discussed changes and some other >> editorialization. I have to leave for an appointment and can't write >> anything now about the changes, but feel free to ask

Re: [HACKERS] WIP fix proposal for bug #6123

2011-07-25 Thread Kevin Grittner
Robert Haas wrote: > On Mon, Jul 25, 2011 at 12:26 PM, Kevin Grittner > wrote: >> There's no doubt that it would be better the way you're >> suggesting; but it looks to me like about five times as many >> lines of code, harder to be sure it's right, and probably forcing >> me to learn a few new s

Re: [HACKERS] Another issue with invalid XML values

2011-07-25 Thread Bernd Helmle
--On 25. Juli 2011 19:07:50 +0200 Florian Pflug wrote: Hm, I have libxml2 2.7.8, installed via Mac Ports, and I cannot reproduce this. Maybe Mac Ports uses a modified libxml2, though. I'll check that. Where did you obtain libxml2 from? This is MacPorts, too: % port installed libxml2 The f

Re: [HACKERS] Problem with pg_upgrade's directory write check on Windows

2011-07-25 Thread Robert Haas
On Sun, Jul 24, 2011 at 5:27 PM, Bruce Momjian wrote: > Alvaro Herrera wrote: >> Excerpts from Bruce Momjian's message of dom jul 24 01:46:08 -0400 2011: >> > Robert Haas wrote: >> >> > > > Should I fix this in pg_upgrade 9.1 for Windows or just in 9.2? ?The >> > > > check works fine on non-Window

Re: [HACKERS] Another issue with invalid XML values

2011-07-25 Thread Florian Pflug
On Jul25, 2011, at 19:37 , Bernd Helmle wrote: > --On 25. Juli 2011 19:07:50 +0200 Florian Pflug wrote: >> Hm, I have libxml2 2.7.8, installed via Mac Ports, and I cannot reproduce >> this. Maybe Mac Ports uses a modified libxml2, though. I'll check that. >> >> Where did you obtain libxml2 from?

Re: [HACKERS] Deferred partial/expression unique constraints

2011-07-25 Thread Jeff Davis
On Fri, 2011-07-22 at 23:35 +0300, Peter Eisentraut wrote: > On ons, 2011-07-13 at 11:26 -0400, Tom Lane wrote: > > Our standard reason for not implementing UNIQUE constraints on > > expressions has been that then you would have a thing that claims to be > > a UNIQUE constraint but isn't representa

Re: [HACKERS] Another issue with invalid XML values

2011-07-25 Thread Bernd Helmle
--On 25. Juli 2011 19:57:40 +0200 Florian Pflug wrote: I got a theory. We do distinguish between libxml2 versions for which the structured and the generic error context handler share the error context (older ones), and those with don't (newer ones). Our configure scripts checks for the availa

Re: [HACKERS] WIP: Fast GiST index build

2011-07-25 Thread Heikki Linnakangas
On 22.07.2011 12:38, Alexander Korotkov wrote: Patch with my try to detect ordered datasets is attached. The implemented idea is desribed below. Index tuples are divided by chunks of 128. On each chunk we measure how much leaf pages where index tuples was inserted don't match those of previous ch

Re: [HACKERS] proposal: a validator for configuration files

2011-07-25 Thread Alexey Klyukin
On Jul 16, 2011, at 9:55 PM, Tom Lane wrote: > I wrote: >> I think that it might be sensible to have the following behavior: > >> 1. Parse the file, where "parse" means collect all the name = value >> pairs. Bail out if we find any syntax errors at that level of detail. >> (With this patch, we

Re: [HACKERS] problem with compiling beta3 on mingw32+WinXP

2011-07-25 Thread pasman pasmański
After googling i found that mingw's gcc works with 64 bit integers. But printf is incompatible :( . Possible workaround: include inttypes.h , define macros and convert printf strings: printf("%" LL,(long long)100) 2011/7/25, pasman pasmański : > Hi. > > When i try to compile postgresql-be

Re: [HACKERS] Another issue with invalid XML values

2011-07-25 Thread Florian Pflug
On Jul25, 2011, at 20:37 , Bernd Helmle wrote: > Ah, but i got now what's wrong here: configure is confusing both libxml2 > installations, and a quick look into config.log proves that: it uses the > xml2-config from the OSX libs (my $PATH has /usr in front of the bindir of > MacPorts, though i seem

Re: [HACKERS] problem with compiling beta3 on mingw32+WinXP

2011-07-25 Thread Andrew Dunstan
On 07/25/2011 02:56 PM, pasman pasmański wrote: After googling i found that mingw's gcc works with 64 bit integers. But printf is incompatible :( . Possible workaround: include inttypes.h , define macros and convert printf strings: printf("%" LL,(long long)100) Postgres builds under

[HACKERS] write scalability

2011-07-25 Thread Robert Haas
I've long harbored a suspicion, based on some testing I did on my home machine, that WALInsertLock is a big performance bottleneck. But I just did some benchmarking that doesn't entirely support that contention. This is on Nate Boley's 32-core machine, with the following settings: max_connection

Re: [HACKERS] Access to current database from C-language function

2011-07-25 Thread Achim Domma
Am 25.07.2011 um 14:48 schrieb Florian Pflug: > A more low-level API is provided by {heap,index}_{beginscan,endscan}, > heap_{insert,update,delete} and index_insert. However, correct handling of > transactions using this API isn't easy - for example, to update a row you'd > first have to find t

Re: [HACKERS] pgbench --unlogged-tables

2011-07-25 Thread Greg Smith
On 07/25/2011 09:23 AM, Robert Haas wrote: At some point, we also need to sort out the scale factor limit issues, so you can make these things bigger. I had a patch to improve that whole situation, but it hasn't seem to nag at me recently. I forget why it seemed less important, but I doub

Re: [HACKERS] write scalability

2011-07-25 Thread Merlin Moncure
On Mon, Jul 25, 2011 at 3:07 PM, Robert Haas wrote: > I've long harbored a suspicion, based on some testing I did on my home > machine, that WALInsertLock is a big performance bottleneck.  But I > just did some benchmarking that doesn't entirely support that > contention.  This is on Nate Boley's

Re: [HACKERS] write scalability

2011-07-25 Thread Alvaro Herrera
Excerpts from Merlin Moncure's message of lun jul 25 17:19:58 -0400 2011: > On Mon, Jul 25, 2011 at 3:07 PM, Robert Haas wrote: > > Experience > > with the read scalability stuff has taught me also to look at which > > LWLocks have the most shared acquisitions, as that can cause spinlock > > and

Re: [HACKERS] sinval synchronization considered harmful

2011-07-25 Thread Noah Misch
On Fri, Jul 22, 2011 at 03:54:03PM -0400, Robert Haas wrote: > On Fri, Jul 22, 2011 at 3:28 PM, Noah Misch wrote: > > This is attractive, and I don't see any problems with it.  (In theory, you > > could > > hit a case where the load of resetState gives an ancient "false" just as the > > counters

Re: [HACKERS] storing TZ along timestamps

2011-07-25 Thread Jim Nasby
On Jul 22, 2011, at 10:33 AM, Robert Haas wrote: > On Thu, Jul 21, 2011 at 7:05 PM, Jim Nasby wrote: >> On Jul 21, 2011, at 5:30 PM, Christopher Browne wrote: >>> - I'd commend capturing NOW() in a timestamptz field. That gives you: >>> 1. What time the DB server thought it was, in terms of UT1

Re: [HACKERS] libedit memory stomp is apparently fixed in OS X Lion

2011-07-25 Thread Jim Nasby
On Jul 22, 2011, at 11:01 AM, Peter Geoghegan wrote: > On 22 July 2011 03:24, Tom Lane wrote: >> I had a bug filed with Apple about that, and today I got some auto-mail >> indicating they'd fixed that bug as of OS X 10.7 (Lion). I don't have >> Lion installed here, but I grabbed the libedit sourc

Re: [HACKERS] Access to current database from C-language function

2011-07-25 Thread Florian Pflug
On Jul25, 2011, at 22:31 , Achim Domma wrote: > Am 25.07.2011 um 14:48 schrieb Florian Pflug: >> A more low-level API is provided by {heap,index}_{beginscan,endscan}, >> heap_{insert,update,delete} and index_insert. However, correct handling of >> transactions using this API isn't easy - for exam

Re: [HACKERS] Update releases

2011-07-25 Thread David Fetter
On Fri, Jul 01, 2011 at 11:59:55PM +0100, Thom Brown wrote: > On 1 July 2011 23:57, David Fetter wrote: > > Folks, > > > > Now that there's a (very minor) crypto fix and a new DST ruleset, when > > can we get the next set of minor revs out the door? > > Do we know how many identified bugs are sti

[HACKERS] Error calling PG_RETURN_NULL()

2011-07-25 Thread Alexandre Savaris
Hi! I'm working on an implementation for a new data type (PostgreSQL version 9.1 beta 3 on Windows 7 32 bits), according to the following rules: - 1. NULL values are stored as is; - 2. character strings (up to 16 bytes) are stored without leading or trailing spaces; - 3. empty character strings

Re: [HACKERS] pgbench --unlogged-tables

2011-07-25 Thread David Fetter
On Fri, Jul 22, 2011 at 10:15:08PM -0400, Greg Smith wrote: > On 07/22/2011 08:15 PM, David Fetter wrote: > >Do you have any theories as to how indexing on SSD speeds things > >up? IIRC you found only marginal benefit in putting WALs there. > >Are there cases that SSD helps more than others when i

Re: [HACKERS] storing TZ along timestamps

2011-07-25 Thread Robert Haas
On Mon, Jul 25, 2011 at 6:26 PM, Jim Nasby wrote: > Hrm, don't we only pull in ZIC info on a reload? Or do we actually refer to > it dynamically all the time? Perhaps we can enforce that we'll only recognize > new TZ info as part of a config reload? Hmm. That might work in theory, but I don't

Re: [HACKERS] vacuumlo patch

2011-07-25 Thread Robert Haas
On Mon, Jul 25, 2011 at 9:37 AM, Tim wrote: > Updated the patch to also apply when the no-action flag is enabled. You may want to read this: http://wiki.postgresql.org/wiki/Submitting_a_Patch And add your patch here: https://commitfest.postgresql.org/action/commitfest_view/open -- Robert Haa

Re: [HACKERS] Another issue with invalid XML values

2011-07-25 Thread Noah Misch
On Mon, Jul 25, 2011 at 09:06:41PM +0200, Florian Pflug wrote: > On Jul25, 2011, at 20:37 , Bernd Helmle wrote: > > Ah, but i got now what's wrong here: configure is confusing both libxml2 > > installations, and a quick look into config.log proves that: it uses the > > xml2-config from the OSX libs

Re: [HACKERS] pgbench--new transaction type

2011-07-25 Thread Jeff Janes
On Sun, Jun 19, 2011 at 9:34 PM, Itagaki Takahiro wrote: > On Mon, Jun 20, 2011 at 07:30, Greg Smith wrote: >> I applied Jeff's patch but changed this to address concerns about the >> program getting stuck running for too long in the function: >> >> #define plpgsql_loops   512 > > Is it OK to def

Re: [HACKERS] Problem with pg_upgrade's directory write check on Windows

2011-07-25 Thread Bruce Momjian
Robert Haas wrote: > On Sun, Jul 24, 2011 at 5:27 PM, Bruce Momjian wrote: > > Alvaro Herrera wrote: > >> Excerpts from Bruce Momjian's message of dom jul 24 01:46:08 -0400 2011: > >> > Robert Haas wrote: > >> > >> > > > Should I fix this in pg_upgrade 9.1 for Windows or just in 9.2? ?The > >> > >

Re: [HACKERS] pgbench--new transaction type

2011-07-25 Thread Greg Smith
On 07/25/2011 08:12 PM, Jeff Janes wrote: In the absence of -s and presence of -f, :scale gets set to 1, rather than to "select count(*) from pgbench_branches". I don't think it is nice to rely on people to correctly specify -s. I would like to change -f so that in the absence of -s it uses the

Re: [HACKERS] write scalability

2011-07-25 Thread Robert Haas
On Mon, Jul 25, 2011 at 4:07 PM, Robert Haas wrote: > As to what that something might be, I reran this last test with > LWLOCK_STATS enabled and here are the top things that are blocking: > > lwlock 310: shacq 96846 exacq 108433 blk 16275 > lwlock 3: shacq 64 exacq 2628381 blk 36027 > lwlock 7: sh

Re: [HACKERS] write scalability

2011-07-25 Thread Greg Smith
On 07/25/2011 04:07 PM, Robert Haas wrote: I did 5-minute pgbench runs with unlogged tables and with permanent tables, restarting the database server and reinitializing the tables between each run. Database scale? One or multiple pgbench worker threads? A reminder on the amount of RAM in the

Re: [HACKERS] psql: bogus descriptions displayed by \d+

2011-07-25 Thread Josh Kupershmidt
On Fri, Jul 22, 2011 at 12:44 PM, Robert Haas wrote: > On Thu, Jul 21, 2011 at 9:17 PM, Josh Kupershmidt wrote: >> Here's a small patch against branch 8.4 to mention support for COMMENT >> ON index_name.column_name. > > I am not in favor of this - because we'd also need to mention every > other r

[HACKERS] Check constraints on partition parents only?

2011-07-25 Thread Jerry Sievers
Hackers; I just noticed that somewhere between 8.2 and 8.4, an exception is raised trying to alter table ONLY some_partition_parent ADD CHECK (foo). I can understand why it makes sense to handle this as an error. Howeverin practice on a few systems that I used to manage this would be a problem.

Re: [HACKERS] Check constraints on partition parents only?

2011-07-25 Thread Andrew Dunstan
On 07/25/2011 10:31 PM, Jerry Sievers wrote: Hackers; I just noticed that somewhere between 8.2 and 8.4, an exception is raised trying to alter table ONLY some_partition_parent ADD CHECK (foo). I can understand why it makes sense to handle this as an error. Howeverin practice on a few system

Re: [HACKERS] Error calling PG_RETURN_NULL()

2011-07-25 Thread Tom Lane
Alexandre Savaris writes: > Hi! I'm working on an implementation for a new data type (PostgreSQL > version 9.1 beta 3 on Windows 7 32 bits), according to the following rules: > - 1. NULL values are stored as is; > - 2. character strings (up to 16 bytes) are stored without leading or > trailing s

Re: [HACKERS] Check constraints on partition parents only?

2011-07-25 Thread Alvaro Herrera
Excerpts from Andrew Dunstan's message of lun jul 25 22:44:32 -0400 2011: > > On 07/25/2011 10:31 PM, Jerry Sievers wrote: > > Hackers; > > > > I just noticed that somewhere between 8.2 and 8.4, an exception is > > raised trying to alter table ONLY some_partition_parent ADD CHECK > > (foo). > 8.4