Re: [HACKERS] basic pgbench runs with various performance-related patches

2012-02-05 Thread Robert Haas
On Sat, Feb 4, 2012 at 11:59 AM, Greg Smith g...@2ndquadrant.com wrote: On 01/24/2012 08:58 AM, Robert Haas wrote: One somewhat odd thing about these numbers is that, on permanent tables, all of the patches seemed to show regressions vs. master in single-client throughput.  That's a slightly

Re: [HACKERS] initdb and fsync

2012-02-05 Thread Jeff Davis
On Sat, 2012-02-04 at 20:18 -0500, Noah Misch wrote: If we add fsync calls to the initdb process, they should cover the entire data directory tree. This patch syncs files that initdb.c writes, but we ought to also sync files that bootstrap-mode backends had written. It doesn't make sense for

[HACKERS] pl/python long-lived allocations in datum-dict transformation

2012-02-05 Thread Jan Urbański
Consider this: create table arrays as select array[random(), random(), random(), random(), random(), random()] as a from generate_series(1, 100); create or replace function plpython_outputfunc() returns void as $$ c = plpy.cursor('select a from arrays') for row in c: pass $$ language

[HACKERS] plpgsql leaking memory when stringifying datums

2012-02-05 Thread Jan Urbański
While chasing a PL/Python memory leak, I did a few tests with PL/pgSQL and I think there are places where memory is not freed sufficiently early. Attached are two functions that when run will make the backend's memory consumption increase until they finish. With both, the cause is

[HACKERS] Report: race conditions in WAL replay routines

2012-02-05 Thread Tom Lane
Pursuant to the recent discussion about bugs 6200 and 6245, I went trawling through all the WAL redo routines looking for bugs such as inadequate locking or transiently trashing shared buffers. Here's what I found: * As we already knew, RestoreBkpBlocks is broken because it transiently trashes a

Re: [HACKERS] JSON output functions.

2012-02-05 Thread Stefan Keller
Hi Andrew Nice work! Just for completeness: Did you also think of including geometry types in JSON output functions in later releases? There's a nice extension of JSON called GeoJSON for a starting point. Yours, Stefan 2012/2/3 Andrew Dunstan and...@dunslane.net: On 02/02/2012 12:20 PM,

Re: [HACKERS] initdb and fsync

2012-02-05 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: On Sat, 2012-02-04 at 20:18 -0500, Noah Misch wrote: If we add fsync calls to the initdb process, they should cover the entire data directory tree. This patch syncs files that initdb.c writes, but we ought to also sync files that bootstrap-mode backends

Re: [HACKERS] JSON output functions.

2012-02-05 Thread Andrew Dunstan
On 02/05/2012 02:31 PM, Stefan Keller wrote: Hi Andrew Nice work! Just for completeness: Did you also think of including geometry types in JSON output functions in later releases? There's a nice extension of JSON called GeoJSON for a starting point. [side note: please don't top-reply on

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-05 Thread Kohei KaiGai
2012年2月1日12:15 Shigeru Hanada shigeru.han...@gmail.com: (2012/01/30 4:39), Kohei KaiGai wrote: I checked the fdw_helper_funcs_v3.patch, pgsql_fdw_v5.patch and pgsql_fdw_pushdown_v1.patch. My comments are below. Thanks for the review! [BUG] Even though pgsql_fdw tries to push-down

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-05 Thread Simon Riggs
On Sun, Feb 5, 2012 at 3:59 AM, Bruce Momjian br...@momjian.us wrote: On Sat, Dec 24, 2011 at 03:56:58PM +, Simon Riggs wrote: Also, as far as I can see this patch usurps the page version field, which I find unacceptably short-sighted.  Do you really think this is the last page layout

Re: [HACKERS] [BUGS] BUG #6425: Bus error in slot_deform_tuple

2012-02-05 Thread Simon Riggs
On Sat, Feb 4, 2012 at 6:49 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: The cause here is data changing underneath the user. Your patch solves the most obvious error, but it still allows other problems if applying the backup block changes data. If the

Re: [HACKERS] Report: race conditions in WAL replay routines

2012-02-05 Thread Simon Riggs
On Sun, Feb 5, 2012 at 7:18 PM, Tom Lane t...@sss.pgh.pa.us wrote: Pursuant to the recent discussion about bugs 6200 and 6245, I went trawling through all the WAL redo routines looking for bugs such as inadequate locking or transiently trashing shared buffers.  Here's what I found: * As we

Re: [HACKERS] Report: race conditions in WAL replay routines

2012-02-05 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Sun, Feb 5, 2012 at 7:18 PM, Tom Lane t...@sss.pgh.pa.us wrote: * seq_redo() has the same disease, since we allow SELECT * FROM sequences. Why do we do that? It's the only existing way to obtain the parameters of a sequence. Even if we felt like

Re: [HACKERS] double writes using double-write buffer approach [WIP]

2012-02-05 Thread Dan Scales
Thanks for the detailed followup. I do see how Postgres is tuned for having a bunch of memory available that is not in shared_buffers, both for the OS buffer cache and other memory allocations. However, Postgres seems to run fine in many large shared_memory configurations that I gave performance

Re: [HACKERS] Report: race conditions in WAL replay routines

2012-02-05 Thread Simon Riggs
On Sun, Feb 5, 2012 at 9:03 PM, Tom Lane t...@sss.pgh.pa.us wrote: * Not exactly a race condition, but: tblspc_redo does ereport(ERROR) if it fails to clean out tablespace directories.  This seems to me to be the height of folly, especially when the failure is more or less an expected case.  

Re: [HACKERS] Report: race conditions in WAL replay routines

2012-02-05 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: Please post the patch rather than fixing directly. There's some subtle stuff there and it would be best to discuss first. Here's a proposed patch for the issues around unlocked updates of shared-memory state. After going through this I believe that

Re: [HACKERS] initdb and fsync

2012-02-05 Thread Noah Misch
On Sun, Feb 05, 2012 at 10:53:20AM -0800, Jeff Davis wrote: On Sat, 2012-02-04 at 20:18 -0500, Noah Misch wrote: If we add fsync calls to the initdb process, they should cover the entire data directory tree. This patch syncs files that initdb.c writes, but we ought to also sync files

Re: [HACKERS] Report: race conditions in WAL replay routines

2012-02-05 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: Please post the patch rather than fixing directly. There's some subtle stuff there and it would be best to discuss first. And here's a proposed patch for not throwing ERROR during replay of DROP TABLESPACE. I had originally thought this would be a

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-05 Thread Shigeru Hanada
2012/2/2 Marko Kreen mark...@gmail.com: I think you want this instead: Â https://commitfest.postgresql.org/action/patch_view?id=769 Somehow I've missed this cool feature. Thanks for the suggestion! -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] disable prompting by default in createuser

2012-02-05 Thread Josh Kupershmidt
On Wed, Feb 1, 2012 at 1:13 PM, Peter Eisentraut pete...@gmx.net wrote: On sön, 2012-01-15 at 18:14 -0500, Josh Kupershmidt wrote: I see this patch includes a small change to dropuser, to make the 'username' argument mandatory if --interactive is not set, for symmetry with createuser's new

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-05 Thread Bruce Momjian
On Sun, Feb 05, 2012 at 08:40:09PM +, Simon Riggs wrote: On Sun, Feb 5, 2012 at 3:59 AM, Bruce Momjian br...@momjian.us wrote: On Sat, Dec 24, 2011 at 03:56:58PM +, Simon Riggs wrote: Also, as far as I can see this patch usurps the page version field, which I find unacceptably

Re: [HACKERS] double writes using double-write buffer approach [WIP]

2012-02-05 Thread Robert Haas
On Sun, Feb 5, 2012 at 4:17 PM, Dan Scales sca...@vmware.com wrote: Thanks for the detailed followup.  I do see how Postgres is tuned for having a bunch of memory available that is not in shared_buffers, both for the OS buffer cache and other memory allocations.  However, Postgres seems to run

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-05 Thread Heikki Linnakangas
On 06.02.2012 05:48, Bruce Momjian wrote: On Sun, Feb 05, 2012 at 08:40:09PM +, Simon Riggs wrote: In the proposed scheme there are two flag bits set on the page to indicate whether the field is used as a checksum rather than a version number. So its possible the checksum could look like a