Re: [HACKERS] basic pgbench runs with various performance-related patches

2012-02-05 Thread Robert Haas
On Sat, Feb 4, 2012 at 11:59 AM, Greg Smith wrote: > On 01/24/2012 08:58 AM, Robert Haas wrote: >> >> One somewhat odd thing about these numbers is that, on permanent >> tables, all of the patches seemed to show regressions vs. master in >> single-client throughput.  That's a slightly difficult re

Re: [HACKERS] initdb and fsync

2012-02-05 Thread Jeff Davis
On Sat, 2012-02-04 at 20:18 -0500, Noah Misch wrote: > If we add fsync calls to the initdb process, they should cover the entire data > directory tree. This patch syncs files that initdb.c writes, but we ought to > also sync files that bootstrap-mode backends had written. It doesn't make sense fo

[HACKERS] pl/python long-lived allocations in datum->dict transformation

2012-02-05 Thread Jan Urbański
Consider this: create table arrays as select array[random(), random(), random(), random(), random(), random()] as a from generate_series(1, 100); create or replace function plpython_outputfunc() returns void as $$ c = plpy.cursor('select a from arrays') for row in c: pass $$ language plpy

[HACKERS] plpgsql leaking memory when stringifying datums

2012-02-05 Thread Jan Urbański
While chasing a PL/Python memory leak, I did a few tests with PL/pgSQL and I think there are places where memory is not freed sufficiently early. Attached are two functions that when run will make the backend's memory consumption increase until they finish. With both, the cause is convert_value_to

[HACKERS] Report: race conditions in WAL replay routines

2012-02-05 Thread Tom Lane
Pursuant to the recent discussion about bugs 6200 and 6245, I went trawling through all the WAL redo routines looking for bugs such as inadequate locking or transiently trashing shared buffers. Here's what I found: * As we already knew, RestoreBkpBlocks is broken because it transiently trashes a

Re: [HACKERS] JSON output functions.

2012-02-05 Thread Stefan Keller
Hi Andrew Nice work! Just for completeness: Did you also think of including geometry types in JSON output functions in later releases? There's a nice extension of JSON called GeoJSON for a starting point. Yours, Stefan 2012/2/3 Andrew Dunstan : > > > On 02/02/2012 12:20 PM, Pavel Stehule wrote

Re: [HACKERS] initdb and fsync

2012-02-05 Thread Tom Lane
Jeff Davis writes: > On Sat, 2012-02-04 at 20:18 -0500, Noah Misch wrote: >> If we add fsync calls to the initdb process, they should cover the entire >> data >> directory tree. This patch syncs files that initdb.c writes, but we ought to >> also sync files that bootstrap-mode backends had writt

Re: [HACKERS] JSON output functions.

2012-02-05 Thread Andrew Dunstan
On 02/05/2012 02:31 PM, Stefan Keller wrote: Hi Andrew Nice work! Just for completeness: Did you also think of including geometry types in JSON output functions in later releases? There's a nice extension of JSON called GeoJSON for a starting point. [side note: please don't top-reply on -h

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-05 Thread Kohei KaiGai
2012年2月1日12:15 Shigeru Hanada : > (2012/01/30 4:39), Kohei KaiGai wrote: >> I checked the "fdw_helper_funcs_v3.patch", "pgsql_fdw_v5.patch" and >> "pgsql_fdw_pushdown_v1.patch". My comments are below. > > Thanks for the review! > >> [BUG] >> Even though pgsql_fdw tries to push-down qualifiers being

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-05 Thread Simon Riggs
On Sun, Feb 5, 2012 at 3:59 AM, Bruce Momjian wrote: > On Sat, Dec 24, 2011 at 03:56:58PM +, Simon Riggs wrote: >> > Also, as far as I can see this patch usurps the page version field, >> > which I find unacceptably short-sighted.  Do you really think this is >> > the last page layout change w

Re: [HACKERS] [BUGS] BUG #6425: Bus error in slot_deform_tuple

2012-02-05 Thread Simon Riggs
On Sat, Feb 4, 2012 at 6:49 PM, Tom Lane wrote: > Simon Riggs writes: >> The cause here is data changing underneath the user. Your patch solves >> the most obvious error, but it still allows other problems if applying >> the backup block changes data. If the backup block doesn't do anything >> at

Re: [HACKERS] Report: race conditions in WAL replay routines

2012-02-05 Thread Simon Riggs
On Sun, Feb 5, 2012 at 7:18 PM, Tom Lane wrote: > Pursuant to the recent discussion about bugs 6200 and 6245, I went > trawling through all the WAL redo routines looking for bugs such as > inadequate locking or transiently trashing shared buffers.  Here's > what I found: > > * As we already knew,

Re: [HACKERS] Report: race conditions in WAL replay routines

2012-02-05 Thread Tom Lane
Simon Riggs writes: > On Sun, Feb 5, 2012 at 7:18 PM, Tom Lane wrote: >> * seq_redo() has the same disease, since we allow SELECT * FROM >> sequences. > Why do we do that? It's the only existing way to obtain the parameters of a sequence. Even if we felt like inventing a different API for doing

Re: [HACKERS] double writes using "double-write buffer" approach [WIP]

2012-02-05 Thread Dan Scales
Thanks for the detailed followup. I do see how Postgres is tuned for having a bunch of memory available that is not in shared_buffers, both for the OS buffer cache and other memory allocations. However, Postgres seems to run fine in many "large shared_memory" configurations that I gave performanc

Re: [HACKERS] Report: race conditions in WAL replay routines

2012-02-05 Thread Simon Riggs
On Sun, Feb 5, 2012 at 9:03 PM, Tom Lane wrote: >>> * Not exactly a race condition, but: tblspc_redo does ereport(ERROR) >>> if it fails to clean out tablespace directories.  This seems to me to be >>> the height of folly, especially when the failure is more or less an >>> expected case.  If the e

Re: [HACKERS] Report: race conditions in WAL replay routines

2012-02-05 Thread Tom Lane
Simon Riggs writes: > Please post the patch rather than fixing directly. There's some subtle > stuff there and it would be best to discuss first. Here's a proposed patch for the issues around unlocked updates of shared-memory state. After going through this I believe that there is little risk of

Re: [HACKERS] initdb and fsync

2012-02-05 Thread Noah Misch
On Sun, Feb 05, 2012 at 10:53:20AM -0800, Jeff Davis wrote: > On Sat, 2012-02-04 at 20:18 -0500, Noah Misch wrote: > > If we add fsync calls to the initdb process, they should cover the entire > > data > > directory tree. This patch syncs files that initdb.c writes, but we ought > > to > > also

Re: [HACKERS] Report: race conditions in WAL replay routines

2012-02-05 Thread Tom Lane
Simon Riggs writes: > Please post the patch rather than fixing directly. There's some subtle > stuff there and it would be best to discuss first. And here's a proposed patch for not throwing ERROR during replay of DROP TABLESPACE. I had originally thought this would be a one-liner s/ERROR/LOG/,

Re: [HACKERS] pgsql_fdw, FDW for PostgreSQL server

2012-02-05 Thread Shigeru Hanada
2012/2/2 Marko Kreen : I think you want this instead: Â https://commitfest.postgresql.org/action/patch_view?id=769 Somehow I've missed this cool feature. Thanks for the suggestion! -- Shigeru Hanada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to yo

Re: [HACKERS] disable prompting by default in createuser

2012-02-05 Thread Josh Kupershmidt
On Wed, Feb 1, 2012 at 1:13 PM, Peter Eisentraut wrote: > On sön, 2012-01-15 at 18:14 -0500, Josh Kupershmidt wrote: >> I see this patch includes a small change to dropuser, to make the >> 'username' argument mandatory if --interactive is not set, for >> symmetry with createuser's new behavior. Th

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-05 Thread Bruce Momjian
On Sun, Feb 05, 2012 at 08:40:09PM +, Simon Riggs wrote: > On Sun, Feb 5, 2012 at 3:59 AM, Bruce Momjian wrote: > > On Sat, Dec 24, 2011 at 03:56:58PM +, Simon Riggs wrote: > >> > Also, as far as I can see this patch usurps the page version field, > >> > which I find unacceptably short-sig

Re: [HACKERS] double writes using "double-write buffer" approach [WIP]

2012-02-05 Thread Robert Haas
On Sun, Feb 5, 2012 at 4:17 PM, Dan Scales wrote: > Thanks for the detailed followup.  I do see how Postgres is tuned for > having a bunch of memory available that is not in shared_buffers, both > for the OS buffer cache and other memory allocations.  However, Postgres > seems to run fine in many

Re: [HACKERS] 16-bit page checksums for 9.2

2012-02-05 Thread Heikki Linnakangas
On 06.02.2012 05:48, Bruce Momjian wrote: On Sun, Feb 05, 2012 at 08:40:09PM +, Simon Riggs wrote: In the proposed scheme there are two flag bits set on the page to indicate whether the field is used as a checksum rather than a version number. So its possible the checksum could look like a v