Re: [HACKERS] PL/perl should fail on configure, not make

2013-01-09 Thread Andrew Dunstan
On 01/08/2013 10:37 PM, Tom Lane wrote: We could try adding an AC_TRY_LINK test using perl_embed_ldflags, but given the weird stuff happening to redefine that value on Windows in plperl/GNUmakefile I think there's a serious risk of breaking Cygwin builds. Since I lack access to either Cygwin

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-09 Thread Amit Kapila
On Tuesday, January 08, 2013 8:57 PM Andres Freund wrote: On 2013-01-08 20:33:28 +0530, Amit Kapila wrote: On Tuesday, January 08, 2013 8:01 PM Andres Freund wrote: On 2013-01-08 19:51:39 +0530, Amit Kapila wrote: On Monday, January 07, 2013 7:15 PM Andres Freund wrote: On

Re: [HACKERS] proposal: Set effective_cache_size to greater of .conf value, shared_buffers

2013-01-09 Thread Benedikt Grundmann
On Wed, Jan 9, 2013 at 2:01 AM, Josh Berkus j...@agliodbs.com wrote: All, Well, the problem of find out the box's physical RAM is doubtless solvable if we're willing to put enough sweat and tears into it, but I'm dubious that it's worth the trouble. The harder part is how to know if

[HACKERS] inconsistent behave of boolean based domains in XML functions

2013-01-09 Thread Pavel Stehule
Hello On Czech pg mailing list was reported issue about problems with boolean based domains and XML functions. There are maybe more issues, but probably there is little bit strange and unexpected result postgres=# CREATE DOMAIN booldomain as bool; CREATE DOMAIN -- fully expected behave

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-09 Thread Andres Freund
On 2013-01-09 14:04:32 +0530, Amit Kapila wrote: On Tuesday, January 08, 2013 8:57 PM Andres Freund wrote: On 2013-01-08 20:33:28 +0530, Amit Kapila wrote: On Tuesday, January 08, 2013 8:01 PM Andres Freund wrote: On 2013-01-08 19:51:39 +0530, Amit Kapila wrote: On Monday, January

Re: [HACKERS] Cascading replication: should we detect/prevent cycles?

2013-01-09 Thread Simon Riggs
On 9 January 2013 01:51, Josh Berkus j...@agliodbs.com wrote: Anyway, I'm not saying we solve this now. I'm saying, put it on the TODO list in case someone has time/an itch to scratch. I think its reasonable to ask whether a usability feature needs to exist whenever a problem is encountered.

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-09 Thread Amit Kapila
On Wednesday, January 09, 2013 2:28 PM Andres Freund wrote: On 2013-01-09 14:04:32 +0530, Amit Kapila wrote: On Tuesday, January 08, 2013 8:57 PM Andres Freund wrote: On 2013-01-08 20:33:28 +0530, Amit Kapila wrote: On Tuesday, January 08, 2013 8:01 PM Andres Freund wrote: On

Re: [HACKERS] PL/perl should fail on configure, not make

2013-01-09 Thread Christoph Berg
Re: Tom Lane 2013-01-09 9802.1357702...@sss.pgh.pa.us Item: there is not a test for perl.h, as such, in configure. There probably should be, just because we have comparable tests for tcl.h and Python.h. However, adding one won't fix your problem on Debian-based distros, because for some

Re: [HACKERS] Re: Proposal: Store timestamptz of database creation on pg_database

2013-01-09 Thread Hannu Krosing
One thing i'd really like to be in this common object info catalog is DDL which created or altered the referenced object.  If we additionally could make it possible to have ordinary triggers on this catalog it would solve most logical DDL replication problems Hannu Sent from Samsung Galaxy

Re: [HACKERS] Re: patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-01-09 Thread Simon Riggs
On 24 December 2012 16:57, Amit Kapila amit.kap...@huawei.com wrote: Performance: Average of 3 runs of pgbench in tps 9.3devel | with trailing null patch --+-- 578.9872 | 573.4980 On balance, it would seem optimizing for this special case would affect

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2013-01-09 Thread Simon Riggs
On 9 January 2013 08:05, Amit kapila amit.kap...@huawei.com wrote: Update patch contains handling of below Comments Thanks Test results with modified pgbench (1800 record size) on the latest patch: -Patch- -tps@-c1- -WAL@-c1- -tps@-c2- -WAL@-c2- Head

[HACKERS] [PATCH 2/2] use pg_malloc instead of an unchecked malloc in pg_resetxlog

2013-01-09 Thread Andres Freund
--- src/bin/pg_resetxlog/pg_resetxlog.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/src/bin/pg_resetxlog/pg_resetxlog.c b/src/bin/pg_resetxlog/pg_resetxlog.c index 8734f2c..60fb30c 100644 --- a/src/bin/pg_resetxlog/pg_resetxlog.c +++

[HACKERS] [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Andres Freund
Hi, As promised here's a patch to provide palloc emulation for frontend-ish environments. The patch: - makes palloc() into a real function so CurrentMemoryContext doesn't need to be provided - provides common pg_(malloc,malloc0, realloc, strdup, free) wrappers and removes various versions of

[HACKERS] information schema parameter_default implementation

2013-01-09 Thread Peter Eisentraut
Here is an implementation of the information_schema.parameters.parameter_default column. I ended up writing a C function to decode the whole thing from the system catalogs, because it was too complicated in SQL, so I abandoned the approach discussed in [0]. [0]:

[HACKERS] [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Andres Freund
--- contrib/oid2name/oid2name.c| 52 + contrib/pg_upgrade/pg_upgrade.h| 5 +- contrib/pg_upgrade/util.c | 49 - contrib/pgbench/pgbench.c | 54 +- src/backend/utils/mmgr/mcxt.c | 78

Re: [HACKERS] [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Heikki Linnakangas
On 09.01.2013 13:27, Andres Freund wrote: - makes palloc() into a real function so CurrentMemoryContext doesn't need to be provided I don't understand the need for this change. Can't you just: #define palloc(s) pg_malloc(s) in the frontend context? - Heikki -- Sent via pgsql-hackers

Re: [HACKERS] Further pg_upgrade analysis for many tables

2013-01-09 Thread Simon Riggs
On 23 November 2012 22:34, Jeff Janes jeff.ja...@gmail.com wrote: I got rid of need_eoxact_work entirely and replaced it with a short list that fulfills the functions of indicating that work is needed, and suggesting which rels might need that work. There is no attempt to prevent duplicates,

Re: [HACKERS] [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Andres Freund
On 2013-01-09 13:46:53 +0200, Heikki Linnakangas wrote: On 09.01.2013 13:27, Andres Freund wrote: - makes palloc() into a real function so CurrentMemoryContext doesn't need to be provided I don't understand the need for this change. Can't you just: #define palloc(s) pg_malloc(s) in the

Re: [HACKERS] Further pg_upgrade analysis for many tables

2013-01-09 Thread Simon Riggs
On 9 November 2012 18:50, Jeff Janes jeff.ja...@gmail.com wrote: quadratic behavior in the resource owner/lock table I didn't want to let that particular phrase go by without saying exactly what behaviour is that?, so we can discuss fixing that also. This maybe something I already know about,

Re: [HACKERS] Re: patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-01-09 Thread Amit Kapila
On Wednesday, January 09, 2013 4:52 PM Simon Riggs wrote: On 24 December 2012 16:57, Amit Kapila amit.kap...@huawei.com wrote: Performance: Average of 3 runs of pgbench in tps 9.3devel | with trailing null patch --+-- 578.9872 | 573.4980 On

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2013-01-09 Thread Amit Kapila
On Wednesday, January 09, 2013 4:57 PM Simon Riggs wrote: On 9 January 2013 08:05, Amit kapila amit.kap...@huawei.com wrote: Update patch contains handling of below Comments Thanks Test results with modified pgbench (1800 record size) on the latest patch: -Patch-

Re: [HACKERS] Re: patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2013-01-09 Thread Simon Riggs
On 9 January 2013 12:06, Amit Kapila amit.kap...@huawei.com wrote: On Wednesday, January 09, 2013 4:52 PM Simon Riggs wrote: On 24 December 2012 16:57, Amit Kapila amit.kap...@huawei.com wrote: Performance: Average of 3 runs of pgbench in tps 9.3devel | with trailing null patch

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-09 Thread Andres Freund
On 2013-01-09 15:06:04 +0530, Amit Kapila wrote: On Wednesday, January 09, 2013 2:28 PM Andres Freund wrote: On 2013-01-09 14:04:32 +0530, Amit Kapila wrote: On Tuesday, January 08, 2013 8:57 PM Andres Freund wrote: On 2013-01-08 20:33:28 +0530, Amit Kapila wrote: On Tuesday,

Re: [HACKERS] [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Magnus Hagander
Am I the only one who finds this way of posting patches really annoying? Here is a patch with no description other than a list of changed files. And discussion happens in a completely different email. What's wrong with just posting the patch as a regular attachment(s) to a regular thread, like

[HACKERS] Commitfest Topics

2013-01-09 Thread Simon Riggs
I've set up commifest topics for CFJan15. This will allow people to move across any patches from earlier commitfests. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list

Re: [HACKERS] [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Andres Freund
On 2013-01-09 13:34:12 +0100, Magnus Hagander wrote: Am I the only one who finds this way of posting patches really annoying? Well, I unsurprisingly don't ;) Here is a patch with no description other than a list of changed files. And discussion happens in a completely different email. They

Re: [HACKERS] [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Magnus Hagander
On Wed, Jan 9, 2013 at 1:47 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-01-09 13:34:12 +0100, Magnus Hagander wrote: Am I the only one who finds this way of posting patches really annoying? Well, I unsurprisingly don't ;) Yeah, that's not surprising :) Here is a patch with no

[HACKERS] askpass program for libpq

2013-01-09 Thread Peter Eisentraut
I would like to have something like ssh-askpass for libpq. The main reason is that I don't want to have passwords in plain text on disk, even if .pgpass is read protected. By getting the password from an external program, I can integrate libpq tools with the host system's key chain or wallet

Re: [HACKERS] [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Michael Paquier
On Wed, Jan 9, 2013 at 9:54 PM, Magnus Hagander mag...@hagander.net wrote: On Wed, Jan 9, 2013 at 1:47 PM, Andres Freund and...@2ndquadrant.com wrote: Yeah, so far it's also just my opinion in the other direction :) Hopefully, some others will have thoughts about it too. Just giving my 2c

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-09 Thread Amit Kapila
On Wednesday, January 09, 2013 5:49 PM Andres Freund wrote: On 2013-01-09 15:06:04 +0530, Amit Kapila wrote: On Wednesday, January 09, 2013 2:28 PM Andres Freund wrote: On 2013-01-09 14:04:32 +0530, Amit Kapila wrote: On Tuesday, January 08, 2013 8:57 PM Andres Freund wrote: On

Re: [HACKERS] [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Andres Freund
On 2013-01-09 22:23:25 +0900, Michael Paquier wrote: On Wed, Jan 9, 2013 at 9:54 PM, Magnus Hagander mag...@hagander.net wrote: On Wed, Jan 9, 2013 at 1:47 PM, Andres Freund and...@2ndquadrant.com wrote: Yeah, so far it's also just my opinion in the other direction :) Hopefully, some

Re: [HACKERS] pg_upgrade with parallel tablespace copying

2013-01-09 Thread Bruce Momjian
Slightly modified patch applied. This is my last planned pg_upgrade change for 9.3. --- On Mon, Jan 7, 2013 at 10:51:21PM -0500, Bruce Momjian wrote: Pg_upgrade by default (without --link) copies heap/index files from

Re: [HACKERS] [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Alvaro Herrera
How hard is the backend hit by palloc being now an additional function call? Would it be a good idea to make it (and friends) STATIC_IF_INLINE? diff --git a/src/include/port/palloc.h b/src/include/port/palloc.h new file mode 100644 index 000..a7900bf --- /dev/null +++

Re: [HACKERS] askpass program for libpq

2013-01-09 Thread Magnus Hagander
On Wed, Jan 9, 2013 at 2:17 PM, Peter Eisentraut pete...@gmx.net wrote: I would like to have something like ssh-askpass for libpq. The main reason is that I don't want to have passwords in plain text on disk, even if .pgpass is read protected. By getting the password from an external

[HACKERS] Re: [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Andres Freund
On 2013-01-09 11:45:12 -0300, Alvaro Herrera wrote: How hard is the backend hit by palloc being now an additional function call? Would it be a good idea to make it (and friends) STATIC_IF_INLINE? diff --git a/src/include/port/palloc.h b/src/include/port/palloc.h new file mode 100644

Re: [HACKERS] PL/perl should fail on configure, not make

2013-01-09 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 01/08/2013 10:37 PM, Tom Lane wrote: We could try adding an AC_TRY_LINK test using perl_embed_ldflags, but given the weird stuff happening to redefine that value on Windows in plperl/GNUmakefile I think there's a serious risk of breaking Cygwin

Re: [HACKERS] [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Andres Freund
On 2013-01-09 11:45:12 -0300, Alvaro Herrera wrote: How hard is the backend hit by palloc being now an additional function call? Would it be a good idea to make it (and friends) STATIC_IF_INLINE? Missed this at first... I don't think there's any measurable hit now as there is no additional

Re: [HACKERS] PL/perl should fail on configure, not make

2013-01-09 Thread Christian Ullrich
* Christoph Berg wrote: Re: Tom Lane 2013-01-09 9802.1357702...@sss.pgh.pa.us and Python.h. However, adding one won't fix your problem on Debian-based distros, because for some wacko reason they put the headers and the shlib .so symlink in different packages, cf

[HACKERS] segmentation fault in pg_basebackup

2013-01-09 Thread Fujii Masao
Hi, The pg_basebackup in HEAD caused a segmentation fault in my box. $ pg_basebackup -D mmm Segmentation fault: 11 The cause is that WriteRecoveryConf() is wrongly called even if -R option is not specified in pg_basebackup. Attached patch fixes this problem. Regards, -- Fujii Masao

Re: [HACKERS] segmentation fault in pg_basebackup

2013-01-09 Thread Magnus Hagander
On Wed, Jan 9, 2013 at 4:56 PM, Fujii Masao masao.fu...@gmail.com wrote: Hi, The pg_basebackup in HEAD caused a segmentation fault in my box. $ pg_basebackup -D mmm Segmentation fault: 11 The cause is that WriteRecoveryConf() is wrongly called even if -R option is not specified in

Re: [HACKERS] PL/perl should fail on configure, not make

2013-01-09 Thread Tom Lane
Christian Ullrich ch...@chrullrich.net writes: * Christoph Berg wrote: Re: Tom Lane 2013-01-09 9802.1357702...@sss.pgh.pa.us I am unfamiliar with a good reason for doing that. (I can certainly see segregating the .a static library, or even not shipping it at all, but what's it save to leave

Re: [HACKERS] [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Wed, Jan 9, 2013 at 1:47 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-01-09 13:34:12 +0100, Magnus Hagander wrote: Am I the only one who finds this way of posting patches really annoying? Well, I unsurprisingly don't ;) Yeah, that's

Re: [HACKERS] [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-01-09 13:46:53 +0200, Heikki Linnakangas wrote: I don't understand the need for this change. Can't you just: #define palloc(s) pg_malloc(s) in the frontend context? Yes, that would be possible, but imo its the inferior solution: I'm with

Re: [HACKERS] [PATCH] Patch to fix missing libecpg_compat.lib and libpgtypes.lib.

2013-01-09 Thread Magnus Hagander
On Tue, Jan 8, 2013 at 4:16 AM, Peter Eisentraut pete...@gmx.net wrote: On Mon, 2012-11-19 at 14:03 +0800, JiangGuiqing wrote: hi I install postgresql-9.1.5 from source code on windows successfully. But there is not libecpg_compat.lib and libpgtypes.lib in the installed lib directory . If

Re: [HACKERS] [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Andres Freund
On 2013-01-09 11:37:46 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-01-09 13:46:53 +0200, Heikki Linnakangas wrote: I don't understand the need for this change. Can't you just: #define palloc(s) pg_malloc(s) in the frontend context? Yes, that would be

Re: [HACKERS] [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Peter Geoghegan
On 9 January 2013 16:27, Tom Lane t...@sss.pgh.pa.us wrote: I'm with Magnus. This is seriously annoying, especially when the discussion thread has a title not closely related to the patch emails. (It doesn't help any that the list server doesn't manage to deliver the emails in order, at

[HACKERS] Re: [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Andres Freund
On 2013-01-09 11:27:46 -0500, Tom Lane wrote: Magnus Hagander mag...@hagander.net writes: On Wed, Jan 9, 2013 at 1:47 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-01-09 13:34:12 +0100, Magnus Hagander wrote: Am I the only one who finds this way of posting patches really

Re: [HACKERS] [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On 2013-01-09 11:37:46 -0500, Tom Lane wrote: I agree that what dirmod is doing is pretty ugly, but it's localized enough that I don't care particularly. (Really, the only reason it's a hack and not The Solution is that at the moment there's only one

Re: [HACKERS] Event Triggers: adding information

2013-01-09 Thread Alvaro Herrera
Dimitri Fontaine escribió: Alvaro Herrera alvhe...@2ndquadrant.com writes: Given the OID, we use the ObjectProperty[] structure to know which cache or catalog to scan in order to get the name and namespace of the object. The changes to make ObjectPropertyType visible to code outside

[HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Andres Freund
On 2013-01-09 11:57:35 -0500, Tom Lane wrote: Andres Freund and...@anarazel.de writes: On 2013-01-09 11:37:46 -0500, Tom Lane wrote: I agree that what dirmod is doing is pretty ugly, but it's localized enough that I don't care particularly. (Really, the only reason it's a hack and not

Re: [HACKERS] PL/perl should fail on configure, not make

2013-01-09 Thread Peter Eisentraut
On 1/9/13 10:30 AM, Christian Ullrich wrote: * Christoph Berg wrote: Re: Tom Lane 2013-01-09 9802.1357702...@sss.pgh.pa.us and Python.h. However, adding one won't fix your problem on Debian-based distros, because for some wacko reason they put the headers and the shlib .so symlink in

Re: [HACKERS] PL/perl should fail on configure, not make

2013-01-09 Thread Peter Eisentraut
On 1/9/13 11:00 AM, Tom Lane wrote: The libperl-dev package, as constituted, doesn't make any sense: it's got the symlink which people need, and a very large static (.a) library that most people don't need. Even worse, you can't tell without close inspection which of those files is actually

[HACKERS] PostgreSQL hackfest @ Developer Conference 2013, Brno, CZ

2013-01-09 Thread Honza Horak
Hi guys, let me announce PostgreSQL hackfest, which is held during Developer Conference 2013. Everyone from users, admins to hackers is welcome. Few words about Developer Conference: Developer Conference is an yearly conference for all Linux and JBoss Developers, Admins and Linux users

Re: [HACKERS] [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Tom Lane
Andres Freund and...@anarazel.de writes: On 2013-01-09 11:57:35 -0500, Tom Lane wrote: My objection is that you're forcing *all* backend code to go through the trampoline. That probably has a negative impact on performance, and if you think it'll get committed without a thorough proof that

Re: [HACKERS] Further pg_upgrade analysis for many tables

2013-01-09 Thread Jeff Janes
On Wed, Jan 9, 2013 at 3:59 AM, Simon Riggs si...@2ndquadrant.com wrote: On 9 November 2012 18:50, Jeff Janes jeff.ja...@gmail.com wrote: quadratic behavior in the resource owner/lock table I didn't want to let that particular phrase go by without saying exactly what behaviour is that?, so

Re: [HACKERS] PL/perl should fail on configure, not make

2013-01-09 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On 1/9/13 11:00 AM, Tom Lane wrote: The libperl-dev package, as constituted, doesn't make any sense: it's got the symlink which people need, and a very large static (.a) library that most people don't need. Even worse, you can't tell without close

Re: [HACKERS] proposal: Set effective_cache_size to greater of .conf value, shared_buffers

2013-01-09 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Well, the problem of find out the box's physical RAM is doubtless solvable if we're willing to put enough sweat and tears into it, but I'm dubious that it's worth the trouble. The harder part is how to know if the box is supposed to be dedicated to the

[HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Andres Freund
On 2013-01-09 12:32:20 -0500, Tom Lane wrote: Andres Freund and...@anarazel.de writes: On 2013-01-09 11:57:35 -0500, Tom Lane wrote: My objection is that you're forcing *all* backend code to go through the trampoline. That probably has a negative impact on performance, and if you think

Re: [HACKERS] proposal: Set effective_cache_size to greater of .conf value, shared_buffers

2013-01-09 Thread Josh Berkus
Dimitri, It seems to me that pgfincore has the smarts we need to know about that, and that Cédric has code and refenrences for making it work on all platforms we care about (linux, bsd, windows for starters). Well, fincore is Linux-only, and for that matter only more recent versions of linux.

Re: [HACKERS] proposal: Set effective_cache_size to greater of .conf value, shared_buffers

2013-01-09 Thread Claudio Freire
On Wed, Jan 9, 2013 at 3:39 PM, Josh Berkus j...@agliodbs.com wrote: It seems to me that pgfincore has the smarts we need to know about that, and that Cédric has code and refenrences for making it work on all platforms we care about (linux, bsd, windows for starters). Well, fincore is

Re: [HACKERS] proposal: Set effective_cache_size to greater of .conf value, shared_buffers

2013-01-09 Thread Josh Berkus
Claudio, Not really. I'm convinced, and not only for e_c_s, that autoconfiguration is within the realm of possibility. Hey, if you can do it, my hat's off to you. In any case, as eavesdroppers can infer a cryptographic key by timing operations or measuring power consumption, I'm pretty sure

Re: [HACKERS] Re: [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-01-09 11:27:46 -0500, Tom Lane wrote: I'd prefer posting a single message with the discussion and the patch(es). If you think it's helpful to split a patch into separate parts for reviewing, add multiple attachments. But my experience is

Re: [HACKERS] [PATCH] PL/Python: Add spidata to all spiexceptions

2013-01-09 Thread Jan Urbański
On 12/12/12 20:21, Karl O. Pinc wrote: There were 2 outstanding issue raised: Is this useful enough/proceeding in the right direction to commit now? Should some of the logic be moved out of a subroutine and into the calling code? I can see arguments to be made for both sides. I'm

Re: [HACKERS] Re: [PATCH 1/2] Provide a common malloc wrappers and palloc et al. emulation for frontend'ish environs

2013-01-09 Thread anara...@anarazel.de
Tom Lane t...@sss.pgh.pa.us schrieb: Andres Freund and...@2ndquadrant.com writes: On 2013-01-09 11:27:46 -0500, Tom Lane wrote: I'd prefer posting a single message with the discussion and the patch(es). If you think it's helpful to split a patch into separate parts for reviewing, add

Re: [HACKERS] [PATCH] PL/Python: Add spidata to all spiexceptions

2013-01-09 Thread Karl O. Pinc
On 01/09/2013 01:08:39 PM, Jan Urbański wrote: I can see arguments to be made for both sides. I'm asking that you think it through to the extent you deem necessary and make changes or not. At that point it should be ready to send to a committer. (I'll re-test first, if you make any

Re: [HACKERS] Further pg_upgrade analysis for many tables

2013-01-09 Thread Simon Riggs
On 9 January 2013 17:50, Jeff Janes jeff.ja...@gmail.com wrote: On Wed, Jan 9, 2013 at 3:59 AM, Simon Riggs si...@2ndquadrant.com wrote: On 9 November 2012 18:50, Jeff Janes jeff.ja...@gmail.com wrote: quadratic behavior in the resource owner/lock table I didn't want to let that particular

Re: [HACKERS] buffer assertion tripping under repeat pgbench load

2013-01-09 Thread Greg Smith
On 12/23/12 3:17 PM, Simon Riggs wrote: We already have PrintBufferLeakWarning() for this, which might be a bit neater. It does look like basically the same info. I hacked the code to generate this warning all the time. Patch from Andres I've been using: WARNING: refcount of buf 1

Re: [HACKERS] Index build temp files

2013-01-09 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Does the user running CREATE INDEX have CREATE permission on those tablespaces? (A quick way to double check is to try to SET temp_tablespaces to that value explicitly.) The code will silently ignore any temp tablespaces you don't have such permission

Re: [HACKERS] Index build temp files

2013-01-09 Thread Stephen Frost
* Stephen Frost (sfr...@snowman.net) wrote: Attached are two test scripts and results from them. The gist of it is this: Now with the attachements. Apologies about that. Note that you'll need to create the temp tablespace first. Thanks, Stephen test_tblspace.sql

Re: [HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: Well, I *did* benchmark it as noted elsewhere in the thread, but thats obviously just machine (E5520 x 2) with one rather restricted workload (pgbench -S -jc 40 -T60). At least its rather palloc heavy. Here are the numbers: before:

Re: [HACKERS] Index build temp files

2013-01-09 Thread Simon Riggs
On 9 January 2013 02:42, Tom Lane t...@sss.pgh.pa.us wrote: Stephen Frost sfr...@snowman.net writes: Here's what we're seeing: postgresql.conf: temp_tablespaces = 'temp_01,temp_02' I have temp file logging on in postgresql.conf, so here's what we see: LOG: temporary file: path

Re: [HACKERS] Index build temp files

2013-01-09 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: Alright, this isn't quite as open-and-shut as it may have originally seemed. We're apparently cacheing the temp tablespaces which should be used, even across set role's and security definer functions, which I would argue isn't correct. Ah. Yeah, that

[HACKERS] Reducing size of WAL record headers

2013-01-09 Thread Simon Riggs
Overall, the WAL record is MAXALIGN'd, so with 8 byte alignment we waste 4 bytes per record. Or put another way, if we could reduce record header by 4 bytes, we would actually reduce it by 8 bytes per record. So looking for ways to do that seems like a good idea. The WAL record header starts with

Re: [HACKERS] Feature Request: pg_replication_master()

2013-01-09 Thread Bruce Momjian
On Thu, Jan 3, 2013 at 07:45:32PM +, Simon Riggs wrote: On 3 January 2013 18:35, Josh Berkus j...@agliodbs.com wrote: Robert, In my view, the biggest problem with recovery.conf is that the parameters in there are not GUCs, which means that all of the infrastructure that we've built

Re: [HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Tom Lane
I wrote: I then applied the palloc.h and mcxt.c hunks of your patch and rebuilt. Now I get an average runtime of 1 ms, a full 2% faster, which is a bit astonishing, particularly because the oprofile results haven't moved much: I studied the assembly code being generated for palloc(), and

Re: [HACKERS] Index build temp files

2013-01-09 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: We do have mechanism that forces search_path to be recomputed across changes of active role, but it's expensive to do that, and it seems of rather debatable value to do it here --- it certainly wouldn't improve Stephen's original problem, much less the

Re: [HACKERS] Index build temp files

2013-01-09 Thread Stephen Frost
* Simon Riggs (si...@2ndquadrant.com) wrote: I think we need to allow a TEMP permission on tablespaces, so that you aren't allowed to create normal objects but you can create TEMP objects and sort files there. I agree that this would be valuable. Would we end up needing to burn another bit

Re: [HACKERS] Reducing size of WAL record headers

2013-01-09 Thread Heikki Linnakangas
On 09.01.2013 22:36, Simon Riggs wrote: Overall, the WAL record is MAXALIGN'd, so with 8 byte alignment we waste 4 bytes per record. Or put another way, if we could reduce record header by 4 bytes, we would actually reduce it by 8 bytes per record. So looking for ways to do that seems like a

Re: [HACKERS] Reducing size of WAL record headers

2013-01-09 Thread Bruce Momjian
On Wed, Jan 9, 2013 at 10:54:33PM +0200, Heikki Linnakangas wrote: On 09.01.2013 22:36, Simon Riggs wrote: Overall, the WAL record is MAXALIGN'd, so with 8 byte alignment we waste 4 bytes per record. Or put another way, if we could reduce record header by 4 bytes, we would actually reduce it

Re: [HACKERS] Reducing size of WAL record headers

2013-01-09 Thread Heikki Linnakangas
On 09.01.2013 22:59, Bruce Momjian wrote: On Wed, Jan 9, 2013 at 10:54:33PM +0200, Heikki Linnakangas wrote: On 09.01.2013 22:36, Simon Riggs wrote: The WAL record header starts with xl_tot_len, a 4 byte field. There is also another field, xl_len. The difference is that xl_tot_len includes

Re: [HACKERS] Index build temp files

2013-01-09 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Tom Lane (t...@sss.pgh.pa.us) wrote: We do have mechanism that forces search_path to be recomputed across changes of active role, but it's expensive to do that, and it seems of rather debatable value to do it here --- it certainly wouldn't improve

Re: [HACKERS] PL/perl should fail on configure, not make

2013-01-09 Thread Andrew Dunstan
On 01/09/2013 10:16 AM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 01/08/2013 10:37 PM, Tom Lane wrote: We could try adding an AC_TRY_LINK test using perl_embed_ldflags, but given the weird stuff happening to redefine that value on Windows in plperl/GNUmakefile I think

Re: [HACKERS] Index build temp files

2013-01-09 Thread Simon Riggs
On 9 January 2013 20:53, Stephen Frost sfr...@snowman.net wrote: * Simon Riggs (si...@2ndquadrant.com) wrote: I think we need to allow a TEMP permission on tablespaces, so that you aren't allowed to create normal objects but you can create TEMP objects and sort files there. I agree that this

Re: [HACKERS] PL/perl should fail on configure, not make

2013-01-09 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 01/09/2013 10:16 AM, Tom Lane wrote: Actually, if we were to try to clean this up, I'd suggest moving that logic into the configure script --- it's not apparent to me why it's a good idea to be changing configure-determined values in the Makefile.

Re: [HACKERS] Reducing size of WAL record headers

2013-01-09 Thread Simon Riggs
On 9 January 2013 21:02, Heikki Linnakangas hlinnakan...@vmware.com wrote: OK, crazy idea, but can we just record xl_len as a difference against xl_tot_len, and shorten the xl_len field? Hmm, so it would essentially be the length of all the backup blocks. perhaps rename it to xl_bkpblk_len.

Re: [HACKERS] Reducing size of WAL record headers

2013-01-09 Thread Simon Riggs
On 9 January 2013 20:54, Heikki Linnakangas hlinnakan...@vmware.com wrote: Here's a better idea: Let's keep xl_tot_len as it is, but move xl_len at the very end of the WAL record, after all the backup blocks. If there are no backup blocks, xl_len is omitted. Seems more robust to keep

Re: [HACKERS] Index build temp files

2013-01-09 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 9 January 2013 20:53, Stephen Frost sfr...@snowman.net wrote: * Simon Riggs (si...@2ndquadrant.com) wrote: I think we need to allow a TEMP permission on tablespaces, so that you aren't allowed to create normal objects but you can create TEMP

Re: [HACKERS] Index build temp files

2013-01-09 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: IIRC, we do have mechanism now whereby a superuser can establish settings for SUSET variables via ALTER ROLE SET/ALTER DATABASE SET, and everything works as expected. I'm not entirely convinced that it works as expected, at least for temp_tablespaces

Re: [HACKERS] Index build temp files

2013-01-09 Thread Simon Riggs
On 9 January 2013 21:21, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 9 January 2013 20:53, Stephen Frost sfr...@snowman.net wrote: * Simon Riggs (si...@2ndquadrant.com) wrote: I think we need to allow a TEMP permission on tablespaces, so that you aren't

Re: [HACKERS] Index build temp files

2013-01-09 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 9 January 2013 21:21, Tom Lane t...@sss.pgh.pa.us wrote: This sounds like rather a lot of work to create a behavior that doesn't solve the originally-complained-of usability problem. All it does is make things even more complicated, and add an

Re: [HACKERS] Reducing size of WAL record headers

2013-01-09 Thread Bruce Momjian
On Wed, Jan 9, 2013 at 09:15:16PM +, Simon Riggs wrote: On 9 January 2013 21:02, Heikki Linnakangas hlinnakan...@vmware.com wrote: OK, crazy idea, but can we just record xl_len as a difference against xl_tot_len, and shorten the xl_len field? Hmm, so it would essentially be the

Re: [HACKERS] PL/perl should fail on configure, not make

2013-01-09 Thread Andrew Dunstan
On 01/09/2013 04:12 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 01/09/2013 10:16 AM, Tom Lane wrote: Actually, if we were to try to clean this up, I'd suggest moving that logic into the configure script --- it's not apparent to me why it's a good idea to be changing

Re: [HACKERS] Index build temp files

2013-01-09 Thread Simon Riggs
On 9 January 2013 21:42, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 9 January 2013 21:21, Tom Lane t...@sss.pgh.pa.us wrote: This sounds like rather a lot of work to create a behavior that doesn't solve the originally-complained-of usability problem. All

Re: [HACKERS] Reducing size of WAL record headers

2013-01-09 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: Overall, the WAL record is MAXALIGN'd, so with 8 byte alignment we waste 4 bytes per record. Or put another way, if we could reduce record header by 4 bytes, we would actually reduce it by 8 bytes per record. So looking for ways to do that seems like a

Re: [HACKERS] Index build temp files

2013-01-09 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 9 January 2013 21:42, Tom Lane t...@sss.pgh.pa.us wrote: If we were designing this from scratch I'd agree that a separate TEMP privilege would be a good thing. But bolting one on now is likely to create more problems than it fixes. Particularly

[HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Andres Freund
On 2013-01-09 15:43:19 -0500, Tom Lane wrote: I wrote: I then applied the palloc.h and mcxt.c hunks of your patch and rebuilt. Now I get an average runtime of 1 ms, a full 2% faster, which is a bit astonishing, particularly because the oprofile results haven't moved much: I studied

[HACKERS] Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Andres Freund
On 2013-01-09 15:07:10 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: Well, I *did* benchmark it as noted elsewhere in the thread, but thats obviously just machine (E5520 x 2) with one rather restricted workload (pgbench -S -jc 40 -T60). At least its rather palloc

Re: [HACKERS] [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-01-09 15:43:19 -0500, Tom Lane wrote: I had thought of proposing that we code palloc() like this: void * palloc(Size size) { MemoryContext context = CurrentMemoryContext; AssertArg(MemoryContextIsValid(context)); if

Re: [HACKERS] [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)

2013-01-09 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: I wonder whether it makes sense to inline the contents pstrdup() additionally? My gut feeling is not, but... I don't recall ever having seen MemoryContextStrdup amount to much, so probably not worth the extra code space.

Re: [HACKERS] Index build temp files

2013-01-09 Thread Simon Riggs
On 9 January 2013 22:09, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 9 January 2013 21:42, Tom Lane t...@sss.pgh.pa.us wrote: If we were designing this from scratch I'd agree that a separate TEMP privilege would be a good thing. But bolting one on now is

  1   2   >