[HACKERS] Re: patch-3 (3-allow-wal-record-header-to-be-split.patch)WAL Format Changes

2012-07-01 Thread Amit kapila
From: Heikki Linnakangas [heikki.linnakan...@enterprisedb.com] Sent: Sunday, July 01, 2012 1:54 AM On 30.06.2012 10:11, Amit kapila wrote: 3. General observation, not related to your changes XLogInsert(RmgrId rmid, uint8 info, XLogRecData *rdata) . . if (freespace == 0) {

Re: [HACKERS] Patch: add conversion from pg_wchar to multibyte

2012-07-01 Thread Alexander Korotkov
On Wed, Jun 27, 2012 at 11:35 PM, Robert Haas robertmh...@gmail.com wrote: It looks to me like pg_wchar2utf_with_len will not work, because unicode_to_utf8 returns its second argument unmodified - not, as your code seems to assume, the byte following what was already written. Fixed. MULE

[HACKERS] [PATCH] Make pg_basebackup configure and start standby

2012-07-01 Thread Boszormenyi Zoltan
Hi, attached is a patch that does $SUBJECT. It's a usability enhancement, to take a backup, write a minimalistic recovery.conf and start the streaming standby in one go. Comments? Best regards, Zoltán Böszörményi -- -- Zoltán Böszörményi Cybertec Schönig

Re: [HACKERS] [PATCH 14/16] Add module to apply changes from an apply-cache using low-level functions

2012-07-01 Thread Alexander Korotkov
Hi, Andres! There is my review of this patch. 1) Patches don't apply cleanly to head. So I used commit bed88fceac04042f0105eb22a018a4f91d64400d as the base for patches, then all the patches close to this apply cleanly. Regression tests pass OK, but it seems that new functionality isn't covered

Re: [HACKERS] Update on the spinlock-pthread_mutex patch experimental: replace s_lock spinlock code with pthread_mutex on linux

2012-07-01 Thread Robert Haas
On Fri, Jun 29, 2012 at 1:07 PM, Nils Goroll sl...@schokola.de wrote: FWIW, I kicked off a looong benchmarking run on this a couple of days ago on the IBM POWER7 box, testing pgbench -S, regular pgbench, and pgbench --unlogged-tables at various client counts with and without the patch; three

Re: [HACKERS] [v9.3] Row-Level Security

2012-07-01 Thread Kohei KaiGai
2012/6/28 Tom Lane t...@sss.pgh.pa.us: Kohei KaiGai kai...@kaigai.gr.jp writes: 2012/6/27 Florian Pflug f...@phlo.org: Hm, what happens if a SECURITY DEFINER functions returns a refcursor? My impression is, here is no matter even if SECURITY DEFINER function returns refcursor. I think

Re: [HACKERS] Update on the spinlock-pthread_mutex patch experimental: replace s_lock spinlock code with pthread_mutex on linux

2012-07-01 Thread Nils Goroll
Thank you, Robert. as this patch was not targeted towards increasing tps, I am at happy to hear that your benchmarks also suggest that performance is comparable. But my main question is: how about resource consumption? For the issue I am working on, my current working hypothesis is that spinning

Re: [HACKERS] Update on the spinlock-pthread_mutex patch experimental: replace s_lock spinlock code with pthread_mutex on linux

2012-07-01 Thread Nils Goroll
test runs on an IBM POWER7 system with 16 cores, 64 hardware threads. Could you add the CPU Type / clock speed please? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] Make pg_basebackup configure and start standby

2012-07-01 Thread Fujii Masao
On Sun, Jul 1, 2012 at 8:02 PM, Boszormenyi Zoltan z...@cybertec.at wrote: Hi, attached is a patch that does $SUBJECT. It's a usability enhancement, to take a backup, write a minimalistic recovery.conf and start the streaming standby in one go. Comments? Could you add the patch to the

Re: [HACKERS] [PATCH] Make pg_basebackup configure and start standby

2012-07-01 Thread Boszormenyi Zoltan
Hi, 2012-07-01 17:38 keltezéssel, Fujii Masao írta: On Sun, Jul 1, 2012 at 8:02 PM, Boszormenyi Zoltan z...@cybertec.at wrote: Hi, attached is a patch that does $SUBJECT. It's a usability enhancement, to take a backup, write a minimalistic recovery.conf and start the streaming standby in one

Re: [HACKERS] [PATCH] Make pg_basebackup configure and start standby

2012-07-01 Thread Magnus Hagander
On Sun, Jul 1, 2012 at 1:02 PM, Boszormenyi Zoltan z...@cybertec.at wrote: Hi, attached is a patch that does $SUBJECT. It's a usability enhancement, to take a backup, write a minimalistic recovery.conf and start the streaming standby in one go. I like the writing of recovery.conf. In fact,

Re: [HACKERS] [PATCH 14/16] Add module to apply changes from an apply-cache using low-level functions

2012-07-01 Thread Alexander Korotkov
On Sun, Jul 1, 2012 at 3:11 PM, Alexander Korotkov aekorot...@gmail.comwrote: 1) Patches don't apply cleanly to head. So I used commit bed88fceac04042f0105eb22a018a4f91d64400d as the base for patches, then all the patches close to this apply cleanly. Regression tests pass OK, but it seems

Re: [HACKERS] [PATCH] Make pg_basebackup configure and start standby

2012-07-01 Thread Fujii Masao
On Mon, Jul 2, 2012 at 12:42 AM, Boszormenyi Zoltan z...@cybertec.at wrote: Hi, 2012-07-01 17:38 keltezéssel, Fujii Masao írta: On Sun, Jul 1, 2012 at 8:02 PM, Boszormenyi Zoltan z...@cybertec.at wrote: Hi, attached is a patch that does $SUBJECT. It's a usability enhancement, to take a

Re: [HACKERS] [PATCH] Make pg_basebackup configure and start standby

2012-07-01 Thread Fujii Masao
On Mon, Jul 2, 2012 at 12:44 AM, Magnus Hagander mag...@hagander.net wrote: On Sun, Jul 1, 2012 at 1:02 PM, Boszormenyi Zoltan z...@cybertec.at wrote: Hi, attached is a patch that does $SUBJECT. It's a usability enhancement, to take a backup, write a minimalistic recovery.conf and start the

[HACKERS] compiler warnings on the buildfarm

2012-07-01 Thread Stefan Kaltenbrunner
seeing some of the latest commits about fixing compiler warnings I took a look at the buildfarm to see if there are any interesting ones there (in total we have a thousends of warnings on the buildfarm but most of those are from very noisy compilers). so in case anybody is interested those are a

Re: [HACKERS] [ADMIN] pg_basebackup blocking all queries with horrible performance

2012-07-01 Thread Fujii Masao
Thanks for the review! On Fri, Jun 29, 2012 at 7:22 PM, Magnus Hagander mag...@hagander.net wrote: On Wed, Jun 27, 2012 at 7:24 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Jun 21, 2012 at 3:18 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 20, 2012 at 7:18 AM, Magnus

Re: [HACKERS] XX000: enum value 117721 not found in cache for enum enumcrash

2012-07-01 Thread Robert Haas
On Sat, Jun 30, 2012 at 5:51 AM, Andres Freund and...@2ndquadrant.com wrote: Currently its possible to cause transactions to fail with ALTER ENUM ADD AFTER/BEFORE: psql 1: CREATE TYPE enumcrash AS ENUM('a', 'b'); CREATE FUNCTION randenum() RETURNS enumcrash LANGUAGE sql AS $$SELECT * FROM

Re: [HACKERS] Update on the spinlock-pthread_mutex patch experimental: replace s_lock spinlock code with pthread_mutex on linux

2012-07-01 Thread Robert Haas
On Sun, Jul 1, 2012 at 11:13 AM, Nils Goroll sl...@schokola.de wrote: as this patch was not targeted towards increasing tps, I am at happy to hear that your benchmarks also suggest that performance is comparable. But my main question is: how about resource consumption? For the issue I am

Re: [HACKERS] Update on the spinlock-pthread_mutex patch experimental: replace s_lock spinlock code with pthread_mutex on linux

2012-07-01 Thread Robert Haas
On Sun, Jul 1, 2012 at 11:18 AM, Nils Goroll sl...@schokola.de wrote: test runs on an IBM POWER7 system with 16 cores, 64 hardware threads. Could you add the CPU Type / clock speed please? cpu : POWER7 (architected), altivec supported clock : 3550.00MHz revision

Re: [HACKERS] new --maintenance-db options

2012-07-01 Thread Robert Haas
On Fri, Jun 29, 2012 at 3:32 PM, Bruce Momjian br...@momjian.us wrote: On Mon, Jun 25, 2012 at 11:57:36AM -0400, Robert Haas wrote: In retrospect, it seems as though it might have been a good idea to make the postgres database read-only and undroppable, so that all client utilities could count

Re: [HACKERS] [ADMIN] pg_basebackup blocking all queries with horrible performance

2012-07-01 Thread Magnus Hagander
On Sun, Jul 1, 2012 at 7:14 PM, Fujii Masao masao.fu...@gmail.com wrote: On Fri, Jun 29, 2012 at 7:22 PM, Magnus Hagander mag...@hagander.net wrote: On Wed, Jun 27, 2012 at 7:24 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Jun 21, 2012 at 3:18 AM, Robert Haas robertmh...@gmail.com

Re: [HACKERS] Update on the spinlock-pthread_mutex patch experimental: replace s_lock spinlock code with pthread_mutex on linux

2012-07-01 Thread Nils Goroll
Hi Robert, Spinlock contentions cause tps to go down. The fact that tps didn't change much in this case suggests that either these workloads don't generate enough spinlock contention to benefit from your patch, or your patch doesn't meaningfully reduce it, or both. We might need a test

Re: [HACKERS] [PATCH] Make pg_basebackup configure and start standby

2012-07-01 Thread Hans-Jürgen Schönig
On Jul 1, 2012, at 5:44 PM, Magnus Hagander wrote: On Sun, Jul 1, 2012 at 1:02 PM, Boszormenyi Zoltan z...@cybertec.at wrote: Hi, attached is a patch that does $SUBJECT. It's a usability enhancement, to take a backup, write a minimalistic recovery.conf and start the streaming standby in

Re: [HACKERS] Support for array_remove and array_replace functions

2012-07-01 Thread Alex Hunsaker
On Sat, Jun 30, 2012 at 3:28 PM, Marco Nenciarini marco.nenciar...@2ndquadrant.it wrote: On 30/06/2012 04:16, Alex Hunsaker wrote: Hi, I've been reviewing this patch. Good documentation, and regression tests. The code looked fine but I didn't care for the code duplication between

Re: [HACKERS] XX000: enum value 117721 not found in cache for enum enumcrash

2012-07-01 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Sat, Jun 30, 2012 at 5:51 AM, Andres Freund and...@2ndquadrant.com wrote: This is not surprising. psql 2's backend finds rows in the index with enum values that are not visible in its mvcc snapshot. I think the problem is that

Re: [HACKERS] XX000: enum value 117721 not found in cache for enum enumcrash

2012-07-01 Thread Tom Lane
I wrote: Robert Haas robertmh...@gmail.com writes: I think the problem is that load_enum_cache_data() uses GetTransactionSnapshot() rather than GetLatestSnapshot(). That would only make the race condition window smaller (ie, hard to reproduce manually like this, but not gone). No, wait, we

Re: [HACKERS] Update on the spinlock-pthread_mutex patch experimental: replace s_lock spinlock code with pthread_mutex on linux

2012-07-01 Thread Jeff Janes
On Fri, Jun 29, 2012 at 10:07 AM, Nils Goroll sl...@schokola.de wrote: On 06/28/12 05:21 PM, Jeff Janes wrote: It looks like the hacked code is slower than the original.  That doesn't seem so good to me.  Am I misreading this? No, you are right - in a way. This is not about maximizing tps,

Re: [HACKERS] pgbench--new transaction type

2012-07-01 Thread Jeff Janes
On Wed, Jun 20, 2012 at 12:32 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 01.06.2012 03:02, Jeff Janes wrote: I've attached a new patch which addresses several of your concerns, and adds the documentation.  The description is much longer than the descriptions of other

Re: [HACKERS] Update on the spinlock-pthread_mutex patch experimental: replace s_lock spinlock code with pthread_mutex on linux

2012-07-01 Thread Nils Goroll
Hi Jeff, It looks like the hacked code is slower than the original. That doesn't seem so good to me. Am I misreading this? No, you are right - in a way. This is not about maximizing tps, this is about maximizing efficiency under load situations But why wouldn't this maximized efficiency

Re: [HACKERS] We probably need autovacuum_max_wraparound_workers

2012-07-01 Thread Jeff Janes
On Thu, Jun 28, 2012 at 6:57 PM, Josh Berkus j...@agliodbs.com wrote: A second obstacle to opportunistic wraparound vacuum is that wraparound vacuum is not interruptable. If you have to kill it off and do something else for a couple hours, it can't pick up where it left off; it needs to scan

[HACKERS] Proof of concept: auto updatable views

2012-07-01 Thread Dean Rasheed
Hi, I've been playing around with the idea of supporting automatically updatable views, and I have a working proof of concept. I've taken a different approach than the previous attempts to implement this feature (e.g., http://archives.postgresql.org/pgsql-hackers/2009-01/msg01746.php), instead

Re: [HACKERS] Update on the spinlock-pthread_mutex patch experimental: replace s_lock spinlock code with pthread_mutex on linux

2012-07-01 Thread Jeff Janes
On Sun, Jul 1, 2012 at 2:28 PM, Nils Goroll sl...@schokola.de wrote: Hi Jeff, It looks like the hacked code is slower than the original.  That doesn't seem so good to me.  Am I misreading this? No, you are right - in a way. This is not about maximizing tps, this is about maximizing

Re: [HACKERS] Proof of concept: auto updatable views

2012-07-01 Thread Darren Duncan
My thoughts on this is that it would be a very valuable feature to have, and would make Postgres views behave more like they always were intended to behave, which is indistinguishible to users from tables in behavior where all possible, and that the reverse mapping would be automatic with the

Re: [HACKERS] XX000: enum value 117721 not found in cache for enum enumcrash

2012-07-01 Thread Robert Haas
On Jul 1, 2012, at 4:18 PM, Tom Lane t...@sss.pgh.pa.us wrote: However, I'm a bit worried by the if (!FirstSnapshotSet) restriction in GetLatestSnapshot. Are we sure that enum comparisons could never happen without a snapshot already being set? What's the point of throwing an error there

Re: [HACKERS] XX000: enum value 117721 not found in cache for enum enumcrash

2012-07-01 Thread Robert Haas
On Jul 2, 2012, at 12:04 AM, Robert Haas robertmh...@gmail.com wrote: On Jul 1, 2012, at 4:18 PM, Tom Lane t...@sss.pgh.pa.us wrote: However, I'm a bit worried by the if (!FirstSnapshotSet) restriction in GetLatestSnapshot. Are we sure that enum comparisons could never happen without a

Re: [HACKERS] XX000: enum value 117721 not found in cache for enum enumcrash

2012-07-01 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Jul 2, 2012, at 12:04 AM, Robert Haas robertmh...@gmail.com wrote: On Jul 1, 2012, at 4:18 PM, Tom Lane t...@sss.pgh.pa.us wrote: However, I'm a bit worried by the if (!FirstSnapshotSet) restriction in GetLatestSnapshot. I don't know whether it

Re: [HACKERS] XX000: enum value 117721 not found in cache for enum enumcrash

2012-07-01 Thread Alvaro Herrera
Excerpts from Tom Lane's message of lun jul 02 00:24:06 -0400 2012: Robert Haas robertmh...@gmail.com writes: On Jul 2, 2012, at 12:04 AM, Robert Haas robertmh...@gmail.com wrote: On Jul 1, 2012, at 4:18 PM, Tom Lane t...@sss.pgh.pa.us wrote: However, I'm a bit worried by the if