Re: [HACKERS] [PATCH] Patch to fix a crash of psql

2012-11-30 Thread Albe Laurenz
Tatsuo Ishii wrote: I think we should detect the cases as much as possible and warn users, rather than silently ignore that fact client encoding != file encoding. I don't think we can detect it in a reliable way, but at least we could check the cases above(sum of PQmblen is not equale to

Re: [HACKERS] DEALLOCATE IF EXISTS

2012-11-30 Thread Vik Reykja
On Tue, Nov 27, 2012 at 3:15 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I fail to see the point of DEALLOCATE IF EXISTS. Do you have real use case for this, or was this just a case of adding IF EXISTS to all commands for the sake of completeness? Usually the client knows what

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Dimitri Fontaine
Markus Wanner mar...@bluegap.ch writes: However, as you say, maybe we need more coding examples. Maybe a minimally usable extra daemon example? Showing how to avoid common pitfalls? Use cases, anybody? :-) What about the PGQ ticker, pgqd?

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Pavel Stehule
2012/11/30 Dimitri Fontaine dimi...@2ndquadrant.fr: Markus Wanner mar...@bluegap.ch writes: However, as you say, maybe we need more coding examples. Maybe a minimally usable extra daemon example? Showing how to avoid common pitfalls? Use cases, anybody? :-) What about the PGQ ticker, pgqd?

Re: [HACKERS] proposal: fix corner use case of variadic fuctions usage

2012-11-30 Thread Vik Reykja
On Sun, Nov 4, 2012 at 12:49 PM, Pavel Stehule pavel.steh...@gmail.comwrote: Hello here is patch, that enables using a variadic parameter modifier for variadic any function. Motivation for this patch is consistent behave of format function, but it fixes behave of all this class variadic

[HACKERS] Review: create extension default_full_version

2012-11-30 Thread Ibrar Ahmed
Hi, I looked at the discussion for this patch and the patch itself. Here are my comments and observations about the patch. What I got from the discussion is that the patch tries to implement a mechanism to install extension from series of SQL scripts from base/full version e.g. if a user wants to

Re: [HACKERS] Refactoring standby mode logic

2012-11-30 Thread Dimitri Fontaine
Hi, Heikki Linnakangas hlinnakan...@vmware.com writes: Attached is a patch to refactor that logic into a more straightforward state machine. It's always been a kind of a state machine, but it's been hard to see, as the code wasn't explicitly written that way. Any objections? On a quick glance

Re: [HACKERS] WIP: index support for regexp search

2012-11-30 Thread Alexander Korotkov
On Thu, Nov 29, 2012 at 5:25 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: One thing that bothers me with this algoritm is that the overflow mechanism is all-or-nothing. In many cases, even when there is a huge number of states in the diagram, you could still extract at least a few

Re: [HACKERS] WIP: index support for regexp search

2012-11-30 Thread Alexander Korotkov
Hi! On Thu, Nov 29, 2012 at 12:58 PM, er e...@xs4all.nl wrote: On Mon, November 26, 2012 20:49, Alexander Korotkov wrote: trgm-regexp-0.6.patch.gz I ran the simple-minded tests against generated data (similar to the ones I did in January 2012). The problems of that older version seem

Re: [HACKERS] WIP: index support for regexp search

2012-11-30 Thread Alexander Korotkov
On Fri, Nov 30, 2012 at 3:20 PM, Alexander Korotkov aekorot...@gmail.comwrote: For depth-first it's not. Oh, I didn't explained it. In order to stop graph processing we need to be sure that we put all outgoing arcs from state or assume that state to be final. In DFS we can be in the final part

[HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Andres Freund
On 2012-11-30 08:52:18 +0530, Pavan Deolasee wrote: On Fri, Nov 30, 2012 at 3:19 AM, Andres Freund and...@2ndquadrant.comwrote: Hi, while looking at trigger.c from the POV of foreign key locks I noticed that GetTupleForTrigger commentless assumes it can just look at a pages content

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Markus Wanner
On 11/30/2012 11:09 AM, Dimitri Fontaine wrote: Markus Wanner mar...@bluegap.ch writes: However, as you say, maybe we need more coding examples. Maybe a minimally usable extra daemon example? Showing how to avoid common pitfalls? Use cases, anybody? :-) What about the PGQ ticker, pgqd?

Re: [HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Alvaro Herrera
Andres Freund escribió: On 2012-11-30 08:52:18 +0530, Pavan Deolasee wrote: On Fri, Nov 30, 2012 at 3:19 AM, Andres Freund and...@2ndquadrant.comwrote: I can't believe this is safe? Just about anything but eviction could happen to that buffer at that moment? Yeah, it does seem fishy

Re: [HACKERS] missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Pavan Deolasee
On Fri, Nov 30, 2012 at 5:28 PM, Andres Freund and...@2ndquadrant.comwrote: That seems to be safe to me. Anything thats been read above can't really change. The tuple is already locked, so a concurrent update/delete has to wait on us. We have a pin on the buffer, so VACUUM or HOT-prune

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Dimitri Fontaine
Markus Wanner mar...@bluegap.ch writes: AFAICS pgqd currently uses libpq, so I think it would rather turn into what I call a background worker, with a connection to Postgres shared memory. I perfectly well see use cases (plural!) for those. What I'm questioning is the use for what I rather

Re: [HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Simon Riggs
On 30 November 2012 11:58, Andres Freund and...@2ndquadrant.com wrote: We only get the pin right there, I don't see any preexisting pin. Seems easy enough to test with an Assert patch. If the Assert doesn't fail, we apply it as documentation of the requirement for a pin. If it fails, we fix

[HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Andres Freund
On 2012-11-30 18:09:49 +0530, Pavan Deolasee wrote: On Fri, Nov 30, 2012 at 5:28 PM, Andres Freund and...@2ndquadrant.comwrote: That seems to be safe to me. Anything thats been read above can't really change. The tuple is already locked, so a concurrent update/delete has to wait

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Alvaro Herrera
Dimitri Fontaine wrote: Markus Wanner mar...@bluegap.ch writes: AFAICS pgqd currently uses libpq, so I think it would rather turn into what I call a background worker, with a connection to Postgres shared memory. I perfectly well see use cases (plural!) for those. What I'm questioning

Re: [HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Andres Freund
On 2012-11-30 12:50:06 +, Simon Riggs wrote: On 30 November 2012 11:58, Andres Freund and...@2ndquadrant.com wrote: We only get the pin right there, I don't see any preexisting pin. Seems easy enough to test with an Assert patch. If the Assert doesn't fail, we apply it as documentation

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Andres Freund
On 2012-11-30 09:57:20 -0300, Alvaro Herrera wrote: Dimitri Fontaine wrote: Markus Wanner mar...@bluegap.ch writes: AFAICS pgqd currently uses libpq, so I think it would rather turn into what I call a background worker, with a connection to Postgres shared memory. I perfectly well see

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Dimitri Fontaine
Andres Freund and...@2ndquadrant.com writes: One of the uses for bgworkers that don't have shmem connection is to have them use libpq connections instead. I don't really see the point of forcing everyone to use backend connections when libpq connections are enough. In particular, they are

Re: [HACKERS] missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Pavan Deolasee
On Fri, Nov 30, 2012 at 6:21 PM, Andres Freund and...@2ndquadrant.comwrote: On 2012-11-30 18:09:49 +0530, Pavan Deolasee wrote: On Fri, Nov 30, 2012 at 5:28 PM, Andres Freund and...@2ndquadrant.com wrote: That seems to be safe to me. Anything thats been read above can't really

Re: [HACKERS] [PATCH] Patch to fix a crash of psql

2012-11-30 Thread Peter Eisentraut
On 11/30/12 3:26 AM, Albe Laurenz wrote: Tatsuo Ishii wrote: postgres=# \i ~/sql CREATE DATABASE You are now connected to database mydb as user t-ishii. CREATE SCHEMA psql:/home/t-ishii/sql:7: warning: possible conflict between client encoding SJIS and input file encoding CREATE TABLE

Re: [HACKERS] missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Pavan Deolasee
On Fri, Nov 30, 2012 at 5:28 PM, Andres Freund and...@2ndquadrant.comwrote: heap_fetch() though looks very similar has an important difference. It might be reading the tuple without lock on it and we need the buffer lock to protect against concurrent update/delete to the tuple. AFAIK once

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Kohei KaiGai
2012/11/30 Dimitri Fontaine dimi...@2ndquadrant.fr: Andres Freund and...@2ndquadrant.com writes: One of the uses for bgworkers that don't have shmem connection is to have them use libpq connections instead. I don't really see the point of forcing everyone to use backend connections when libpq

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Markus Wanner
On 11/30/2012 01:40 PM, Dimitri Fontaine wrote: I totally missed the need to connect to shared memory to be able to connect to a database and query it. Can't we just link the bgworkder code to the client libpq library, just as plproxy is doing I believe? Well, sure you can use libpq. But so

[HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Andres Freund
On 2012-11-30 18:35:05 +0530, Pavan Deolasee wrote: On Fri, Nov 30, 2012 at 6:21 PM, Andres Freund and...@2ndquadrant.comwrote: On 2012-11-30 18:09:49 +0530, Pavan Deolasee wrote: On Fri, Nov 30, 2012 at 5:28 PM, Andres Freund and...@2ndquadrant.com wrote: That seems to

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Markus Wanner
On 11/30/2012 01:59 PM, Andres Freund wrote: On 2012-11-30 09:57:20 -0300, Alvaro Herrera wrote: One of the uses for bgworkers that don't have shmem connection is to have them use libpq connections instead. I don't really see the point of forcing everyone to use backend connections when libpq

Re: [HACKERS] missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Pavan Deolasee
On Fri, Nov 30, 2012 at 6:55 PM, Andres Freund and...@2ndquadrant.comwrote: Hm? It doesn't move the page contents around but it moves the ItemId array during completely normal operation (c.f. needshuffle in PageAddItem). Or am I missing something? I think that probably only used for

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Dimitri Fontaine
Kohei KaiGai kai...@kaigai.gr.jp writes: One thing we have to pay attention is, the backend code cannot distinguish connection from pgworker via libpq from other regular connections, from perspective of access control. Even if we implement major portion with C-function, do we have a reliable

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Alvaro Herrera
Markus Wanner wrote: On 11/30/2012 01:40 PM, Dimitri Fontaine wrote: I totally missed the need to connect to shared memory to be able to connect to a database and query it. Can't we just link the bgworkder code to the client libpq library, just as plproxy is doing I believe? Well, sure

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Kohei KaiGai
2012/11/30 Dimitri Fontaine dimi...@2ndquadrant.fr: Kohei KaiGai kai...@kaigai.gr.jp writes: One thing we have to pay attention is, the backend code cannot distinguish connection from pgworker via libpq from other regular connections, from perspective of access control. Even if we implement

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Markus Wanner
Alvaro, On 11/30/2012 02:44 PM, Alvaro Herrera wrote: So it makes easier to have processes that need to run alongside postmaster. That's where we are in respectful disagreement, then. As I don't think that's easier, overall, but in my eye, this looks like a foot gun. As long as things like

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Markus Wanner
On 11/30/2012 03:16 PM, Kohei KaiGai wrote: This feature does not enforce them to implement with this new framework. If they can perform as separate daemons, it is fine enough. I'm not clear on what exactly you envision, but if a process needs access to shared buffers, it sounds like it should

Re: [HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Andres Freund
On 2012-11-30 13:57:46 +0100, Andres Freund wrote: On 2012-11-30 12:50:06 +, Simon Riggs wrote: On 30 November 2012 11:58, Andres Freund and...@2ndquadrant.com wrote: We only get the pin right there, I don't see any preexisting pin. Seems easy enough to test with an Assert patch.

Re: [HACKERS] json accessors

2012-11-30 Thread Andrew Dunstan
On 11/29/2012 06:34 PM, Merlin Moncure wrote: On Thu, Nov 29, 2012 at 4:14 PM, Andrew Dunstan and...@dunslane.net wrote: There are many things wrong with this. First, converting to hstore so you can call populate_record is quite horrible and ugly and inefficient. And it's dependent on having

Re: [HACKERS] json accessors

2012-11-30 Thread Merlin Moncure
On Fri, Nov 30, 2012 at 8:38 AM, Andrew Dunstan and...@dunslane.net wrote: OK, so based on this discussion, I'm thinking of the following: ok, this is looking awesome -- couple naming suggestions (see inline): * keep the original functions and operators. json_keys is still required for

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Kohei KaiGai
2012/11/30 Markus Wanner mar...@bluegap.ch: On 11/30/2012 03:16 PM, Kohei KaiGai wrote: This feature does not enforce them to implement with this new framework. If they can perform as separate daemons, it is fine enough. I'm not clear on what exactly you envision, but if a process needs

Re: [HACKERS] json accessors

2012-11-30 Thread Andrew Dunstan
On 11/30/2012 09:51 AM, Merlin Moncure wrote: Two questions: 1) is it possible for these to work without a polymorphic object passed through as hstore does (null::foo)? select populate_record(anyelement, record, json) I don't understand the question. The API I'm suggesting is exactly in

Re: [HACKERS] json accessors

2012-11-30 Thread Merlin Moncure
On Fri, Nov 30, 2012 at 9:02 AM, Andrew Dunstan and...@dunslane.net wrote: On 11/30/2012 09:51 AM, Merlin Moncure wrote: Two questions: 1) is it possible for these to work without a polymorphic object passed through as hstore does (null::foo)? select populate_record(anyelement, record,

Re: [HACKERS] json accessors

2012-11-30 Thread Hannu Krosing
On 11/30/2012 03:38 PM, Andrew Dunstan wrote: On 11/29/2012 06:34 PM, Merlin Moncure wrote: On Thu, Nov 29, 2012 at 4:14 PM, Andrew Dunstan and...@dunslane.net wrote: There are many things wrong with this. First, converting to hstore so you can call populate_record is quite horrible and ugly

Re: [HACKERS] [PATCH] Patch to fix a crash of psql

2012-11-30 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On 11/30/12 3:26 AM, Albe Laurenz wrote: If there is no possibility for false positives, I'd say that the possible should go. Maybe it should even be an error and no warning then. Yes, encoding mismatches are generally an error. I think the message

Re: [HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: But a failing Assert obviously proofs something. It doesn't prove that the code is unsafe though. I am suspicious that it is safe, because it's pretty darn hard to believe we'd not have seen field reports of problems with triggers otherwise. It's

Re: [HACKERS] json accessors

2012-11-30 Thread Hannu Krosing
On 11/30/2012 04:29 PM, Andrew Dunstan wrote: On 11/30/2012 10:04 AM, Hannu Krosing wrote: OK, so based on this discussion, I'm thinking of the following: * keep the original functions and operators. json_keys is still required for the case where the json is not flat. * json_each(json)

Re: [HACKERS] [PATCH] Patch to fix a crash of psql

2012-11-30 Thread Tatsuo Ishii
Peter Eisentraut pete...@gmx.net writes: On 11/30/12 3:26 AM, Albe Laurenz wrote: If there is no possibility for false positives, I'd say that the possible should go. Maybe it should even be an error and no warning then. Yes, encoding mismatches are generally an error. I think the

Re: [HACKERS] json accessors

2012-11-30 Thread Andrew Dunstan
On 11/30/2012 10:59 AM, Hannu Krosing wrote: Btw, how does current json type handle code pages - is json always utf-8 even when server encoding is not ? if so then we could at least have a shortcut conversion of json to utf8-text which can skip codepage changes. IIRC json is stored

Re: [HACKERS] [PATCH] Patch to fix a crash of psql

2012-11-30 Thread Tatsuo Ishii
I wonder about the possible. Could there be false positives with the test? If yes, I don't like the idea. Yes, there could be false positives. It was just because the input file was broken. In the real world, I think probably encoding mismatch is the most possible cause, but this is not

Re: [HACKERS] [PATCH] Patch to fix a crash of psql

2012-11-30 Thread Tatsuo Ishii
I think the message should be more precise. Nobody will know what an encoding conflict is. The error condition is last multibyte character ran over end of file or something like that, which should be clearer. last multibyte character ran over is not precise at all because the error was

Re: [HACKERS] WIP json generation enhancements

2012-11-30 Thread Andrew Dunstan
On 11/27/2012 02:38 PM, Andrew Dunstan wrote: On 11/26/2012 12:31 PM, Robert Haas wrote: On Mon, Nov 26, 2012 at 11:43 AM, Andrew Dunstan and...@dunslane.net wrote: I don't understand why you would want to create such a cast. If the cast doesn't exist it will do exactly what it does now,

Re: [HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2012-11-30 10:42:21 -0500, Tom Lane wrote: I am suspicious that it is safe, because it's pretty darn hard to believe we'd not have seen field reports of problems with triggers otherwise. It's not like this is a seldom-executed code path. Thats

Re: [HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Andres Freund
On 2012-11-30 12:53:27 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2012-11-30 10:42:21 -0500, Tom Lane wrote: I am suspicious that it is safe, because it's pretty darn hard to believe we'd not have seen field reports of problems with triggers otherwise. It's

[HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Andres Freund
Hi, The subject says it all. There are workloads where its detrimental, but in general having it default to on improver experience tremendously because getting conflicts because of vacuum is rather confusing. In the workloads where it might not be a good idea (very long queries on the standby,

Re: [HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Tom Lane
BTW, I went looking for other places that might have a similar mistake. I found that contrib/pageinspect/btreefuncs.c pokes around in btree pages without any buffer lock, which seems pretty broken --- don't we want it to take share lock? regards, tom lane -- Sent via

[HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Andres Freund
On 2012-11-30 14:08:05 -0500, Tom Lane wrote: BTW, I went looking for other places that might have a similar mistake. I found that contrib/pageinspect/btreefuncs.c pokes around in btree pages without any buffer lock, which seems pretty broken --- don't we want it to take share lock? I seem to

Re: [HACKERS] missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2012-11-30 14:08:05 -0500, Tom Lane wrote: BTW, I went looking for other places that might have a similar mistake. I found that contrib/pageinspect/btreefuncs.c pokes around in btree pages without any buffer lock, which seems pretty broken ---

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Simon Riggs
On 30 November 2012 19:02, Andres Freund and...@2ndquadrant.com wrote: The subject says it all. There are workloads where its detrimental, but in general having it default to on improver experience tremendously because getting conflicts because of vacuum is rather confusing. In the

Re: [HACKERS] pg_basebackup is taking backup of extra files inside a tablespace directory

2012-11-30 Thread Robert Haas
On Wed, Nov 28, 2012 at 1:55 AM, Hari Babu haribabu.ko...@huawei.com wrote: I think backup should be done only files and folders present inside '/opt/tblspc/PG_*' directory (TABLESPACE_VERSION_DIRECTORY). Not all the files and folders in side '/opt/tblspc.' directory. I think I agree. The

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Robert Haas
On Fri, Nov 30, 2012 at 2:02 PM, Andres Freund and...@2ndquadrant.com wrote: Does anybody have an argument against changing the default value? Well, the disadvantage of it is that the standby can bloat the master, which might be surprising to some people, too. But I don't really have a lot of

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-30 Thread Robert Haas
On Wed, Nov 28, 2012 at 9:47 AM, Amit kapila amit.kap...@huawei.com wrote: 5. PERSISTENT Keyword is added to the reserved keyword list. As it was giving some errors given below while parsing gram.y 15 shift/reduce conflicts . Allow me to be the first to say that any syntax for this

Re: [HACKERS] [PATCH] Patch to fix a crash of psql

2012-11-30 Thread Tom Lane
Tatsuo Ishii is...@postgresql.org writes: TBH I think that a message here is unnecessary; it's sufficient to ensure psql doesn't crash. The backend will produce a better message than this anyway once the data gets there, and that way we don't have to invent a new error recovery path inside

Re: [HACKERS] [PATCH] make -jN check fails / unset MAKEFLAGS in pg_regress.c

2012-11-30 Thread Robert Haas
On Thu, Nov 29, 2012 at 8:50 AM, Andres Freund and...@2ndquadrant.com wrote: Hi, Currently make -jN check fails during creating temporary installation with: make[1]: *** read jobs pipe: Invalid argument. Stop. make[1]: *** Waiting for unfinished jobs make[2]: warning: jobserver

Re: [HACKERS] initdb.c::main() too large

2012-11-30 Thread Bruce Momjian
On Thu, Nov 29, 2012 at 11:23:59PM -0500, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: In looking to add an fsync-only option to initdb, I found its main() function to be 743 lines long, and very hard to understand. The attached patch moves much of that code into separate

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-30 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Nov 28, 2012 at 9:47 AM, Amit kapila amit.kap...@huawei.com wrote: 5. PERSISTENT Keyword is added to the reserved keyword list. As it was giving some errors given below while parsing gram.y 15 shift/reduce conflicts . Allow me to be the

Re: [HACKERS] Enabling frontend-only xlog desc routines

2012-11-30 Thread Robert Haas
On Thu, Nov 29, 2012 at 3:03 PM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: The other interesting question remaining is what to do about the rm_desc function in rmgr.c. We're split between these two ideas: Why try to link rmgr.c into frontend versions

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Josh Berkus
In the workloads where it might not be a good idea (very long queries on the standby, many dead tuples on the primary) you need to think very carefuly about the strategy of avoiding conflicts anyway, and explicit configuration is required as well. Does anybody have an argument against

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Heikki Linnakangas
On 30.11.2012 21:02, Andres Freund wrote: Hi, The subject says it all. There are workloads where its detrimental, but in general having it default to on improver experience tremendously because getting conflicts because of vacuum is rather confusing. In the workloads where it might not be a

Re: [HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Tom Lane
BTW, on further inspection, there's yet another reason why we've not heard about this from the field: even if all the wrong things happen and GetTupleForTrigger manages to copy bogus data for the old tuple, that data *is not passed to the trigger function*. The only thing we do with it is decide

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Claudio Freire
On Fri, Nov 30, 2012 at 5:46 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Think of someone setting up a test server, by setting it up as a standby from the master. Now, when someone holds a transaction open in the test server, you get bloat in the master. Or if you set up a standby

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: While we're talking about changing defaults, how about changing the default value of the recovery.conf parameter 'standby_mode' to on? Not sure about anybody else, but I never want it any other way. Dunno, it's been only a couple of days since there

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Heikki Linnakangas
On 30.11.2012 22:49, Claudio Freire wrote: On Fri, Nov 30, 2012 at 5:46 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Think of someone setting up a test server, by setting it up as a standby from the master. Now, when someone holds a transaction open in the test server, you get bloat

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Kevin Grittner
Claudio Freire wrote: Without hot standby feedback, reporting queries are impossible. I've experienced it. Cancellations make it impossible to finish any decently complex reporting query. With what setting of max_standby_streaming_delay? I would rather default that to -1 than default

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Claudio Freire
On Fri, Nov 30, 2012 at 6:06 PM, Kevin Grittner kgri...@mail.com wrote: Without hot standby feedback, reporting queries are impossible. I've experienced it. Cancellations make it impossible to finish any decently complex reporting query. With what setting of max_standby_streaming_delay? I

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Tom Lane
Claudio Freire klaussfre...@gmail.com writes: Without hot standby feedback, reporting queries are impossible. I've experienced it. Cancellations make it impossible to finish any decently complex reporting query. The original expectation was that slave-side cancels would be infrequent. Maybe

Re: [HACKERS] Removing PD_ALL_VISIBLE

2012-11-30 Thread Jeff Davis
On Mon, 2012-11-26 at 16:55 -0600, Merlin Moncure wrote: Based on previous measurements, I think there *is* contention pinning the root of an index. Currently, I believe it's largely overwhelmed by contention from other sources, such as the buffer manager lwlocks and the very-evil

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Kevin Grittner
Claudio Freire wrote: With what setting of max_standby_streaming_delay? I would rather default that to -1 than default hot_standby_feedback on. That way what you do on the standby only affects the standby. 1d Was there actually a transaction hanging open for an entire day on the standby?

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-30 Thread Bruce Momjian
On Thu, Nov 29, 2012 at 12:59:19PM -0500, Bruce Momjian wrote: I have polished up the patch (attached) and it is ready for application to 9.3. Applied. --- Since there is no pg_dump/pg_restore pipe parallelism, I had the

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Claudio Freire
On Fri, Nov 30, 2012 at 6:20 PM, Kevin Grittner kgri...@mail.com wrote: Claudio Freire wrote: With what setting of max_standby_streaming_delay? I would rather default that to -1 than default hot_standby_feedback on. That way what you do on the standby only affects the standby. 1d Was

Re: [HACKERS] initdb.c::main() too large

2012-11-30 Thread Bruce Momjian
On Thu, Nov 29, 2012 at 11:12:23PM -0500, Bruce Momjian wrote: In looking to add an fsync-only option to initdb, I found its main() function to be 743 lines long, and very hard to understand. The attached patch moves much of that code into separate functions, which will make initdb.c easier

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Heikki Linnakangas
On 30.11.2012 23:40, Claudio Freire wrote: On Fri, Nov 30, 2012 at 6:20 PM, Kevin Grittnerkgri...@mail.com wrote: Claudio Freire wrote: With what setting of max_standby_streaming_delay? I would rather default that to -1 than default hot_standby_feedback on. That way what you do on the

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Claudio Freire
On Fri, Nov 30, 2012 at 6:49 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I have most certainly managed databases where holding up vacuuming on the source would cripple performance to the point that users would have demanded that any other process causing it must be immediately

Re: [HACKERS] initdb.c::main() too large

2012-11-30 Thread Andrew Dunstan
On 11/30/2012 04:45 PM, Bruce Momjian wrote: On Thu, Nov 29, 2012 at 11:12:23PM -0500, Bruce Momjian wrote: In looking to add an fsync-only option to initdb, I found its main() function to be 743 lines long, and very hard to understand. The attached patch moves much of that code into separate

Re: [HACKERS] initdb.c::main() too large

2012-11-30 Thread Bruce Momjian
On Fri, Nov 30, 2012 at 06:06:39PM -0500, Andrew Dunstan wrote: On 11/30/2012 04:45 PM, Bruce Momjian wrote: On Thu, Nov 29, 2012 at 11:12:23PM -0500, Bruce Momjian wrote: In looking to add an fsync-only option to initdb, I found its main() function to be 743 lines long, and very hard to

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Andres Freund
On 2012-11-30 14:35:37 -0500, Robert Haas wrote: On Fri, Nov 30, 2012 at 2:02 PM, Andres Freund and...@2ndquadrant.com wrote: Does anybody have an argument against changing the default value? Well, the disadvantage of it is that the standby can bloat the master, which might be surprising to

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Daniel Farina
On Fri, Nov 30, 2012 at 11:35 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Nov 30, 2012 at 2:02 PM, Andres Freund and...@2ndquadrant.com wrote: Does anybody have an argument against changing the default value? Well, the disadvantage of it is that the standby can bloat the master,

Re: [HACKERS] [PATCH] make -jN check fails / unset MAKEFLAGS in pg_regress.c

2012-11-30 Thread Andres Freund
On 2012-11-30 14:41:14 -0500, Robert Haas wrote: On Thu, Nov 29, 2012 at 8:50 AM, Andres Freund and...@2ndquadrant.com wrote: Hi, Currently make -jN check fails during creating temporary installation with: make[1]: *** read jobs pipe: Invalid argument. Stop. make[1]: *** Waiting for

Re: [HACKERS] [PATCH] make -jN check fails / unset MAKEFLAGS in pg_regress.c

2012-11-30 Thread Andres Freund
Yes. obviously. Stupid, errr, fingers. This time round it really was fatfingering the wrong key after having been climbing for 4h. Really. Trivially updated patch attached. -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Magnus Hagander
On Fri, Nov 30, 2012 at 9:46 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 30.11.2012 21:02, Andres Freund wrote: Hi, The subject says it all. There are workloads where its detrimental, but in general having it default to on improver experience tremendously because getting

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Magnus Hagander
On Fri, Nov 30, 2012 at 10:09 PM, Tom Lane t...@sss.pgh.pa.us wrote: Claudio Freire klaussfre...@gmail.com writes: Without hot standby feedback, reporting queries are impossible. I've experienced it. Cancellations make it impossible to finish any decently complex reporting query. The

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Andres Freund
On 2012-11-30 22:46:06 +0200, Heikki Linnakangas wrote: On 30.11.2012 21:02, Andres Freund wrote: Hi, The subject says it all. There are workloads where its detrimental, but in general having it default to on improver experience tremendously because getting conflicts because of vacuum is

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Josh Berkus
All: Well, the problem is that we have three configurations which only work for one very common scenario: - reporting slave: feedback off, very long replication_delay - load-balancing slave: feedback on, short replication_delay - backup/failover slave: feedback off, short replication_delay I

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Andres Freund
On 2012-11-30 16:09:15 -0500, Tom Lane wrote: Claudio Freire klaussfre...@gmail.com writes: Without hot standby feedback, reporting queries are impossible. I've experienced it. Cancellations make it impossible to finish any decently complex reporting query. The original expectation was

Re: Use of fsync; was Re: [HACKERS] Pg_upgrade speed for many tables

2012-11-30 Thread Bruce Momjian
On Mon, Nov 26, 2012 at 02:43:19PM -0500, Bruce Momjian wrote: In any event, I think the documentation should caution that the upgrade should not be deemed to be a success until after a system-wide sync has been done. Even if we use the link rather than copy method, are we sure that

[HACKERS] --single-transaction hack to pg_upgrade does not work

2012-11-30 Thread Tom Lane
Some of the buildfarm members are failing the pg_upgrade regression test since commit 12ee6ec71f8754ff3573711032b9b4d5a764ba84. I can duplicate it here, and the symptom is: pg_restore: creating TYPE float8range pg_restore: creating TYPE insenum pg_restore: [archiver (db)] Error while PROCESSING