Re: [HACKERS] [PATCH] Patch to fix a crash of psql

2012-11-30 Thread Albe Laurenz
Tatsuo Ishii wrote: > I think we should detect the cases as much as possible and warn users, > rather than silently ignore that fact client encoding != file > encoding. I don't think we can detect it in a reliable way, but at > least we could check the cases above(sum of PQmblen is not equale to >

Re: [HACKERS] DEALLOCATE IF EXISTS

2012-11-30 Thread Vik Reykja
On Tue, Nov 27, 2012 at 3:15 PM, Heikki Linnakangas wrote: > I fail to see the point of DEALLOCATE IF EXISTS. Do you have real use case > for this, or was this just a case of adding IF EXISTS to all commands for > the sake of completeness? > > Usually the client knows what statements have been pr

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Dimitri Fontaine
Markus Wanner writes: >> However, as you say, maybe we need more coding examples. > > Maybe a minimally usable extra daemon example? Showing how to avoid > common pitfalls? Use cases, anybody? :-) What about the PGQ ticker, pgqd? https://github.com/markokr/skytools/tree/master/sql/ticker htt

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Pavel Stehule
2012/11/30 Dimitri Fontaine : > Markus Wanner writes: >>> However, as you say, maybe we need more coding examples. >> >> Maybe a minimally usable extra daemon example? Showing how to avoid >> common pitfalls? Use cases, anybody? :-) > > What about the PGQ ticker, pgqd? > > https://github.com/mar

Re: [HACKERS] proposal: fix corner use case of variadic fuctions usage

2012-11-30 Thread Vik Reykja
On Sun, Nov 4, 2012 at 12:49 PM, Pavel Stehule wrote: > Hello > > here is patch, that enables using a variadic parameter modifier for > variadic "any" function. > > Motivation for this patch is consistent behave of "format" function, > but it fixes behave of all this class variadic functions. > >

[HACKERS] Review: create extension default_full_version

2012-11-30 Thread Ibrar Ahmed
Hi, I looked at the discussion for this patch and the patch itself. Here are my comments and observations about the patch. What I got from the discussion is that the patch tries to implement a mechanism to install extension from series of SQL scripts from base/full version e.g. if a user wants to

Re: [HACKERS] Refactoring standby mode logic

2012-11-30 Thread Dimitri Fontaine
Hi, Heikki Linnakangas writes: > Attached is a patch to refactor that logic into a more straightforward state > machine. It's always been a kind of a state machine, but it's been hard to > see, as the code wasn't explicitly written that way. Any objections? On a quick glance over, looks good to

Re: [HACKERS] WIP: index support for regexp search

2012-11-30 Thread Alexander Korotkov
On Thu, Nov 29, 2012 at 5:25 PM, Heikki Linnakangas wrote: > One thing that bothers me with this algoritm is that the overflow > mechanism is all-or-nothing. In many cases, even when there is a huge > number of states in the diagram, you could still extract at least a few > trigrams that must be

Re: [HACKERS] WIP: index support for regexp search

2012-11-30 Thread Alexander Korotkov
Hi! On Thu, Nov 29, 2012 at 12:58 PM, er wrote: > On Mon, November 26, 2012 20:49, Alexander Korotkov wrote: > > > trgm-regexp-0.6.patch.gz > > I ran the simple-minded tests against generated data (similar to the ones > I did in January 2012). > The problems of that older version seem pretty muc

Re: [HACKERS] WIP: index support for regexp search

2012-11-30 Thread Alexander Korotkov
On Fri, Nov 30, 2012 at 3:20 PM, Alexander Korotkov wrote: > For depth-first it's not. > Oh, I didn't explained it. In order to stop graph processing we need to be sure that we put all outgoing arcs from state or assume that state to be final. In DFS we can be in the final part of graph producing

[HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Andres Freund
On 2012-11-30 08:52:18 +0530, Pavan Deolasee wrote: > On Fri, Nov 30, 2012 at 3:19 AM, Andres Freund wrote: > > > Hi, > > > > while looking at trigger.c from the POV of foreign key locks I noticed > > that GetTupleForTrigger commentless assumes it can just look at a pages > > content without a > >

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Markus Wanner
On 11/30/2012 11:09 AM, Dimitri Fontaine wrote: > Markus Wanner writes: >>> However, as you say, maybe we need more coding examples. >> >> Maybe a minimally usable extra daemon example? Showing how to avoid >> common pitfalls? Use cases, anybody? :-) > > What about the PGQ ticker, pgqd? > > ht

Re: [HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Alvaro Herrera
Andres Freund escribió: > On 2012-11-30 08:52:18 +0530, Pavan Deolasee wrote: > > On Fri, Nov 30, 2012 at 3:19 AM, Andres Freund > > wrote: > > > I can't believe this is safe? Just about anything but eviction could > > > happen to that buffer at that moment? Yeah, it does seem fishy to me as wel

Re: [HACKERS] missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Pavan Deolasee
On Fri, Nov 30, 2012 at 5:28 PM, Andres Freund wrote: > > > > > > That seems to be safe to me. Anything thats been read above can't really > > change. The tuple is already locked, so a concurrent update/delete has to > > wait on us. We have a pin on the buffer, so VACUUM or HOT-prune can't > > hap

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Dimitri Fontaine
Markus Wanner writes: > AFAICS pgqd currently uses libpq, so I think it would rather turn into > what I call a background worker, with a connection to Postgres shared > memory. I perfectly well see use cases (plural!) for those. > > What I'm questioning is the use for what I rather call "extra dae

Re: [HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Simon Riggs
On 30 November 2012 11:58, Andres Freund wrote: > We only get the pin right there, I don't see any preexisting pin. Seems easy enough to test with an Assert patch. If the Assert doesn't fail, we apply it as "documentation" of the requirement for a pin. If it fails, we fix the bug. -- Simon

[HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Andres Freund
On 2012-11-30 18:09:49 +0530, Pavan Deolasee wrote: > On Fri, Nov 30, 2012 at 5:28 PM, Andres Freund wrote: > > > > > > > > > > That seems to be safe to me. Anything thats been read above can't really > > > change. The tuple is already locked, so a concurrent update/delete has to > > > wait on us.

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Alvaro Herrera
Dimitri Fontaine wrote: > Markus Wanner writes: > > AFAICS pgqd currently uses libpq, so I think it would rather turn into > > what I call a background worker, with a connection to Postgres shared > > memory. I perfectly well see use cases (plural!) for those. > > > > What I'm questioning is the u

Re: [HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Andres Freund
On 2012-11-30 12:50:06 +, Simon Riggs wrote: > On 30 November 2012 11:58, Andres Freund wrote: > > > We only get the pin right there, I don't see any preexisting pin. > > Seems easy enough to test with an Assert patch. > > If the Assert doesn't fail, we apply it as "documentation" of the > req

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Andres Freund
On 2012-11-30 09:57:20 -0300, Alvaro Herrera wrote: > Dimitri Fontaine wrote: > > Markus Wanner writes: > > > AFAICS pgqd currently uses libpq, so I think it would rather turn into > > > what I call a background worker, with a connection to Postgres shared > > > memory. I perfectly well see use ca

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Dimitri Fontaine
Andres Freund writes: >> One of the uses for bgworkers that don't have shmem connection is to >> have them use libpq connections instead. I don't really see the point >> of forcing everyone to use backend connections when libpq connections >> are enough. In particular, they are easier to port fr

Re: [HACKERS] missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Pavan Deolasee
On Fri, Nov 30, 2012 at 6:21 PM, Andres Freund wrote: > On 2012-11-30 18:09:49 +0530, Pavan Deolasee wrote: > > On Fri, Nov 30, 2012 at 5:28 PM, Andres Freund >wrote: > > > > > > > > > > > > > > That seems to be safe to me. Anything thats been read above can't > really > > > > change. The tuple i

Re: [HACKERS] [PATCH] Patch to fix a crash of psql

2012-11-30 Thread Peter Eisentraut
On 11/30/12 3:26 AM, Albe Laurenz wrote: > Tatsuo Ishii wrote: >> postgres=# \i ~/sql >> CREATE DATABASE >> You are now connected to database "mydb" as user "t-ishii". >> CREATE SCHEMA >> psql:/home/t-ishii/sql:7: warning: possible conflict between client > encoding SJIS and input file >> encoding

Re: [HACKERS] missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Pavan Deolasee
On Fri, Nov 30, 2012 at 5:28 PM, Andres Freund wrote: > > > > heap_fetch() though looks very similar has an important difference. It > > might be reading the tuple without lock on it and we need the buffer lock > > to protect against concurrent update/delete to the tuple. AFAIK once the > > tuple

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Kohei KaiGai
2012/11/30 Dimitri Fontaine : > Andres Freund writes: >>> One of the uses for bgworkers that don't have shmem connection is to >>> have them use libpq connections instead. I don't really see the point >>> of forcing everyone to use backend connections when libpq connections >>> are enough. In pa

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Markus Wanner
On 11/30/2012 01:40 PM, Dimitri Fontaine wrote: > I totally missed the need to connect to shared memory to be able to > connect to a database and query it. Can't we just link the bgworkder > code to the client libpq library, just as plproxy is doing I believe? Well, sure you can use libpq. But so

[HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Andres Freund
On 2012-11-30 18:35:05 +0530, Pavan Deolasee wrote: > On Fri, Nov 30, 2012 at 6:21 PM, Andres Freund wrote: > > > On 2012-11-30 18:09:49 +0530, Pavan Deolasee wrote: > > > On Fri, Nov 30, 2012 at 5:28 PM, Andres Freund > >wrote: > > > > > > > > > > > > > > > > > > That seems to be safe to me. Anyt

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Markus Wanner
On 11/30/2012 01:59 PM, Andres Freund wrote: > On 2012-11-30 09:57:20 -0300, Alvaro Herrera wrote: >> One of the uses for bgworkers that don't have shmem connection is to >> have them use libpq connections instead. I don't really see the point >> of forcing everyone to use backend connections when

Re: [HACKERS] missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Pavan Deolasee
On Fri, Nov 30, 2012 at 6:55 PM, Andres Freund wrote: > > > Hm? It doesn't move the page contents around but it moves the ItemId > array during completely normal operation (c.f. needshuffle in > PageAddItem). Or am I missing something? > > I think that probably only used for non-heap pages. For he

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Dimitri Fontaine
Kohei KaiGai writes: > One thing we have to pay attention is, the backend code cannot distinguish > connection from pgworker via libpq from other regular connections, from > perspective of access control. > Even if we implement major portion with C-function, do we have a reliable way > to prohibit

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Alvaro Herrera
Markus Wanner wrote: > On 11/30/2012 01:40 PM, Dimitri Fontaine wrote: > > I totally missed the need to connect to shared memory to be able to > > connect to a database and query it. Can't we just link the bgworkder > > code to the client libpq library, just as plproxy is doing I believe? > > Well

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Kohei KaiGai
2012/11/30 Dimitri Fontaine : > Kohei KaiGai writes: >> One thing we have to pay attention is, the backend code cannot distinguish >> connection from pgworker via libpq from other regular connections, from >> perspective of access control. >> Even if we implement major portion with C-function, do

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Markus Wanner
Alvaro, On 11/30/2012 02:44 PM, Alvaro Herrera wrote: > So it > makes easier to have processes that need to run alongside postmaster. That's where we are in respectful disagreement, then. As I don't think that's easier, overall, but in my eye, this looks like a foot gun. As long as things like p

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Kohei KaiGai
2012/11/30 Markus Wanner : > Alvaro, > > On 11/30/2012 02:44 PM, Alvaro Herrera wrote: >> So it >> makes easier to have processes that need to run alongside postmaster. > > That's where we are in respectful disagreement, then. As I don't think > that's easier, overall, but in my eye, this looks lik

Re: [HACKERS] WIP: index support for regexp search

2012-11-30 Thread Heikki Linnakangas
On 30.11.2012 13:20, Alexander Korotkov wrote: On Thu, Nov 29, 2012 at 5:25 PM, Heikki Linnakangas wrote: Would it be safe to simply stop short the depth-first search on overflow, and proceed with the graph that was constructed up to that point? For depth-first it's not. But your proposal na

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Markus Wanner
On 11/30/2012 03:16 PM, Kohei KaiGai wrote: > This feature does not enforce them to implement with this new framework. > If they can perform as separate daemons, it is fine enough. I'm not clear on what exactly you envision, but if a process needs access to shared buffers, it sounds like it should

Re: [HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Andres Freund
On 2012-11-30 13:57:46 +0100, Andres Freund wrote: > On 2012-11-30 12:50:06 +, Simon Riggs wrote: > > On 30 November 2012 11:58, Andres Freund wrote: > > > > > We only get the pin right there, I don't see any preexisting pin. > > > > Seems easy enough to test with an Assert patch. > > > > If t

Re: [HACKERS] json accessors

2012-11-30 Thread Andrew Dunstan
On 11/29/2012 06:34 PM, Merlin Moncure wrote: On Thu, Nov 29, 2012 at 4:14 PM, Andrew Dunstan wrote: There are many things wrong with this. First, converting to hstore so you can call populate_record is quite horrible and ugly and inefficient. And it's dependent on having hstore loaded - you c

Re: [HACKERS] json accessors

2012-11-30 Thread Merlin Moncure
On Fri, Nov 30, 2012 at 8:38 AM, Andrew Dunstan wrote: > OK, so based on this discussion, I'm thinking of the following: ok, this is looking awesome -- couple naming suggestions (see inline): > * keep the original functions and operators. json_keys is still >required for the case where the

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Kohei KaiGai
2012/11/30 Markus Wanner : > On 11/30/2012 03:16 PM, Kohei KaiGai wrote: >> This feature does not enforce them to implement with this new framework. >> If they can perform as separate daemons, it is fine enough. > > I'm not clear on what exactly you envision, but if a process needs > access to shar

Re: [HACKERS] json accessors

2012-11-30 Thread Andrew Dunstan
On 11/30/2012 09:51 AM, Merlin Moncure wrote: Two questions: 1) is it possible for these to work without a polymorphic object passed through as hstore does (null::foo)? select populate_record(anyelement, record, json) I don't understand the question. The API I'm suggesting is exactly in lin

Re: [HACKERS] json accessors

2012-11-30 Thread Merlin Moncure
On Fri, Nov 30, 2012 at 9:02 AM, Andrew Dunstan wrote: > > On 11/30/2012 09:51 AM, Merlin Moncure wrote: >> >> >> Two questions: >> 1) is it possible for these to work without a polymorphic object >> passed through as hstore does (null::foo)? >> select populate_record(anyelement, record, json) >

Re: [HACKERS] json accessors

2012-11-30 Thread Hannu Krosing
On 11/30/2012 03:38 PM, Andrew Dunstan wrote: On 11/29/2012 06:34 PM, Merlin Moncure wrote: On Thu, Nov 29, 2012 at 4:14 PM, Andrew Dunstan wrote: There are many things wrong with this. First, converting to hstore so you can call populate_record is quite horrible and ugly and inefficient. An

Re: [HACKERS] Review: Extra Daemons / bgworker

2012-11-30 Thread Markus Wanner
On 11/30/2012 03:58 PM, Kohei KaiGai wrote: > It seemed to me you are advocating that any use case of background- > worker can be implemented with existing separate daemon approach. That sounds like a misunderstanding. All I'm advocating is that only 3rd-party processes with a real need (like acce

Re: [HACKERS] json accessors

2012-11-30 Thread Andrew Dunstan
On 11/30/2012 10:04 AM, Hannu Krosing wrote: OK, so based on this discussion, I'm thinking of the following: * keep the original functions and operators. json_keys is still required for the case where the json is not flat. * json_each(json) => setof (text, text) errors if the json is

Re: [HACKERS] [PATCH] Patch to fix a crash of psql

2012-11-30 Thread Tom Lane
Peter Eisentraut writes: > On 11/30/12 3:26 AM, Albe Laurenz wrote: >> If there is no possibility for false positives, I'd say >> that the "possible" should go. Maybe it should even be >> an error and no warning then. > Yes, encoding mismatches are generally an error. > I think the message shou

Re: [HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Tom Lane
Andres Freund writes: > But a failing Assert obviously proofs something. It doesn't prove that the code is unsafe though. I am suspicious that it is safe, because it's pretty darn hard to believe we'd not have seen field reports of problems with triggers otherwise. It's not like this is a seldo

[HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Andres Freund
On 2012-11-30 10:42:21 -0500, Tom Lane wrote: > Andres Freund writes: > > But a failing Assert obviously proofs something. > > It doesn't prove that the code is unsafe though. Hehe. > I am suspicious that it is safe, because it's pretty darn hard to > believe we'd not have seen field reports of

Re: [HACKERS] json accessors

2012-11-30 Thread Hannu Krosing
On 11/30/2012 04:29 PM, Andrew Dunstan wrote: On 11/30/2012 10:04 AM, Hannu Krosing wrote: OK, so based on this discussion, I'm thinking of the following: * keep the original functions and operators. json_keys is still required for the case where the json is not flat. * json_each(json)

Re: [HACKERS] [PATCH] Patch to fix a crash of psql

2012-11-30 Thread Tatsuo Ishii
> Peter Eisentraut writes: >> On 11/30/12 3:26 AM, Albe Laurenz wrote: >>> If there is no possibility for false positives, I'd say >>> that the "possible" should go. Maybe it should even be >>> an error and no warning then. > >> Yes, encoding mismatches are generally an error. > >> I think the

Re: [HACKERS] json accessors

2012-11-30 Thread Andrew Dunstan
On 11/30/2012 10:59 AM, Hannu Krosing wrote: Btw, how does current json type handle code pages - is json always utf-8 even when server encoding is not ? if so then we could at least have a shortcut conversion of json to utf8-text which can skip codepage changes. IIRC json is stored and

Re: [HACKERS] [PATCH] Patch to fix a crash of psql

2012-11-30 Thread Tatsuo Ishii
> I wonder about the "possible". > > Could there be false positives with the test? > If yes, I don't like the idea. Yes, there could be false positives. It was just because the input file was broken. In the real world, I think probably encoding mismatch is the most possible cause, but this is not

Re: [HACKERS] [PATCH] Patch to fix a crash of psql

2012-11-30 Thread Tatsuo Ishii
> I think the message should be more precise. Nobody will know what an > "encoding conflict" is. The error condition is "last multibyte > character ran over end of file" or something like that, which should be > clearer. "last multibyte character ran over" is not precise at all because the error

Re: [HACKERS] WIP json generation enhancements

2012-11-30 Thread Andrew Dunstan
On 11/27/2012 02:38 PM, Andrew Dunstan wrote: On 11/26/2012 12:31 PM, Robert Haas wrote: On Mon, Nov 26, 2012 at 11:43 AM, Andrew Dunstan wrote: I don't understand why you would want to create such a cast. If the cast doesn't exist it will do exactly what it does now, i.e. use the type's ou

Re: [HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Tom Lane
Andres Freund writes: > On 2012-11-30 10:42:21 -0500, Tom Lane wrote: >> I am suspicious that it is safe, because it's pretty darn hard to >> believe we'd not have seen field reports of problems with triggers >> otherwise. It's not like this is a seldom-executed code path. > Thats true, but I th

Re: [HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Andres Freund
On 2012-11-30 12:53:27 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2012-11-30 10:42:21 -0500, Tom Lane wrote: > >> I am suspicious that it is safe, because it's pretty darn hard to > >> believe we'd not have seen field reports of problems with triggers > >> otherwise. It's not like this

[HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Andres Freund
Hi, The subject says it all. There are workloads where its detrimental, but in general having it default to on improver experience tremendously because getting conflicts because of vacuum is rather confusing. In the workloads where it might not be a good idea (very long queries on the standby, m

Re: [HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Tom Lane
BTW, I went looking for other places that might have a similar mistake. I found that contrib/pageinspect/btreefuncs.c pokes around in btree pages without any buffer lock, which seems pretty broken --- don't we want it to take share lock? regards, tom lane -- Sent via pgs

[HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Andres Freund
On 2012-11-30 14:08:05 -0500, Tom Lane wrote: > BTW, I went looking for other places that might have a similar mistake. > I found that contrib/pageinspect/btreefuncs.c pokes around in btree > pages without any buffer lock, which seems pretty broken --- don't we > want it to take share lock? I seem

Re: [HACKERS] missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Tom Lane
Andres Freund writes: > On 2012-11-30 14:08:05 -0500, Tom Lane wrote: >> BTW, I went looking for other places that might have a similar mistake. >> I found that contrib/pageinspect/btreefuncs.c pokes around in btree >> pages without any buffer lock, which seems pretty broken --- don't we >> want i

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Simon Riggs
On 30 November 2012 19:02, Andres Freund wrote: > The subject says it all. > > There are workloads where its detrimental, but in general having it > default to on improver experience tremendously because getting conflicts > because of vacuum is rather confusing. > > In the workloads where it migh

Re: [HACKERS] pg_basebackup is taking backup of extra files inside a tablespace directory

2012-11-30 Thread Robert Haas
On Wed, Nov 28, 2012 at 1:55 AM, Hari Babu wrote: > I think backup should be done only files and folders present inside > '/opt/tblspc/PG_*' directory (TABLESPACE_VERSION_DIRECTORY). > Not all the files and folders in side '/opt/tblspc.' directory. I think I agree. The current behavior would hav

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Robert Haas
On Fri, Nov 30, 2012 at 2:02 PM, Andres Freund wrote: > Does anybody have an argument against changing the default value? Well, the disadvantage of it is that the standby can bloat the master, which might be surprising to some people, too. But I don't really have a lot of skin in this game. Whi

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-30 Thread Robert Haas
On Wed, Nov 28, 2012 at 9:47 AM, Amit kapila wrote: > 5. PERSISTENT Keyword is added to the reserved keyword list. As it was giving > some errors given below while parsing gram.y > 15 shift/reduce conflicts . Allow me to be the first to say that any syntax for this feature that involve

Re: [HACKERS] [PATCH] Patch to fix a crash of psql

2012-11-30 Thread Tom Lane
Tatsuo Ishii writes: >> TBH I think that a message here is unnecessary; it's sufficient to >> ensure psql doesn't crash. The backend will produce a better message >> than this anyway once the data gets there, and that way we don't have to >> invent a new error recovery path inside psql. As-is, t

Re: [HACKERS] [PATCH] make -jN check fails / unset MAKEFLAGS in pg_regress.c

2012-11-30 Thread Robert Haas
On Thu, Nov 29, 2012 at 8:50 AM, Andres Freund wrote: > Hi, > > Currently "make -jN check" fails during "creating temporary installation" > with: > make[1]: *** read jobs pipe: Invalid argument. Stop. > make[1]: *** Waiting for unfinished jobs > make[2]: warning: jobserver unavailable: using

Re: [HACKERS] initdb.c::main() too large

2012-11-30 Thread Bruce Momjian
On Thu, Nov 29, 2012 at 11:23:59PM -0500, Tom Lane wrote: > Bruce Momjian writes: > > In looking to add an fsync-only option to initdb, I found its main() > > function to be 743 lines long, and very hard to understand. > > > The attached patch moves much of that code into separate functions, > >

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-30 Thread Tom Lane
Robert Haas writes: > On Wed, Nov 28, 2012 at 9:47 AM, Amit kapila wrote: >> 5. PERSISTENT Keyword is added to the reserved keyword list. As it was >> giving some errors given below while parsing gram.y >> 15 shift/reduce conflicts . > Allow me to be the first to say that any syntax for this fe

Re: [HACKERS] Enabling frontend-only xlog "desc" routines

2012-11-30 Thread Robert Haas
On Thu, Nov 29, 2012 at 3:03 PM, Tom Lane wrote: > Alvaro Herrera writes: >> The other interesting question remaining is what to do about the rm_desc >> function in rmgr.c. We're split between these two ideas: > > Why try to link rmgr.c into frontend versions at all? Just make > a new table fil

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Josh Berkus
> In the workloads where it might not be a good idea (very long queries on > the standby, many dead tuples on the primary) you need to think very > carefuly about the strategy of avoiding conflicts anyway, and explicit > configuration is required as well. > > Does anybody have an argument against

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Heikki Linnakangas
On 30.11.2012 21:02, Andres Freund wrote: Hi, The subject says it all. There are workloads where its detrimental, but in general having it default to on improver experience tremendously because getting conflicts because of vacuum is rather confusing. In the workloads where it might not be a go

Re: [HACKERS] Re: missing LockBuffer(buffer, BUFFER_LOCK_SHARE) in trigger.c GetTupleForTrigger?

2012-11-30 Thread Tom Lane
BTW, on further inspection, there's yet another reason why we've not heard about this from the field: even if all the wrong things happen and GetTupleForTrigger manages to copy bogus data for the old tuple, that data *is not passed to the trigger function*. The only thing we do with it is decide w

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Claudio Freire
On Fri, Nov 30, 2012 at 5:46 PM, Heikki Linnakangas wrote: > > Think of someone setting up a test server, by setting it up as a standby > from the master. Now, when someone holds a transaction open in the test > server, you get bloat in the master. Or if you set up a standby for > reporting purpos

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Tom Lane
Robert Haas writes: > While we're talking about changing defaults, how about changing the > default value of the recovery.conf parameter 'standby_mode' to on? > Not sure about anybody else, but I never want it any other way. Dunno, it's been only a couple of days since there was a thread about so

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Heikki Linnakangas
On 30.11.2012 22:49, Claudio Freire wrote: On Fri, Nov 30, 2012 at 5:46 PM, Heikki Linnakangas wrote: Think of someone setting up a test server, by setting it up as a standby from the master. Now, when someone holds a transaction open in the test server, you get bloat in the master. Or if you

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Kevin Grittner
Claudio Freire wrote: > Without hot standby feedback, reporting queries are impossible. > I've experienced it. Cancellations make it impossible to finish > any decently complex reporting query. With what setting of max_standby_streaming_delay? I would rather default that to -1 than default hot_st

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Claudio Freire
On Fri, Nov 30, 2012 at 6:06 PM, Kevin Grittner wrote: > >> Without hot standby feedback, reporting queries are impossible. >> I've experienced it. Cancellations make it impossible to finish >> any decently complex reporting query. > > With what setting of max_standby_streaming_delay? I would rath

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Tom Lane
Claudio Freire writes: > Without hot standby feedback, reporting queries are impossible. I've > experienced it. Cancellations make it impossible to finish any > decently complex reporting query. The original expectation was that slave-side cancels would be infrequent. Maybe there's some fixing/t

Re: [HACKERS] Removing PD_ALL_VISIBLE

2012-11-30 Thread Jeff Davis
On Mon, 2012-11-26 at 16:55 -0600, Merlin Moncure wrote: > > Based on previous measurements, I think there *is* contention pinning > > the root of an index. Currently, I believe it's largely overwhelmed > > by contention from other sources, such as the buffer manager lwlocks > > and the very-evil

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Kevin Grittner
Claudio Freire wrote: >> With what setting of max_standby_streaming_delay? I would rather >> default that to -1 than default hot_standby_feedback on. That >> way what you do on the standby only affects the standby. > > 1d Was there actually a transaction hanging open for an entire day on the sta

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-30 Thread Bruce Momjian
On Thu, Nov 29, 2012 at 12:59:19PM -0500, Bruce Momjian wrote: > I have polished up the patch (attached) and it is ready for application > to 9.3. Applied. --- > Since there is no pg_dump/pg_restore pipe parallelism, I had t

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Claudio Freire
On Fri, Nov 30, 2012 at 6:20 PM, Kevin Grittner wrote: > Claudio Freire wrote: > >>> With what setting of max_standby_streaming_delay? I would rather >>> default that to -1 than default hot_standby_feedback on. That >>> way what you do on the standby only affects the standby. >> >> 1d > > Was ther

Re: [HACKERS] initdb.c::main() too large

2012-11-30 Thread Bruce Momjian
On Thu, Nov 29, 2012 at 11:12:23PM -0500, Bruce Momjian wrote: > In looking to add an fsync-only option to initdb, I found its main() > function to be 743 lines long, and very hard to understand. > > The attached patch moves much of that code into separate functions, > which will make initdb.c eas

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Heikki Linnakangas
On 30.11.2012 23:40, Claudio Freire wrote: On Fri, Nov 30, 2012 at 6:20 PM, Kevin Grittner wrote: Claudio Freire wrote: With what setting of max_standby_streaming_delay? I would rather default that to -1 than default hot_standby_feedback on. That way what you do on the standby only affects th

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Claudio Freire
On Fri, Nov 30, 2012 at 6:49 PM, Heikki Linnakangas wrote: >>> I have most certainly managed databases where holding up vacuuming >>> on the source would cripple performance to the point that users >>> would have demanded that any other process causing it must be >>> immediately canceled. And canc

Re: [HACKERS] initdb.c::main() too large

2012-11-30 Thread Andrew Dunstan
On 11/30/2012 04:45 PM, Bruce Momjian wrote: On Thu, Nov 29, 2012 at 11:12:23PM -0500, Bruce Momjian wrote: In looking to add an fsync-only option to initdb, I found its main() function to be 743 lines long, and very hard to understand. The attached patch moves much of that code into separate

Re: [HACKERS] initdb.c::main() too large

2012-11-30 Thread Bruce Momjian
On Fri, Nov 30, 2012 at 06:06:39PM -0500, Andrew Dunstan wrote: > > On 11/30/2012 04:45 PM, Bruce Momjian wrote: > >On Thu, Nov 29, 2012 at 11:12:23PM -0500, Bruce Momjian wrote: > >>In looking to add an fsync-only option to initdb, I found its main() > >>function to be 743 lines long, and very ha

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Andres Freund
On 2012-11-30 14:35:37 -0500, Robert Haas wrote: > On Fri, Nov 30, 2012 at 2:02 PM, Andres Freund wrote: > > Does anybody have an argument against changing the default value? > > Well, the disadvantage of it is that the standby can bloat the master, > which might be surprising to some people, too.

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Daniel Farina
On Fri, Nov 30, 2012 at 11:35 AM, Robert Haas wrote: > On Fri, Nov 30, 2012 at 2:02 PM, Andres Freund wrote: >> Does anybody have an argument against changing the default value? > > Well, the disadvantage of it is that the standby can bloat the master, > which might be surprising to some people,

Re: [HACKERS] [PATCH] make -jN check fails / unset MAKEFLAGS in pg_regress.c

2012-11-30 Thread Andres Freund
On 2012-11-30 14:41:14 -0500, Robert Haas wrote: > On Thu, Nov 29, 2012 at 8:50 AM, Andres Freund wrote: > > Hi, > > > > Currently "make -jN check" fails during "creating temporary installation" > > with: > > make[1]: *** read jobs pipe: Invalid argument. Stop. > > make[1]: *** Waiting for unfini

Re: [HACKERS] [PATCH] make -jN check fails / unset MAKEFLAGS in pg_regress.c

2012-11-30 Thread Andres Freund
> Yes. obviously. Stupid, errr, fingers. This time round it really was fatfingering the wrong key after having been climbing for 4h. Really. Trivially updated patch attached. -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Serv

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Magnus Hagander
On Fri, Nov 30, 2012 at 9:46 PM, Heikki Linnakangas wrote: > On 30.11.2012 21:02, Andres Freund wrote: >> >> Hi, >> >> The subject says it all. >> >> There are workloads where its detrimental, but in general having it >> default to on improver experience tremendously because getting conflicts >> b

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Magnus Hagander
On Fri, Nov 30, 2012 at 10:09 PM, Tom Lane wrote: > Claudio Freire writes: >> Without hot standby feedback, reporting queries are impossible. I've >> experienced it. Cancellations make it impossible to finish any >> decently complex reporting query. > > The original expectation was that slave-sid

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Andres Freund
On 2012-11-30 22:46:06 +0200, Heikki Linnakangas wrote: > On 30.11.2012 21:02, Andres Freund wrote: > >Hi, > > > >The subject says it all. > > > >There are workloads where its detrimental, but in general having it > >default to on improver experience tremendously because getting conflicts > >becaus

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Josh Berkus
All: Well, the problem is that we have three configurations which only work for one very common scenario: - reporting slave: feedback off, very long replication_delay - load-balancing slave: feedback on, short replication_delay - backup/failover slave: feedback off, short replication_delay I don

Re: [HACKERS] Hot Standby Feedback should default to on in 9.3+

2012-11-30 Thread Andres Freund
On 2012-11-30 16:09:15 -0500, Tom Lane wrote: > Claudio Freire writes: > > Without hot standby feedback, reporting queries are impossible. I've > > experienced it. Cancellations make it impossible to finish any > > decently complex reporting query. > > The original expectation was that slave-side

Re: Use of fsync; was Re: [HACKERS] Pg_upgrade speed for many tables

2012-11-30 Thread Bruce Momjian
On Mon, Nov 26, 2012 at 02:43:19PM -0500, Bruce Momjian wrote: > > >> In any event, I think the documentation should caution that the > > >> upgrade should not be deemed to be a success until after a system-wide > > >> sync has been done. Even if we use the link rather than copy method, > > >> are

[HACKERS] --single-transaction hack to pg_upgrade does not work

2012-11-30 Thread Tom Lane
Some of the buildfarm members are failing the pg_upgrade regression test since commit 12ee6ec71f8754ff3573711032b9b4d5a764ba84. I can duplicate it here, and the symptom is: pg_restore: creating TYPE float8range pg_restore: creating TYPE insenum pg_restore: [archiver (db)] Error while PROCESSING T